Psychometrics and validation of health scales and questionnaires
Validate the instrument and analyse what it measures: reliability, clinical scales and patient-reported outcomes
What it is
Psychometric validation of questionnaires and statistical analysis of scales and patient-reported outcomes (PROs) in health studies: anxiety, depression, quality of life, caregiver burden and other constructs measured with validated instruments.
It covers both sides of the problem: checking that the instrument measures reliably, and analysing rigorously what that instrument captures.
Objectives
- Assess the reliability and internal consistency of a questionnaire before drawing conclusions from it.
- Analyse scales and patient-reported outcomes as outcome variables in pre-post and comparative studies.
- Relate scale results to clinical and biological variables when the study allows it.
- Handle scoring, dimensions and questionnaire-specific missing data with sound judgement.
Instrument validation
- Reliability and internal consistency using Cronbach's alpha (raw and standardised).
- Item refinement by examining whether alpha improves when each question is deleted.
- Checking the consistency of the questionnaire by stratifying by subgroup.
Scales as outcomes
- Pre-post comparison of clinical scales (anxiety, depression, quality of life).
- Mixed-effects models for repeated measures in longitudinal follow-up.
- Analysis by demographic subgroup and with a gender perspective.
Scales and biology
- Association of patient-reported outcomes with clinical and inflammatory biomarkers.
- Mediation analysis between biological and psychological variables.
- Reading psychological data and biological markers together.
Questionnaire design and handling
- Scoring structure, aggregation of dimensions and handling of questionnaire-specific missing data.
- Assessment of Likert-type scales and other ordinal responses with methods suited to their nature.
- Preparation of scale data for analysis and publication.
What it is for
- Make sure a questionnaire measures reliably before basing clinical or research conclusions on its results.
- Analyse the effect of an intervention on anxiety, depression or quality-of-life scales with appropriate longitudinal methods.
- Connect what patients feel or perceive with what their biological markers show, when the study collects both.
A scale may not measure what you think it does, and that turns your results into statistical noise
A poorly validated scale produces results that look solid but are not: if the instrument is not reliable, it does not matter how elegant the subsequent analysis is. Validating the questionnaire first and then analysing it with the right method are two steps, and both are needed.
Validate the scale before you trust it
Before analysing anything, we assess the reliability of the questionnaire: internal consistency with Cronbach's alpha, the behaviour of each item and the stability of the scale across subgroups. If an item takes away more than it adds, this is documented. That way, the conclusions rest on an instrument that has been checked, not assumed.
Longitudinal analysis of what the scale measures
Clinical scales are usually measured before and after an intervention, in the same patients. We analyse them with mixed-effects models for repeated measures that respect that structure, instead of simple comparisons that ignore within-subject correlation, and we break the results down by subgroup when the study requires it.
What patients feel and what biology shows
This is what sets us apart. When a study measures psychological scales and biological markers at the same time, we analyse the relationship between the two: for example, symptoms of anxiety or depression and inflammatory markers. It is a field in which the team has its own scientific track record, not an improvised application.
A questionnaire has its own statistics
A scale is not just another variable: it has its own dimensions, scoring rules and patterns of non-response. We handle scoring, the aggregation of dimensions and missing data with the judgement the instrument demands, so that what goes into the analysis is the right measure.
What working with us looks like
We apply our D²I (Data to Insights) methodology to every service. A structured, iterative and transparent process that turns complex data into actionable knowledge.
01
Kick-off and problem definition
We meet with you to understand the clinical problem, the data available and the objectives of the project. We agree on deliverables, timelines and communication channels, and we sign whatever confidentiality agreements you need to protect your data.
02
Data review and preparation
We assess the quality of your data, identify inconsistencies and limitations, and structure the data for analysis. If critical information is missing, we tell you before going any further. We document every cleaning decision so that the process is traceable and reproducible. In this service, this includes assessing the reliability of the questionnaire, reviewing how the items behave, resolving the scoring and dimensions of the scale, and handling missing data before any substantive analysis.
03
Analysis and modelling
We apply the statistical methodology that fits your questions. We work iteratively: we share preliminary results with you, discuss the findings and adjust the approach where needed. We do not work in the dark until the final delivery. For this service, we analyse the scales as outcomes with models suited to their longitudinal structure, break the results down by subgroup and, when the study allows it, relate patient-reported outcomes to biological markers.
04
Communicating results
We turn the findings into the format you need: a technical report, tables and figures for publication, an interactive dashboard or an executive presentation. We always include the clinical interpretation, not just the figures.
05
Impact review session
30 days after delivery we meet with you again for a 60-minute strategy session. By then you have had time to present the results internally, the real implementation questions have come up and you can assess the impact of the work with some perspective. We answer your questions, adjust interpretations where necessary and agree together on the way forward.
This is our general process. If you want to understand how we apply it technically to each type of analysis, see our full D²I methodology.
Have a data project on your hands?
Tell us about your case and we will work out together how to approach it.
Book a sessionWhere we have applied it
Frequently asked questions
Why validate the questionnaire before analysing its results?
Because a poorly validated scale produces results that look solid but are not: if the instrument does not show sufficient reliability in the sample analysed, it does not matter how elegant the subsequent analysis is. This applies to any study that relies on a questionnaire, such as those for anxiety, depression, quality of life or caregiver burden, as a source for its conclusions. The service covers both sides of the problem, which are two distinct steps and both necessary: checking that the instrument measures consistently, and analysing rigorously what that instrument captures. Validating reliability before the analysis avoids basing clinical or research conclusions on a scale that, unnoticed, is adding noise instead of signal. A scale may not measure what you think it does, and that turns an impeccable analysis into an unfounded conclusion. The order matters: first the reliability of the instrument is reviewed in those data, then what it measures is analysed.
How is the reliability of a scale assessed? Is Cronbach's alpha enough?
Reliability is assessed through internal consistency, with Cronbach's alpha in its raw and standardised forms as the starting point, complemented by an analysis of how each item behaves and of the scale's stability across subgroups. It applies before a questionnaire is used as a study variable, so that a scale is not accepted without being checked. Cronbach's alpha is a central piece, but not the only one. We examine whether alpha improves when each item is deleted, which identifies items worth reviewing before the scale is interpreted, and we explore internal consistency across relevant subgroups, because a scale can behave differently depending on the profile. One caveat: alpha measures internal consistency; on its own it does not ensure that the scale captures the construct it is meant to, and it is interpreted alongside the rest of the evidence, not as an automatic seal of approval. What is documented is an instrument with its reliability recorded and its items reviewed, not a questionnaire assumed to be valid without further checks.
Do you also analyse change in scales after an intervention, not just their validity?
Yes: the service analyses scales as outcome variables; it does not stop at validating them. This applies when a clinical scale, such as those for anxiety, depression or quality of life, is measured before and after an intervention in the same patients, or compared between groups. These repeated measures are analysed with mixed-effects models that respect the longitudinal structure and the within-subject correlation, instead of simple comparisons that ignore it and underestimate uncertainty. When the study requires it, the results are disaggregated by demographic subgroup and with a gender perspective, because the change observed in a scale may not be homogeneous. Validating the instrument and analysing what it measures are the two sides of the service: one ensures that the scale is reliable; the other estimates rigorously how it changes. What you receive is an analysis of the patient-reported outcome with the method its structure demands, not a comparison that ignores how it was collected.
My study measures psychological scales and also biomarkers. Can you relate the two?
Yes, and it is one of the distinctive strengths of the service: when a study collects psychological scales and biological markers at the same time, we analyse the relationship between them. It applies to studies that measure, for example, symptoms of anxiety or depression alongside inflammatory markers, and want to read the psychological and biological data together rather than treat them as separate analyses. We analyse the association between patient-reported outcomes and clinical and inflammatory biomarkers and, when temporal order, design and assumptions allow it, a mediation analysis between biological and psychological variables can be considered. This intersection of psychometrics, biochemistry and immunology, and biostatistics rests on the team's own scientific track record in the field, not on an improvised application. An honest limit applies: relating scales and biology in an observational design estimates associations and, where appropriate, mediation pathways under explicit assumptions; on its own, it does not demonstrate biological causality. What you receive is a joint, rigorous reading of what patients perceive and what their markers show.
My data are Likert scales and some items were left unanswered. Do they need special handling?
Yes: a scale is not just another variable. It has its own dimensions, scoring rules and patterns of non-response, which are handled with specific criteria. This applies to any questionnaire with a scale structure, where treating the items simply as numbers can distort the result. Likert-type and other ordinal responses are assessed in a way that respects their ordinal nature when the analysis works at item level, with a justification of when an aggregate score can be treated as an interval scale. Scoring, the aggregation of dimensions and missing data are handled according to the rules of the instrument: a missing value on an item of a subscale is not treated like any other missing value, because it affects how the dimension score is built. Preparing the scale data properly before the analysis is what ensures that what goes into the model is the right measure. Treating the questionnaire with its own statistics prevents generic handling from introducing an error at the root.
Do you adapt or cross-culturally validate a new instrument (measurement invariance, validated translation)?
No: the service validates the reliability and consistency of a questionnaire and analyses what it measures, but it does not cover building new instruments or their formal cross-cultural adaptation. This limit needs stating because "validating a scale" can be understood in two different ways. What is included: reliability and internal consistency (Cronbach's alpha), analysis of item behaviour, stability across subgroups and the statistical analysis of the scale as an outcome. What is not included: formal cross-cultural adaptation with validated forward and back translation, measurement invariance analyses between groups, or item response theory (IRT) techniques for building or calibrating instruments. That is a line of psychometrics concerned with construction and adaptation, with its own framework, and it lies outside the scope of this service. The boundary is clear: checking the reliability of an instrument and analysing its results, yes; building or formally adapting the scale, no. If you need that line of work, you will find an explicit limit here, not a vague promise.
What do I need to provide to validate a scale, and what do you do?
You need the data from the scales already collected, that is, the responses to the questionnaire items, together with the clinical, demographic or biological variables of the study that you want to relate to them. The client administers the questionnaire and collects the scales; the service validates reliability, analyses the scales as outcomes and relates them to the other variables when the study allows it. It helps to provide the structure of the instrument, with its dimensions and scoring rules, so that it can be handled correctly, although resolving the scoring and dimensions is partly our work too. The data do not need to arrive clean: the questionnaire's own scoring, dimensions and missing data are handled as part of the work before the analysis. What you receive is the validation and the analysis; administering the scale to participants and deciding the study design belong to the research team, who sign and lead the publication.
What do you deliver at the end of a scale validation?
You receive the instrument validation and the scale analysis, interpreted and documented: the assessment of reliability and consistency with the items reviewed; the analysis of the scales as outcomes with the method suited to their structure, using mixed-effects models for repeated measures and breakdown by subgroup; and, where the study collects both, the relationship between patient-reported outcomes and biological markers. It applies if you need your scale results to reach publication or a clinical decision with methodological backing. The material is prepared for integration into the manuscript, with the rationale for the validation and analysis decisions documented. The research team collects the scales and signs the work; the service provides the validation, the analysis and their interpretation. What you receive is not software output run on a questionnaire, but documented reliability of the instrument in your sample and an analysis of what it measures that stands up to review.
Shall we talk?
Tell us what you are working on and we will give you a straight answer on how we can help.