- Home
- /
- By your needs
- /
- Find out what works
Did what you did work? And what is worth changing?
Evaluating is not justifying what has already been decided. It means estimating methodically whether an intervention had the effect it was aiming for, anticipating what would happen with a different decision and finding where the room for improvement lies.
Every intervention, programme or process starts from a hypothesis: that it will improve something. Estimating whether it contributed to the change, and to what extent, requires building a valid comparison and controlling for alternative explanations as far as the design and the data allow. Without that work, it is easy to attribute to an intervention a change that would have happened anyway, or to overlook an effect that was masked by other factors.
It is the statistical work that turns "we think it works" into "this is what the data show, with this degree of uncertainty". It covers four strands, depending on the decision in front of you: estimating the real impact of an intervention, anticipating scenarios before deciding, comparing value and prioritising resources, and improving how services and flows work. You can start with whichever applies to you.
This is for you if…
You launched an intervention or a programme and need to know whether it had the effect you were aiming for, separating its impact from what would have happened anyway.
You are weighing up a decision (a policy, an investment, a change) and want to anticipate its effects with a model before committing, not after.
You have to choose between several options with limited resources and need to compare their value on quantitative criteria, not just on intuition or the budget available.
You suspect that a service, a patient flow or a process could work better, and you want to locate with data where the room for improvement lies.
How we can help
Estimate the real impact
Estimate the effect of an intervention that has already taken place, separating it from what would have happened without it.
Estimating the effect of an intervention after it has been implemented: how much the outcome changed and what part of that change can reasonably be attributed to the intervention, under the assumptions of the design and the analysis. The statistical work that distinguishes attributable change from change that would have happened anyway.
View service →When the question is whether A caused B and not just whether they go together, causal inference provides the framework for estimating effects from observational data: control of confounders, matching, weighting, instrumental variables. Estimating a causal effect outside a randomised trial, with the assumptions always on the table.
View service →Breaking an effect down into its pathways (what part passes through an intermediate mechanism) and studying in which subgroups it is larger or smaller. Studying which mechanisms are compatible with the observed effect and how it may vary across contexts or subgroups.
View service →Generating evidence on outcomes, utilisation and effectiveness in routine practice from registries, clinical records or observational cohorts, rather than from the controlled conditions of a trial. Useful when you want to know what happens in routine practice, with the methodological rigour that non-experimental data require.
View service →The statistical planning and analysis of studies that evaluate a nutrition intervention: structuring the comparison, defining the variables, controlling for confounders and estimating the effect. The statistical work of the study; the intervention is defined and delivered by the research team.
View service →Anticipate scenarios and adoption
Model what would happen before deciding, instead of waiting to find out.
Estimating the likely effect of an intervention before it is launched, combining available data and explicit assumptions in a model. Anticipating the order of magnitude of an effect to inform a decision, with the model's uncertainty stated, not hidden.
View service →Exploring what would happen under different decisions without having to test them in reality: building a model of the system and running it under alternative scenarios. Comparing policy options by their simulated effects, with the assumptions of each scenario made explicit.
View service →Studying how the adoption of an intervention or product is distributed across a population and which profiles drive it or hold it back: segmentation, scoring, adoption curves. Characterising who adopts, at what pace and which factors are associated with adoption, to gauge the real reach of an intervention.
View service →Compare value and prioritise resources
Weigh cost against outcome to decide where to allocate limited resources.
Comparing the value of different options by relating their costs to their outcomes: cost-effectiveness, cost-utility and cost-benefit analysis. It provides the quantitative framework for prioritising; cost and effect figures are estimated from the available data and literature, not promised in advance.
View service →Quantifying the burden of a disease or risk factor on a population by combining mortality and morbidity into comparable indicators. A summary measure for gauging the magnitude of a health problem and prioritising between problems that compete for the same resources.
View service →Improve services, flows and efficiency
Locate with data where a service or a process can work better.
Comparing the performance of units, centres or departments with methods that account for the differences in their starting points: case-mix adjustment, standardisation, detection of deviations. Making units with different patient profiles and starting conditions more comparable, so that each one can be placed relative to the others.
View service →Analysing how patients move through a department and where bottlenecks build up: length of stay, transitions between states, use of capacity. The quantitative basis for understanding why a flow becomes congested and where it is worth studying or prioritising possible improvements.
View service →Studying the relationship between the resources a service consumes and the outcomes it produces. Identifying differences in the relationship between resources and outcomes, and locating areas where there may be room for efficiency gains, on the basis of the operational data available.
View service →Not sure which of these fits your situation?
Tell us what stage you are at and we will tell you what work is needed, or whether none is. No commitment, and to the point.
Let's talk about what you want to evaluateHow we have approached it in other projects
Before you take the step
How do I know whether I need to estimate the impact of something already done or anticipate the impact of something I'm considering?
The distinction lies in the timing of the decision: if the intervention has already happened and you want to know what effect it had, it is impact evaluation; if you have not launched it yet and want to anticipate what would happen, it is modelling before you decide. Recognising which of the two you are in is the first piece of orientation: estimating the real impact looks back, on data already collected; anticipating scenarios looks forward, with a model and explicit assumptions. You don't need to know which method applies to your case; it is enough to know whether the decision in front of you concerns something you have already done or something you are still weighing up. Comparing the value of several options and improving how a service works also belong here.
Can the impact of an intervention be evaluated if there was no control group?
Yes, although the absence of a control group shapes the method and the strength of the conclusions; it does not prevent evaluation. It is a common situation with real-world interventions, which are often rolled out without an experimental design because random allocation was not possible or not ethical. To estimate the effect outside a randomised trial, the causal inference framework is applied to observational data: building a valid comparison, controlling for confounders and making explicit the assumptions under which the estimate holds. What we avoid is attributing to the intervention a change that would have happened anyway because of the underlying trend. The honest limit: without randomisation, causal attribution rests on assumptions that are stated and examined, not on the certainty of an experiment.
I have to choose between several options with limited resources. What do I need to compare them?
To compare options on quantitative criteria, what you need above all is to define which outcome matters and what each alternative costs; from there, value is weighed against resources. This applies when you have to prioritise between interventions, investments or changes that compete for the same budget, and intuition or the spending available is not enough. The comparison is approached in two ways, depending on the question: economic evaluation, which relates the costs and outcomes of different options, and burden of disease, which gauges the weight of problems that compete for the same resources. Cost and effect figures are estimated from the available data and literature; they are not promised in advance. What is delivered is the quantitative framework for prioritising on solid grounds, not a closed recommendation of what to choose.
I suspect a service or a process could work better, but I don't know where the room for improvement is. Where do you start?
You start by locating with data where the inefficiency or the delay is concentrated, which does not always coincide with where it is perceived. This arises when a service, a patient flow or a process shows signs that it could work better and you want to act on the point that really limits it, not on the most visible one. Improvement is approached from several angles depending on the problem: comparing the performance of units while accounting for their different starting points, analysing how patients move and where bottlenecks form, or studying the relationship between the resources a process consumes and the outcomes it produces. Adding resources at a point that is not the limiting one uses up budget without solving anything. Redesign and implementation remain with the team that knows the operational context.
What I want to evaluate is a nutrition intervention. Does it fit here, or do I need a specific nutrition service?
It fits here, and it has its own service within this path: analysis of nutrition intervention studies, under “Estimate the real impact”. The statistical work of evaluating a nutrition intervention is the same kind of work as evaluating any other intervention. This arises when you need to estimate whether a food or nutrition intervention had (or would have) the intended effect on a population or clinical indicator. The division of roles is clear: the research team defines and delivers the intervention; the statistical work structures the comparison, controls for confounders and estimates the effect. The fact that the domain is nutrition does not change the logic of the evaluation, only the nature of the indicator. It can come in either through estimating the impact of an intervention already carried out or through anticipating the impact of one under consideration. What is evaluated is the intervention, with statistical methods, not the design of the nutrition programme itself, which belongs to whoever puts it in place.
Can an evaluation also tell me why something didn't work, not just whether it worked?
It can get closer to the why, within what the data and the design allow, although explaining the mechanism is a more demanding question than estimating the effect. This arises when knowing whether an intervention had an impact is not enough and you want to understand through which pathways it operated or in which subgroups it worked better or worse. For this there are approaches that break an effect down into the pathways it may pass through and study how it varies across contexts or subgroups, always under explicit assumptions. What we avoid is presenting a mechanistic explanation as proven when the data only allow it to be suggested. The honest limit: identifying why something did not work from observational data produces hypotheses compatible with what was observed, not a proven cause, which is useful for deciding whether to adjust the intervention or where to look next.
An evaluation produces results that almost always need to be communicated: to a funder, to a committee, to a management team. When you have the result and need others to understand it and use it to make decisions, the natural next step is to communicate results and build tools they can work with. And if you are still defining what you want to evaluate and with what data, let's talk first.
Shall we talk?
Tell us what you are working on and we will give you a straight answer on how we can help.