Causal inference and observational study design
Design and analysis with a causal approach, to answer questions about the effects of interventions, exposures and policies when randomisation is not possible
What it is
A methodological framework for estimating causal effects from observational data. It combines study design, the formalisation of causal assumptions in DAGs and statistical techniques that control confounding explicitly.
It includes propensity score methods, g-computation, instrumental variables, difference-in-differences and causal mediation analysis. It applies when the question is "what effect does X have on Y?" and no randomised trial is available.
Objectives
- Estimate the causal effect of an intervention, exposure or policy on an outcome of interest from observational data, explicitly controlling measured confounding and assessing sensitivity to unmeasured confounding.
- Formalise the study's causal assumptions in DAGs before running any model, identifying confounders, mediators and colliders.
- Decompose causal effects into direct and indirect effects (mediation) and assess whether the effect varies across subpopulations (moderation).
DAGs and the formalisation of causal assumptions
- Building directed acyclic graphs (DAGs) to formalise the causal structure of the problem before the analysis.
- Identifying the minimal adjustment set: which variables to control for and which to leave alone (mediators, colliders).
- Assessing whether the causal question is identifiable with the available data, and being transparent when it is not.
Propensity score methods and group balance
- Propensity score matching (nearest neighbour, caliper, optimal) with post-matching balance assessment.
- Inverse probability of treatment weighting (IPTW) with weight stabilisation and truncation.
- Propensity score stratification and doubly robust estimation to protect against model misspecification.
Causal mediation and effect decomposition
- Estimation of the ACME (average causal mediation effect) and ADE (average direct effect) with bootstrap, for continuous and binary outcomes.
- Moderation analysis with interaction terms and visualisation of conditional effects.
- Sensitivity analysis for violation of the sequential ignorability assumption.
Advanced methods and policy evaluation
- G-computation to estimate marginal population effects.
- Difference-in-differences to evaluate the impact of interventions or policies with pre/post data.
- Quantitative sensitivity analysis for unmeasured confounding (E-value, bias analysis) to assess the robustness of causal conclusions.
What it is for
- Estimate the effect of a health intervention when no clinical trial is available, using data from registries, cohorts or administrative databases with an explicit causal design.
- Evaluate the impact of a public health policy or a protocol change by comparing pre- and post-implementation periods while controlling for pre-existing trends.
- Respond to reviewers who demand causal evidence, not just adjusted associations, with DAGs, sensitivity analyses and a formal discussion of the identification assumptions.
Adjusting for confounders is not causal inference. It is the first step, often mistaken for the last
A multivariable regression adjusted for age and sex does not answer causal questions — it answers questions of conditional association. If you want to know whether your intervention causes an effect, you need a design with explicit causal assumptions, an adjustment set derived from the causal structure of the problem (not from statistical significance), and sensitivity analyses that quantify how much unmeasured confounding it would take to invalidate your conclusions.
The DAG comes before the model, not after
We build the causal graph with you before deciding which variables to adjust for. If your DAG shows that a variable is a collider, adjusting for it introduces bias instead of removing it. If it shows that it is a mediator, adjusting for it removes part of the effect you want to estimate. The causal structure determines the model, not the other way round.
Propensity scores with balance diagnostics, not just p-values
A propensity score that shrinks the p-value of the differences between groups does not ensure good balance. We assess balance with standardised differences, variance ratios and visual comparison of distributions. If matching or weighting does not achieve acceptable balance, we report it and assess alternatives — we don't force a result.
Sensitivity analyses that quantify the threat instead of ignoring it
Every observational causal estimate rests on assumptions that cannot be verified. Rather than assuming they hold and moving on to the next section of the manuscript, we provide E-values or quantitative bias analysis that state exactly how much unmeasured confounding would be needed to nullify the estimated effect. That turns an implicit assumption into a quantifiable argument.
Impact evaluation of policies and population interventions
Difference-in-differences and regression discontinuity designs make it possible to evaluate the impact of protocol changes, new health policies or population interventions by exploiting temporal or administrative variation. When there is no randomised control group, the study design becomes the analysis — and good design matters more than sophisticated modelling.
What working with us looks like
We apply our D²I (Data to Insights) methodology to every service. A structured, iterative and transparent process that turns complex data into actionable knowledge.
01
Kick-off and problem definition
We meet with you to understand the clinical problem, the data available and the objectives of the project. We agree on deliverables, timelines and communication channels, and we sign whatever confidentiality agreements you need to protect your data.
02
Data review and preparation
We assess the quality of your data, identify inconsistencies and limitations, and structure the data for analysis. If critical information is missing, we tell you before going any further. We document every cleaning decision so that the process is traceable and reproducible. In causal inference, this includes building the DAG with the research team, identifying the minimal adjustment set, assessing whether the available data allow the causal effect of interest to be identified, and preparing the variables for the chosen estimation strategy (matching, weighting, g-computation).
03
Analysis and modelling
We apply the statistical methodology that fits your questions. We work iteratively: we share preliminary results with you, discuss the findings and adjust the approach where needed. We do not work in the dark until the final delivery. For this service, we implement the chosen causal strategy (propensity score matching or weighting, causal mediation, difference-in-differences, g-computation), assess balance and assumptions, run sensitivity analyses for unmeasured confounding, and document explicitly which causal assumptions support each conclusion. All in R, with reproducible code.
04
Communicating results
We turn the findings into the format you need: a technical report, tables and figures for publication, an interactive dashboard or an executive presentation. We always include the clinical interpretation, not just the figures.
05
Impact review session
30 days after delivery we meet with you again for a 60-minute strategy session. By then you have had time to present the results internally, the real implementation questions have come up and you can assess the impact of the work with some perspective. We answer your questions, adjust interpretations where necessary and agree together on the way forward.
This is our general process. If you want to understand how we apply it technically to each type of analysis, see our full D²I methodology.
Have a data project on your hands?
Tell us about your case and we will work out together how to approach it.
Book a sessionFrequently asked questions
I already adjust my regression for age, sex and other confounders. Isn't that causal inference?
No. Adjusting for confounders is a first step that is often mistaken for the last, but an adjusted regression answers a question of conditional association, not a causal one. This matters when the real question is "what effect does X have on Y?" and not just "are X and Y associated after adjustment?". The difference is fundamental. A multivariable regression adjusted for age and sex estimates a conditional association. Estimating a causal effect requires a design with explicit causal assumptions, an adjustment set derived from the causal structure of the problem rather than from statistical significance, and a sensitivity analysis that assesses how far the conclusion would depend on possible unmeasured confounding. Choosing which variables to adjust for by looking at p-values, without a causal framework, can introduce bias instead of removing it. What sets causal inference apart is not a more sophisticated model, but making explicit and defensible the assumptions under which an association can be read as an effect.
What is a DAG and why is it built before the model?
A DAG (directed acyclic graph) is the formal representation of the causal structure of the problem: what causes what, according to domain knowledge, drawn before any model is run. It applies to any study with a causal aim, because the causal structure determines which variables to adjust for, not the other way round. The DAG identifies the minimal adjustment set, that is, which variables to control for and, above all, which to leave alone: adjusting for a mediator removes part of the effect you want to estimate, and adjusting for a collider introduces bias where there was none. It also shows whether the causal question is identifiable at all with the available data, so that you can be transparent when it is not. Building the DAG with the research team, who bring the domain knowledge, is what anchors the whole analysis that follows. Deciding the adjustment without a DAG means choosing blindly which biases are corrected and which are created.
How can groups be compared without the randomisation of a trial? What are propensity scores?
Without randomisation, the groups differ in characteristics that affect the outcome, and propensity score methods aim to balance those differences so that the comparison is more defensible. They apply when you want to estimate the effect of an intervention or exposure with observational data from registries, cohorts or administrative databases. The propensity score, the probability of receiving the treatment given the covariates, is used to match, to weight (IPTW with stabilisation and truncation) or to stratify. It can be combined with doubly robust estimation, which improves protection against misspecification if at least one of the models involved is correctly specified. The key lies in the balance assessment: a propensity score is validated by checking that the covariates end up balanced between groups, using standardised differences and comparison of distributions, not a p-value. If matching or weighting does not achieve acceptable balance, this is reported and alternatives are assessed; a result is not forced. Balancing the observed groups reduces bias, but it does not replace randomisation or remove unmeasured confounding.
If a study is observational, can causality not be proven? How reliable is the conclusion?
Every causal estimate from observational data rests on assumptions that cannot be verified with the data themselves, so the job is not to "prove causality" but to make those assumptions explicit and quantify how sensitive the conclusion is to their violation. This applies to any observational study with a causal question, where honesty about the assumptions is part of the rigour. Instead of taking the assumptions for granted and moving on, we run a quantitative sensitivity analysis, such as the E-value or quantitative bias analysis. It estimates, under explicit assumptions, what magnitude of unmeasured confounding would be compatible with explaining away or weakening the estimated effect. That turns an implicit assumption into a quantifiable argument: an effect that only weakens under implausible confounding scenarios is more defensible than one that vanishes with a small bias. The conclusion is not presented as definitive proof equivalent to a trial, but as a causal estimate with its identification assumptions stated and its robustness quantified. That transparency is exactly what a demanding reviewer asks for.
Can you evaluate the impact of a policy or protocol change by comparing before and after?
Yes, if the design allows a defensible comparison; observing a before and an after is not enough. It applies when there is no randomised control group, but there are pre- and post-implementation data and a group or discontinuity from which a reasonable counterfactual can be built, and you want to separate the effect of the change from the trends that were already under way. Difference-in-differences compares the trajectory of the affected group with that of a comparison group, under the parallel trends assumption, which must be examined, not assumed. Depending on the case, regression discontinuity designs may also fit. The underlying premise is that, without randomisation, the study design becomes the analysis: designing the comparison well matters more than applying a sophisticated model to a poorly framed comparison. What you receive is an impact estimate with its design and assumptions made explicit, not an automatic attribution of the observed change to the policy.
How does causal inference differ from mediation and moderation analysis?
Causal inference and observational study design is the broader umbrella: estimating the effect of an intervention or exposure on an outcome when there is no randomisation, with DAGs, propensity scores, difference-in-differences, g-computation and sensitivity analysis. Causal mediation and moderation analysis is one part of that umbrella, with its own audience and its own point of entry. It fits when the question is why or for whom an effect occurs: decomposing it into direct and indirect effects through a mediator, or seeing whether it varies across subgroups. They do not answer quite the same question. Mediation decomposes an effect into direct and indirect pathways under additional causal assumptions; general causal inference estimates the total or marginal effect of an exposure, intervention or policy. If you are looking specifically for "mediation analysis", there is a dedicated service for it; if your causal question is broader, this is where you start. The two are handled in coordination: mediation is a special case within this framework.
How does causal inference differ from real-world evidence (RWE)?
The methodological toolkit overlaps, with propensity scores, observational design and confounding control, but the framing and the context differ. Real-world evidence fits when the frame is generating clinical evidence from real-world data for a biopharmaceutical or medical affairs context, with its own language and its own audiences. Causal inference and observational study design is the frame for causal questions in health research and public health: the effects of interventions, exposures or policies in registries, cohorts or administrative data. Neither is more rigorous than the other: they share methods, but they address the same family of questions from different framings. If the need is framed as generating real-world evidence for biopharma, the natural frame is RWE; if it is framed as estimating a causal effect in a research or public health study, it is this service. The choice is one of context, not of method.
What do you deliver, and how do you respond to reviewers who ask for causal evidence?
At the end of a causal inference analysis you receive the effect estimate together with everything needed to defend how it was obtained: the DAG that formalises the assumptions, a justified adjustment set and the estimation strategy chosen for the question (matching, weighting, g-computation, difference-in-differences or mediation). We include balance checks and sensitivity analyses for unmeasured confounding. The R code makes it possible to trace each result back to the model, variables and assumptions behind it. For peer review, we set out the identification assumptions and show what changes when they are challenged. The research team contributes the subject knowledge, decides the substantive interpretation and signs the work; we provide the causal framework and estimation. The result is not an unqualified causal claim, but an estimate whose identification, uncertainty and robustness can be examined.
Shall we talk?
Tell us what you are working on and we will give you a straight answer on how we can help.