Methodological reanalysis of meta-analyses on biofortified crops
An independent statistical review with metafor to assess the robustness of the evidence on the effectiveness of vitamin A, iron and zinc biofortification
Independently assessing whether the conclusions of a draft meta-analysis on biofortification withstand a reanalysis with more robust statistical methods
HarvestPlus needed an independent statistical review of a draft systematic review and meta-analysis article on the effectiveness of biofortified crops on nutritional indicators. The aim was not to repeat the calculations, but to check whether the conclusions held up under updated methodology.
The original article analysed 18 primary studies on crops biofortified with vitamin A, iron and zinc, assessing their effect on 8 different nutritional biomarkers (serum beta-carotene, serum retinol, retinol-binding protein, haemoglobin, ferritin, soluble transferrin receptor, body iron and plasma zinc). The meta-analyses had been run with the DerSimonian-Laird method, the classical random-effects estimator.
The methodological challenge was twofold. First, to reproduce the original results and verify the integrity of the data and calculations. Second, to reanalyse each meta-analysis with REML and the Knapp-Hartung adjustment — a methodological combination with better inferential properties than DerSimonian-Laird, especially relevant when there are few studies — to assess whether the effectiveness conclusions held under more cautious inference. The results were to be delivered in a bilingual report accompanied by a fully reproducible repository.
3 micronutrients
vitamin A, iron and zinc, analysed with multiple biomarkers per micronutrient
8 biomarkers
reanalysed with REML + Knapp-Hartung
18 primary studies
included in the original systematic review
A reanalysis pipeline with metafor: reproducing the original, updating the methodology and diagnosing robustness
Reproducing the original meta-analysis and reanalysing it with REML + Knapp-Hartung
Exact reproduction of the results with DerSimonian-Laird to verify the integrity of the data and calculations. A parallel reanalysis with REML (Restricted Maximum Likelihood) and the Knapp-Hartung adjustment, for their better inferential properties when there are few studies. Systematic comparison of both sets of results for each biomarker, documenting the cases where the conclusions diverged — especially in analyses with few studies, where DL tends to underestimate uncertainty.
Heterogeneity, study influence and exploration of possible small-study effects
For each meta-analysis, heterogeneity, outlying and influential studies and the stability of the results were assessed using prediction intervals, the Viechtbauer and Cheung method and Baujat plots. Funnel plots, Egger's test and trim-and-fill were used on an exploratory basis and interpreted with particular caution, given the small number of studies available for each biomarker. Key result: extreme heterogeneity (I² > 85%) in most analyses, with conclusions that depended on individual studies.
Meta-regression with moderators and multicollinearity analysis
Meta-regression models built with contextual moderators (country, crop type, target population). Multicollinearity between categorical moderators assessed with Cramér's V. Models compared by AIC, BIC and log-likelihood. In the exploratory serum retinol model, the combination of country and crop type was associated with a 79.8% reduction in residual heterogeneity (p=0.038). However, the small number of studies and the strong association between the moderators (Cramér's V > 0.60) limit the stability and interpretability of the result.
Independent evidence for biofortification policy decisions on a global scale
Main conclusion: the body of studies analysed does not provide enough evidence for definitive recommendations
The reanalysis showed that the limitations of the available body of studies — few studies per biomarker, high heterogeneity and instability when individual studies were excluded — prevented definitive, generalisable conclusions from being drawn from those meta-analyses. Although some analyses suggest promising effects (serum retinol for vitamin A, soluble transferrin receptor for iron), the high uncertainty and variability rule out reliable generalisations.
An updated estimation method with a direct impact on inference
Replacing DerSimonian-Laird with REML + Knapp-Hartung widened the confidence intervals and changed the statistical significance in several analyses, especially those with fewer than 5 studies. The report documents, case by case, where the conclusions change under a more cautious inferential approach, giving HarvestPlus an honest basis for communicating the state of the evidence to donors and policy decision-makers.
A reproducible repository delivered as a programme asset
Delivered as a bilingual (Spanish/English) Quarto book with a version-controlled repository and dependency management through renv. HarvestPlus can reuse the full pipeline when new primary studies are published, updating the meta-analyses without recoding. Custom, encapsulated functions for meta-regression, model comparison and diagnostics.
What this case shows about the way we work
An independent reanalysis, not a mere reproduction
We did not stop at verifying that the original calculations were correct. We ran a full reanalysis with updated methodology that changed the conclusions in several cases. That independent judgement is what the client needed to make informed decisions about how to communicate its results.
A documented methodological correction with practical implications
Every case where REML + Knapp-Hartung diverges from DerSimonian-Laird is documented, explaining why the original method can produce less conservative intervals and what the consequences are for each biomarker.
Custom functions and a reusable deliverable
A meta-regression pipeline with encapsulated functions that make it possible to add new moderators or new studies without rewriting code. Automated model comparison by AIC, BIC and log-likelihood. A deliverable designed to be a programme asset, not a report that is read once.
Independent judgement and transparent communication
The reanalysis concluded that the available evidence did not allow sufficiently robust policy recommendations to be made. Communicating that uncertainty clearly gives the client a more reliable basis for interpreting the results, prioritising new research and presenting the evidence to donors and decision-makers.
Capabilities deployed in this project
Shall we talk?
Tell us what you are working on and we will give you a straight answer on how we can help.