Meta-analysis and evidence synthesis for agricultural and food science

Evidence synthesis in agricultural and food science

Agricultural and food science asks how food is grown, processed and consumed, and what this means for productivity, the environment, safety and health. Typical synthesis questions include the yield effect of organic versus conventional management, the effect of cover crops or no-till on soil carbon and yield, the benefits of crop diversification, how processing changes nutrient content, the prevalence of contaminants in foods, and the effect of fortification on nutritional status. Meta-analysis is common in agronomy and has been widely used to compare farming systems.

The evidence has features that call for care. Yield and soil responses vary with climate, soil, crop and management, so heterogeneity is large and pooled averages hide differences. Systems compared under one label, such as organic farming, are diverse. Trials are conducted on research stations under better management than many farms. Our methods follow meta-analysis and systematic review practice, adapted to these features. This page builds on the general guidance for agriculture and food sciences. Related methods appear under plant sciences.

Comparing farming systems

A review should state how each comparison is defined, code the treatment details (rates, timing, duration, species), and use matched comparisons when available, in which both systems share site and year. Unmatched comparisons are confounded by site and farmer. Yield differences reported in meta-analyses of organic farming have varied with crop type, management practices and the comparability of the studies, and these are good examples of why a headline percentage is not enough.

Site and season variability

Agronomic effects depend on weather in the season, soil and climate. The same practice can raise yield in a dry site and lower it in a wet one. A review should extract site coordinates or climate zone, soil properties, year and crop, and test them as moderators, but because many are related, few conclusions can be drawn from a small set of studies. Long-term experiments are particularly valuable, since single-season results can be misleading, but they come from a small number of sites. Reviews should report effects with prediction intervals, to show how much results could differ at a new site.

Studies in multi-year experiments provide repeated measures from the same plots. Treating each year as independent overstates information, so multilevel models with random effects for site and plot are needed.

Soil and environmental outcomes

Soil carbon, nitrogen losses, greenhouse gas fluxes and biodiversity are measured with methods that differ in depth, timing and detection limits. Soil carbon change is slow and requires long experiments or very careful sampling. Comparisons of soil carbon between tillage systems depend strongly on sampling depth, since no-till tends to concentrate carbon near the surface, and deeper samples can reduce or remove apparent differences. Gas flux measurements are variable in time and space. A review should code depth, method and duration, and should not extrapolate short-term chamber measurements to annual budgets without discussion of uncertainty.

Food composition, processing and safety

Studies of food composition measure nutrients, contaminants and bioactive compounds in samples that vary with variety, growing conditions, storage and analytical method. Reviews comparing organic with conventional foods, for example, have found differences in some compounds but with high variability, and uncertain relevance for health. Studies of contamination, including pathogens, mycotoxins and pesticide residues, report prevalence in samples, and the sampling plan matters: random retail samples differ from samples drawn after a complaint. Prevalence is pooled using the methods in the prevalence meta-analysis page, with attention to detection limits and test sensitivity. This service does not certify food safety or give dietary advice.

Nutrition and health outcomes

Interventions such as fortification, biofortified crops, school feeding and agricultural programs aim to improve nutrition. Outcomes include dietary intake, biomarkers (such as hemoglobin or serum retinol) and child growth. Trials give causal estimates but often have short follow-up, and effects on growth are small. Biomarker thresholds and assays differ. A review should use appropriate measures for trials and observational studies, and link to the nutrition and public health reviews for clinical outcomes.

Farm-level economics and adoption

Plot trials rarely measure labor, costs, prices or risk, which determine whether a practice is adopted. Farm surveys do, but practices are chosen by farmers, so comparisons are confounded. A synthesis on adoption or profitability should keep experimental and survey evidence separate and should describe the economic outcomes carefully, with currencies and years standardized. Where results rely on assumptions about prices, sensitivity analysis shows how conclusions change. Evidence on smallholder farms in low-income countries differs from that on large commercial farms, and reviews should state which they cover.

Publication bias and trial-station effects

Experiment-station trials are managed intensively and may overestimate what farmers achieve. Positive findings for new practices are published more often. We use funnel-based methods adapted to multilevel data, compare journal articles with technical reports and theses, and check whether effects differ between research stations and on-farm trials.

Trade-offs and multiple outcomes

Agricultural practices usually affect several outcomes at once: yield, profit, soil health, water quality, biodiversity, greenhouse gas emissions and labor. A practice that improves one may worsen another, and a review that reports only the favorable ones gives a lopsided picture. We extract all outcomes that the included studies report on, show them side by side and note how many studies address each. Where studies measure emissions per unit area and per unit of product, both are reported, since a lower yield can turn a lower emission per hectare into a higher one per tonne. Choosing between them is a value judgment that depends on the question, and the review should not make it for the reader.

Evidence on trade-offs rarely comes from a single study, so cross-study comparisons are made on the basis of different sites and years. We say plainly when the evidence is indirect, and we avoid summing outcomes into a single score unless a transparent weighting is given.

Yield stability, risk and climate adaptation

Farmers care about the variability of yield as well as its average, because a poor season can be costly. Some practices, such as diversified rotations, are reported to reduce year-to-year variability, and others raise the average while increasing risk. Meta-analysis of variability uses effect sizes such as the log variability ratio, which require standard deviations across years or across replicates, and the distinction between these two kinds of variability should be explicit. Adaptation studies test crop varieties or practices under heat and drought, usually in managed trials, and results depend on how well the stress reflects real events. The review should describe the stress conditions in terms that allow readers to compare them with their own situation.

Common pitfalls we look for

• Using unmatched system comparisons as if they isolated the practice.
• Reporting a global average yield effect without the range across sites.
• Treating repeated years or plots as independent.
• Ignoring sampling depth in soil carbon comparisons.
• Mixing contamination samples from random and complaint-based sampling.
• Assuming research-station yields hold on farms.

Planning an agricultural or food science synthesis

We help define the system, practice or food, the comparator and the outcomes, plan searches in CAB Abstracts, AGRICOLA, Web of Science, Scopus and FSTA, and set up coding of site, soil, climate, crop, management, duration and measurement details. See the meta-analysis service for scope and process.

An invented example of reading a yield effect

Suppose a review of an alternative management practice reports an average yield change of minus 12 percent compared with the conventional comparator across 100 invented comparisons, with a prediction interval from minus 35 to plus 18 percent. The average suggests a lower yield. The interval says that in some conditions the alternative could match or exceed the conventional system. If the yield gap is 5 percent in matched comparisons with good management and 20 percent in unmatched ones, the design matters. A farmer or policy maker would also want to know about costs and prices, which the yield data do not show, so the review would describe yield as one outcome among several.

Coding and transparency

Coding frames record crop and variety, site and climate, soil type, treatment details, comparator, duration, plot size, number of replicates, the experimental unit, the year of measurement, the methods of measurement and funding. Two coders work independently on a sample and the coded data and code are shared with the final report. Studies that report results for multiple years are coded so that dependence can be modelled.

How we support research projects in this area

Boundaries of this service

An agricultural and food science synthesis describes average effects across published trials, surveys and laboratory studies. It does not provide farm advice, agronomic recommendations, food safety certification or dietary advice. Results vary widely with site, season and management, and plot-scale trials may not represent farms.

Frequently asked questions