Meta-analysis in life and biological sciences

Evidence synthesis in biology

Meta-analysis has been used in biology since the 1990s, and ecology and evolutionary biology were early adopters outside medicine. Biologists often have many small experiments on the same question: the effect of warming on plant growth, of a diet on lifespan, of a gene variant on a trait. Combining them gives a more general answer than any single experiment and shows how the effect varies with species, habitat or method.

The field differs from clinical research in several ways. Studies are rarely registered. Sample sizes are small, the units are plots, tanks, cages or animals, and one paper often reports many comparisons. Outcomes are continuous and measured on scales that differ between papers. The target is often a general principle, such as how species respond to temperature, and not an estimate for one population. As a result, heterogeneity is expected and is part of the finding; the review explains it with moderators and does not treat it as a nuisance.

This page describes how we approach such reviews. Methods are covered in detail under meta-analysis, meta-regression and systematic review.

Effect sizes in biological data

The log response ratio needs positive means and an approximate sampling variance that depends on sample size and the coefficient of variation. When means are close to zero or sample sizes are very small, the approximation can be biased, and small-sample corrected versions have been proposed. The choice of effect size should be fixed in the protocol and checked against the data.

Some reviews study variation as well as averages. Effects on the coefficient of variation or on the standard deviation can matter, for example when a treatment makes responses more uniform or more erratic. Such analyses need effect sizes designed for variance and a check of the mean-variance relationship.

Dependence: phylogeny, shared control, many effects per study

The most common error in biological meta-analysis is treating dependent effect sizes as independent. There are three main sources. First, one study often reports several effect sizes, for different outcomes, time points or species, and these share a design and an author team. Second, effect sizes may share a control group, so they are correlated. Third, species are related by descent, so closely related species resemble each other for reasons other than the treatment.

Multilevel models address the first source by adding random effects for study and for the individual effect size. Shared controls need sampling covariances or a model that accounts for them, or the control sample can be split between comparisons as an approximate fix. Phylogenetic meta-analysis adds a random effect whose covariance follows a phylogenetic tree, which is available in R packages such as metafor and MCMCglmm. Nakagawa and Santos reviewed these issues in 2012, and Nakagawa and colleagues later proposed ten appraisal questions that a biologist can ask of any published meta-analysis.

Robust variance estimation is a useful complement: it gives valid standard errors for clustered data when the exact covariance structure is not known. The report should state which approach was used, why and how results change under another.

Preclinical and animal studies

Animal studies sit between biology and medicine. Systematic review of such studies is used to judge whether a treatment should go forward to human trials, to find the reasons for failure of translation and to improve the design of later experiments. The CAMARADES group developed much of the methodology, and SYRCLE published a risk-of-bias tool adapted from the Cochrane tool for animal intervention studies. Its domains include sequence generation, baseline characteristics, allocation concealment, random housing, blinding and selective outcome reporting.

Features that need attention in preclinical synthesis include the unit of analysis (animal, cage or litter), multiple treatment groups sharing one control, outcomes reported only in graphs, and strong evidence of publication bias in many fields. Reviews often report the proportion of studies that describe randomization or blinding, because these reporting rates are themselves informative. Vesterinen and colleagues give a practical guide to the analysis of data from animal studies.

Small-study effects and publication bias

Funnel plots, Egger's regression and trim-and-fill are used in biology as in medicine, but they assume independent effect sizes. With multilevel data, a modified Egger test that includes the study-level random effect, or a regression on the standard error or sampling variance within a multilevel model, is more suitable. Time-lag bias, where early studies report larger effects than later ones, is also tested in biological syntheses by regressing the effect size on publication year.

Selective reporting within studies is harder to see. A paper may report only the significant outcomes of many measured. Where raw data or supplementary tables exist, extracting all measured outcomes reduces this risk. As in other fields, the right conclusion from a bias test is a statement about the strength of evidence, not a claim that bias is excluded.

Moderators, heterogeneity and generality

Biological meta-analyses are usually run to find out how general an effect is. The heterogeneity statistics are therefore central. A high I-squared is expected when studies span species, habitats and methods, and a review that reports only the pooled mean misses the main result. Reviewers report the between-study variance, a prediction interval and the proportion of variance at each level of a multilevel model: between studies, between effect sizes within studies and sampling error.

Moderators are chosen in advance on biological grounds, such as taxonomic group, latitude, experimental duration, life stage, and whether the study was done in the laboratory or the field. A meta-regression tests them, with the limits that apply to study-level data: confounding between moderators, few studies in some categories and the ecological problem. Many reviews also report the results of a model with all moderators and of models with one at a time, because moderators are often correlated. For example, studies on tropical species may also be the ones done in short field seasons.

It helps to plot the data. Orchard plots, which show individual effect sizes sized by precision with the pooled mean and interval, are common in ecology and evolution and display heterogeneity directly. Reports should state how many studies and species contribute to each level of a moderator, so readers can see where the evidence is thin.

Common problems we look for

• Pseudoreplication of effect sizes. Entering every comparison from a paper as an independent study. The fix is a multilevel model or robust variance estimation.
• Unexplained effect-size choices. Changing the measure after seeing the data, or using the log response ratio with means near zero.
• Weighting by sample size alone. Using the number of animals or plots as a weight when variances are available, or the reverse.
• Missing variance information. Studies that report no standard deviation or sample size are dropped without checking whether this causes bias. Imputation methods exist, and their effect should be tested.
• Overstated generality. Presenting a pooled mean from a few well-studied species as a statement about all organisms.

Search, extraction and reproducibility

Searches in biology use databases such as Web of Science, Scopus, BIOSIS and CAB Abstracts, with PubMed for biomedical topics, and often include dissertations and reports. Data are often in figures and need extraction with digitizing tools, with a record of the tool and of checks between extractors. The PRISMA-EcoEvo extension (O'Dea and colleagues, 2021) adapts PRISMA 2020 to ecology and evolutionary biology. It asks for reporting of the effect size choice, the handling of non-independence and the data and code that allow the analysis to be reproduced.

We recommend making the extracted dataset and analysis scripts available when the journal or funder allows, since reproducibility is an expectation in the field.

Specialties and sub-fields

Each sub-field has its own designs and effect measures. Pages for sub-fields are added as they are completed.

How we support research projects in this area

Boundaries of this service

Evidence synthesis summarizes published and available results. It does not replace laboratory work, and a pooled effect from published experiments does not show that an effect will hold in a new system. Many biological meta-analyses combine heterogeneous experiments, and the pooled estimate is a summary of variation, not a single true value. We do not provide clinical advice, and we do not generate experimental data.

Where a question is better answered by a systematic map of the literature, or where too few studies exist to pool, we say so at the outset.

Frequently asked questions