Meta-analysis and evidence synthesis for ecology

Evidence synthesis in ecology

Ecology and evolutionary biology have embraced meta-analysis to answer questions that single experiments cannot: how warming affects growth across species, whether biodiversity increases ecosystem productivity, how invasive species change native communities, how predators affect prey, and what effect habitat loss has on populations. Studies span many organisms, ecosystems and methods, so generality is a central question in every synthesis, and synthesis methods are designed to estimate both an average and the variation around it.

Ecological data bring particular challenges. Species are related through evolutionary history, so results from closely related species are not independent. A paper may report many comparisons from the same site or experiment. Responses are often ratios, which are skewed. Study systems are not a random sample of nature. Our methods follow meta-analysis and systematic review practice, adapted to these features. This page builds on the general guidance for life sciences.

Effect sizes for ecological data

The log response ratio is popular because many ecological outcomes are ratios and positive, and because proportional change is a natural scale: a treatment that increases biomass by 20 percent has a response ratio of 1.2 and a log value of about 0.18. A review should specify the effect size in the protocol, check that data meet its assumptions and report results back-transformed to the percentage change for interpretation. Missing standard deviations are common, and imputation, if used, should be explained and tested in sensitivity analysis.

Phylogenetic non-independence and multilevel structure

Effect sizes from related species are likely to be more alike than those from distant species, and effect sizes from the same study, experiment or site share conditions. Treating them as independent overstates precision. Multilevel meta-analytic models include random effects for study, for the observation within study and, where a phylogeny is available, for species and for the phylogenetic relationship. The variance partition shows how much heterogeneity is attributable to each level. Reviews in ecology that ignore phylogeny when it matters can give misleading confidence intervals; those that include it should state how the tree was obtained and how branch lengths were handled.

Models of this kind need a reasonable number of species and studies to estimate the variance components. With few studies, estimates are imprecise, and the review should say that instead of reporting zero variance as if it were established.

Scope and generality

Ecological meta-analyses often claim general results, but the evidence is concentrated in particular taxa (plants and insects), ecosystems (temperate grasslands and forests) and regions (North America and Europe). Systematic maps and evidence atlases describe where research has been done and where it has not, and they are a useful first step. A review should report the distribution of studies by taxon, biome and country, and be cautious about extrapolating to under-studied groups. Studies in the lab or mesocosm and studies in natural settings differ in realism and control, so setting is a standard moderator.

Observational data and space-for-time substitution

Many ecological questions are answered with observational comparisons, such as sites along a gradient of disturbance or elevation. Substituting space for time assumes that differences between sites show how one site would change over time, which can fail when sites differ in other ways. Long-term monitoring data are valuable but unevenly distributed. A review should code the design (manipulative experiment, natural experiment, observational gradient, long-term series) and the controls for confounders, and compare effects across design types. Causal wording should be restricted to manipulative designs.

Climate and global change responses

Syntheses on warming, elevated carbon dioxide, nitrogen addition and drought combine experiments that differ in the amount of change imposed, duration and the realism of the treatment. Dose varies, and effects may not be linear. Meta-regression on the magnitude of warming or on duration helps, and a review should report effects per unit change when the data allow, and describe the range of treatments. Short experiments may capture initial responses that differ from long-term responses after acclimation, and the review should look at duration as a moderator, reporting how many experiments ran for more than one growing season or generation.

Biodiversity and ecosystem function

Studies of diversity and ecosystem function manipulate the number of species in plots and measure productivity or other processes. Meta-analyses have shown, on average, positive effects of species richness on biomass in experiments, and the strength depends on the system, the duration and the diversity range. Observational studies of natural gradients produce different patterns because diversity covaries with environment. A review should separate experiments from observations, report the experimental range of richness and avoid extrapolating beyond it. Metrics of diversity also differ (richness, evenness, functional or phylogenetic diversity), and conclusions should say which was used.

Publication bias and time-lag effects

Ecology has shown evidence of small-study effects and of decline effects, where early studies report larger effects than later ones. Tests for funnel asymmetry, trim-and-fill and regression methods that account for multilevel structure are used, along with models that include publication year and the journal impact factor as moderators. Because these tests can mislead under high heterogeneity, we report several and discuss assumptions. Searching theses, reports and non-English literature reduces bias, and many ecological studies are in languages other than English.

Variance, variability and responses beyond the mean

Most syntheses ask whether a treatment changes the average. Some questions concern variability: whether a stressor makes populations more variable, or whether a trait is more variable in one group than another. Effect sizes for variability, such as the log variability ratio and the log coefficient of variation ratio, compare spreads between groups, and they depend on the mean in different ways. If means differ between groups, a difference in standard deviation may simply follow from the mean, and the coefficient of variation ratio addresses that. Reviews should say which question is asked, report both mean and variance effects where data allow, and avoid claiming that a treatment makes responses more unpredictable unless a variability effect size has been tested and the intervals support it.

Common pitfalls we look for

• Treating observations from one study or one species group as independent.
• Using lnRR with zero or near-zero means without addressing bias.
• Imputing missing SDs without sensitivity analysis.
• Extrapolating from well-studied taxa and regions to the whole of nature.
• Using space-for-time results as if they showed change over time.
• Reporting zero variance components when data cannot estimate them.

Planning an ecology synthesis

We help define the question using a framework such as PICO adapted to ecology (population, exposure or intervention, comparator, outcome), plan searches in Web of Science, Scopus, CAB Abstracts, BIOSIS and Google Scholar, and set up coding of taxon, system, design, treatment magnitude, duration and location. Reporting follows PRISMA extensions including PRISMA-EcoEvo for ecology and evolution where relevant. See the meta-analysis service for scope and process.

An invented example

Suppose a multilevel model of 200 effect sizes from 40 invented studies of a warming treatment on plant growth gives an average log response ratio of 0.12, which corresponds to about a 13 percent increase in growth (the exponential of 0.12 is about 1.13). The variance partition shows that most heterogeneity lies among effect sizes within studies and among species, and little between studies. The prediction interval runs from about minus 0.20 to 0.44, so for a new species the response could be a decline or a large increase. The accurate summary is that warming tends to raise growth in the studied plants but with wide variation, and that the data cannot predict the response of a specific species not included.

Coding and data management

Ecological data extraction often involves figures, because means and errors are plotted and not tabulated. We extract values with digitizing software, record the method and check a sample with a second coder. Standard errors are converted to standard deviations using the sample size. We record the identity of the study system in sufficient detail to match species to a phylogeny, store the coded data and R code in a repository and share them with the final report.

Responsible interpretation

Ecological findings feed into conservation and policy. A review makes clear what its evidence covers, what is uncertain and where data are lacking. It does not provide management advice for a particular site or species, which requires local expertise and information that a synthesis does not have.

How we support research projects in this area

Boundaries of this service

An ecology synthesis describes average effects and their variation across published studies. It does not provide conservation management advice, environmental impact assessment, species identification or predictions for a specific site. The studies represent a non-random sample of taxa, systems and regions, so results may not extend to unstudied groups.

Frequently asked questions