What is meta-analysis?

Definition in plain terms

A meta-analysis is a statistical method for combining the quantitative results of separate studies that address the same question. Each study supplies an estimate of an effect, such as the difference in an outcome between a treatment group and a control group, together with a measure of how precisely that effect was estimated. The meta-analysis gives each estimate a weight, usually larger for more precise studies, averages them, and reports a single pooled estimate with a confidence interval.

Combining studies serves three purposes. It increases precision, because individual studies are often too small to estimate an effect reliably. It allows consistency to be checked, since an effect seen in one setting can be compared with the effect in others. And it makes disagreement between studies measurable and explorable instead of leaving it as an impression. The name reflects the idea of an analysis of analyses: the data being analyzed are the results of earlier studies and not the raw observations of individual participants, although meta-analysis of participant-level data is also possible.

A meta-analysis is a method of analysis, not a research design in itself. It is usually carried out as part of a systematic review, which is the structured process of finding, selecting and appraising the studies whose results are then combined. The distinction is explained in meta-analysis versus systematic review. A meta-analysis that pools whichever studies were convenient to find has no protection against selective inclusion, and its pooled estimate describes only those studies.

A short history

Pooling results across studies is older than the term. In 1904 the statistician Karl Pearson combined data from several studies of typhoid inoculation to judge its effect, which is often cited as an early example. In the 1930s Ronald Fisher described a method for combining the probabilities from independent tests, which dealt with significance and not with the size of effects.

The word meta-analysis was introduced by Gene Glass in 1976, and the approach was applied the following year by Smith and Glass to a large body of psychotherapy outcome studies. Through the 1980s and 1990s the methods spread through clinical medicine, supported by organizations such as Cochrane, which developed shared standards for searching, appraising and pooling trials. Today meta-analysis is used across the health sciences and in psychology, education, management, economics and ecology, and the statistical methods have developed considerably, including random-effects models, prediction intervals, network meta-analysis and Bayesian approaches.

How a meta-analysis works

The statistics are the last part of a longer process. A meta-analysis is only as sound as the steps that lead up to it.

1. Define the question

   The question is stated precisely, including the population, the intervention or exposure, the comparator and the outcome. Eligibility criteria follow from it and are written down before searching begins.

2. Find and select studies

   Databases and registries are searched with documented strategies, and studies are screened against the eligibility criteria. The process is reported with a flow diagram, as set out in PRISMA 2020.

3. Extract effect estimates

   From each study, the effect estimate and its standard error are extracted, or computed from the reported counts, means and standard deviations. Study characteristics and risk-of-bias judgments are recorded alongside.

4. Weight and pool

   Each study receives a weight that reflects the precision of its estimate, and the weighted average is computed under a fixed-effect or random-effects model. See fixed-effect and random-effects models.

5. Examine differences and bias

   Heterogeneity between studies is quantified and explored, sensitivity analyses test the robustness of the result, and small-study effects are assessed. The certainty of the overall conclusion is then judged, for example with GRADE.

A worked example

The example below uses six simulated studies. The numbers were generated for illustration and do not come from real research. Each study reports a standardized mean difference (SMD), where a positive value favors the intervention, together with its standard error.

In a fixed-effect analysis each study is weighted by the inverse of its variance, so Study D, with the smallest standard error, carries the most weight at 31.2 percent and Study C, with the largest, carries the least at 7.8 percent. The weighted average is an SMD of 0.22, with a 95 percent confidence interval from 0.11 to 0.33.

A random-effects analysis first estimates how much the true effects vary between studies. Here the Q statistic is 6.16 on 5 degrees of freedom, I² is 19 percent and the between-study variance (τ²) is 0.0045. Because the true effects are allowed to differ, each study's weight is adjusted so that the weights become more even, as the last column shows. The pooled SMD is 0.22 with a 95 percent confidence interval from 0.10 to 0.35, which is wider than the fixed-effect interval because it carries the uncertainty about between-study variation.

The result has a second interpretation that is easy to miss. The 95 percent prediction interval, which describes where the true effect in a new study would be expected to fall, runs from -0.03 to 0.48 and includes zero. The pooled average is clearly above zero, but a future study in a different setting could still show no benefit or a small harm. A reader who looked only at the confidence interval would reach a more confident conclusion than the data support.

How to read a meta-analysis

Pooled estimate

The weighted average effect across studies. Its meaning depends on the effect measure: a ratio of 1 or a difference of 0 means no effect.

Confidence interval

The range of values for the average effect that are compatible with the data. A narrow interval signals precision, not correctness.

Heterogeneity

How much the true effects differ between studies, usually summarized by I² and the between-study variance. See heterogeneity and I².

Prediction interval

The range in which the true effect of a new study would be expected to fall. See prediction intervals.

Forest plot

The standard display: one line per study, with a diamond for the pooled result. See how to read a forest plot.

Funnel plot

A graph of each study's effect against its precision that can reveal small-study effects, including publication bias. See funnel plots.

Beyond these outputs, the reader should check three things that the numbers do not show: how the studies were found, how their risk of bias was assessed, and how many of the included studies contribute to each conclusion. A tidy forest plot built on a biased set of studies is still biased.

Types of meta-analysis

The basic procedure has many extensions, each suited to a different kind of question or data.

• Pairwise meta-analysis compares two conditions and is the standard approach for trials of an intervention against a control.
• Network meta-analysis compares several interventions at once using direct and indirect evidence.
• Bayesian meta-analysis combines study data with prior distributions and is useful when studies are few.
• Meta-regression relates effect sizes to characteristics of the studies.
• Diagnostic accuracy meta-analysis pools sensitivity and specificity.
• Dose-response meta-analysis examines how an outcome changes with the amount of an exposure.
• Individual participant data meta-analysis combines participant-level data instead of published summaries.

How meta-analysis is used across fields

The questions differ by discipline, although the logic is the same. The examples below are illustrative questions, not findings.

• Medicine and health sciences: does a drug reduce mortality compared with a placebo, or how accurate is a screening test?
• Psychology: how large is the average effect of a therapy on symptoms, and does it depend on the population?
• Education: what is the average effect of a teaching method on achievement, and does it vary by grade level?
• Management: how strongly are two organizational constructs related across firms and settings?
• Economics: how responsive is labor supply to changes in wages, as estimated across studies?
• Ecology and life sciences: how does an environmental change affect growth or survival across experiments and species?

Common misunderstandings

More studies always give a better answer

Adding studies that share the same bias gives a more precise estimate of a biased quantity. The quality and independence of the studies matter as much as their number.

A meta-analysis is automatically the strongest evidence

It is only as strong as the studies it combines and the process that selected them. Certainty ratings such as GRADE exist because pooled evidence can still be of low certainty.

It combines apples and oranges

This is a fair concern when studies ask different questions. It is addressed by precise eligibility criteria and by examining heterogeneity, and in some cases the right decision is not to pool.

A non-significant pooled result means there is no effect

It may reflect an imprecise estimate. The confidence interval shows which effect sizes remain plausible, including clinically or practically important ones.

Counting significant studies is an adequate summary

Vote counting ignores the size and precision of each study and can mislead in either direction. Weighted pooling does not have this flaw.

Heterogeneity is simply a defect

Differences between studies are information. They can show where an effect is larger or smaller and prompt better questions.

Strengths and limitations

The main strengths are precision, transparency and the ability to examine consistency across settings. A well-conducted meta-analysis shows how a conclusion depends on the choices behind it, and it makes disagreement between studies visible.

The main limitations follow from its dependence on the underlying studies. Publication bias and selective reporting can distort the available evidence. Bias that affects all included studies in the same way cannot be corrected statistically. Observational studies are exposed to confounding that no pooling can remove. With few studies, estimates of between-study variation are imprecise and several diagnostic checks lose power. A pooled number can also hide meaningful differences between subgroups. These are reasons to report a meta-analysis carefully, and sometimes reasons to choose a different synthesis method; see methods for the alternatives.

Planning your own meta-analysis

Frequently asked questions