The short answer
A systematic review is a research method for identifying, appraising and summarizing all the studies that address a defined question, using explicit and reproducible procedures that limit bias. A meta-analysis is a statistical method for combining the numerical results of separate studies into a pooled estimate. The first is about how studies are found and judged. The second is about how their results are combined. Because pooling only makes sense for a set of studies that were chosen carefully, the two are usually done together: a systematic review identifies the studies, and a meta-analysis, where the studies are similar enough, combines their results.
The simplest way to remember it is that a systematic review is a process and a meta-analysis is a calculation. The process can be carried out without the calculation, and the calculation can be carried out, badly, without the process. The detailed methods are on the pages for systematic review and meta-analysis.
Side by side
| Systematic review | Meta-analysis | |
|---|---|---|
| What it is | A structured process of finding and appraising evidence | A statistical technique for combining results |
| Central question | What does all the relevant evidence show? | What is the combined size of the effect? |
| Main steps | Question, protocol, search, selection, appraisal, synthesis, reporting | Effect sizes, weighting, model, heterogeneity, sensitivity analyses |
| Needs | A focused question and eligibility criteria | Compatible numerical results from several studies |
| Output | A summary of evidence, with certainty and limitations | A pooled estimate with its interval and heterogeneity |
| Reporting guideline | PRISMA 2020 | PRISMA 2020; MOOSE for observational studies |
| Can stand alone? | Yes, with narrative synthesis | Only meaningfully on a systematically identified set of studies |
How they fit together
In a typical review of an intervention, the systematic review comes first. A question is framed, a protocol is written, studies are found by a comprehensive search, selected by criteria, extracted and appraised. If the studies are sufficiently similar in populations, interventions, comparators and outcomes, the results are combined in a meta-analysis, which provides a pooled estimate, examines the differences between studies and tests how robust the result is. The certainty of the evidence is then rated, and the whole is reported. The meta-analysis occupies one stage within the review, which is why many papers are titled a systematic review and meta-analysis, to signal both.
The dependence matters. The pooled estimate from a meta-analysis is only as representative as the set of studies that went into it. If the studies were chosen by convenience, or by finding only those that were easy to find, the estimate describes that selection and not the evidence. The systematic review is what gives a meta-analysis its credibility. The analysis is described in the guide what is meta-analysis?
A systematic review without a meta-analysis
Many systematic reviews do not include a meta-analysis, and that is legitimate. There are several reasons. The studies may be too few, so that a pooled estimate would add nothing or be unstable. They may differ too much in population, intervention, comparator or outcome, so that an average would be meaningless. They may report results in forms that cannot be combined. Or the question may not be about an effect at all, as in a review of qualitative studies. In these cases, the review reports a structured narrative synthesis, in which studies are grouped in a way that makes sense, a common metric is used where possible, the results are tabulated and described, and conclusions are drawn with an assessment of certainty. The SWiM guideline sets out how to report such a synthesis without meta-analysis.
A common error is to treat a review without pooling as a lesser product. If the studies cannot be sensibly combined, then not pooling is the correct decision, and a pooled estimate imposed on them would be misleading. See systematic review.
A meta-analysis without a systematic review
It is possible to pool results from a set of studies that was not found by a systematic search, and it is a common source of poor science. A researcher may combine the studies they happen to know of, the ones from one database, or the most prominent ones. The statistics can be carried out perfectly, and the result will still be unreliable, because the selection of studies is biased in ways that the analysis cannot see or correct. Such an analysis may be defensible as a pooled analysis of a defined set of studies, for example the trials done by one group, or the trials in one development program, provided it is described accurately and the conclusions are limited to those studies. It should not be presented as a summary of the evidence on a question.
Journals and reviewers have become wary of analyses with no systematic search, and many reject them or ask for a full search. Authors who describe their work as a meta-analysis should say how the studies were identified. See the comparison of review types.
An example of the two in one project
Consider a project on whether a school-based mindfulness program reduces anxiety in children. The systematic review part begins with a protocol that states the population, the intervention, the comparator and the outcome. The team searches several databases and trial registers, screens a few thousand records in duplicate, includes twenty-two randomized trials, extracts the data and assesses risk of bias with RoB 2. At that point the meta-analysis begins: they compute a standardized mean difference for each trial, pool the effects with a random-effects model, examine heterogeneity, run sensitivity analyses that exclude trials at high risk of bias, and check for small-study effects. Then, back in the review, they rate the certainty of the evidence and write conclusions that match it. If instead only six trials had used comparable measures, the team might have pooled those and described the others narratively, and if the trials had differed too much, they might not have pooled at all. The example is invented to show how the two fit together.
What editors and reviewers expect
Editors and peer reviewers distinguish the two quickly. For a paper described as a systematic review, they look for a protocol or registration, the search strategies, the flow diagram, the criteria and the risk-of-bias assessment, in line with PRISMA 2020. For a meta-analysis, they look at the choice of effect measure and model, the handling of heterogeneity, the sensitivity analyses and the assessment of publication bias. A paper that claims both must satisfy both sets of expectations. Authors who label a convenience-sample meta-analysis as a systematic review are likely to be asked to justify it or to change the label. Equally, a review that did not pool and has been criticized for it can answer that the studies were not comparable, if the report shows why. Matching the label to the work, and reporting it fully, is the best way through review. See PRISMA 2020.
Naming your work accurately
- Systematic review and meta-analysis when a systematic search and selection were followed by pooling.
- Systematic review when studies were found and appraised systematically and not pooled, with the reason stated.
- Meta-analysis alone only when the studies were not found by a systematic process, with the selection described honestly, or when the report concerns only the analysis part of a larger review.
- Pooled analysis for the combination of data from specified studies, such as a planned analysis of several trials by one group.
- Not systematic when no protocol, no comprehensive search and no explicit criteria were used. In that case a narrative review is the accurate term.
The title and abstract should identify the design, since this affects indexing, how the paper is found and how readers weigh it. PRISMA 2020 asks that the report be identified as a systematic review in the title. A misleading label can lead to rejection or criticism.
Deciding whether to pool
The decision to carry out a meta-analysis is made in the review, usually in the protocol, and confirmed when the data are in. The first consideration is similarity: do the studies ask the same question, in comparable populations, with comparable interventions and outcomes? The second is data: are the results available in forms that can be combined? The third is number: is there more than one study, and are there enough to make the average meaningful? The fourth is direction: if some studies show benefit and others harm, the average may conceal an important finding, and describing the pattern may be better than pooling. And the fifth is purpose: does a pooled estimate answer a question the readers have? If the answer to the first three is yes, a meta-analysis is usually justified. See heterogeneity.
Where the terms came from
The word meta-analysis was introduced by Gene Glass in 1976 for the statistical analysis of the results of many studies, and the methods were applied from the 1970s and 1980s in education, psychology and medicine. The term systematic review came into use in health care in the late 1980s and 1990s, as the movement for evidence-based medicine and the founding of the Cochrane Collaboration stressed that how studies are found matters as much as how they are combined. Early reviews and meta-analyses were sometimes done with unsystematic searches, and the consequences, such as conflicting meta-analyses of the same question, helped to establish the standards for the review process. The reporting guidelines that followed, QUOROM in 1999 and then PRISMA, reflected the view that the review and the analysis should be reported together.
Common mistakes
- Using the two terms as synonyms.
- Calling a review systematic when it has no protocol, no documented search or no explicit criteria.
- Pooling results from a convenience sample of studies and presenting the number as the effect.
- Believing that a systematic review must include a meta-analysis, and pooling studies that are not comparable to satisfy this.
- Believing that a meta-analysis is automatically the best evidence.
- Failing to say in the title and abstract which design was used.
Support
Both are available separately or together: the systematic review service covers the search, selection and appraisal, and the meta-analysis service covers the statistical analysis.
Get a quoteTell us what you have and what you need.
Frequently asked questions
Is a meta-analysis a type of systematic review?
Not exactly. A meta-analysis is a statistical step that is usually part of a systematic review. They are different things that are often done together.
Can I do a systematic review without a meta-analysis?
Yes. When studies are too few or too different to pool, a structured narrative synthesis is reported, following guidance such as SWiM.
Can I do a meta-analysis without a systematic review?
Statistically yes, but the result describes only the studies you included. Without a systematic search it should not be presented as a summary of the evidence.
Which should I choose for my thesis or paper?
If your aim is to answer a focused question from the existing literature, a systematic review, with a meta-analysis if the studies are suitable. The question and the available studies decide.
What should I call my paper?
Systematic review and meta-analysis if you did both, systematic review if you did not pool, and be accurate about the methods in the title and abstract.
Is a meta-analysis always stronger evidence?
No. It depends on the quality of the studies and of the selection process. A well-conducted narrative systematic review can be more trustworthy than a poorly conducted meta-analysis.
References
- Higgins JPT, Thomas J, Chandler J, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions. Cochrane; current version available at training.cochrane.org/handbook.
- Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. doi:10.1136/bmj.n71
- Glass GV. Primary, secondary, and meta-analysis of research. Educ Res. 1976;5(10):3-8.
- Campbell M, McKenzie JE, Sowden A, et al. Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ. 2020;368:l6890.
- Mulrow CD. The medical review article: state of the science. Ann Intern Med. 1987;106(3):485-488.
- Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Lancet. 1999;354(9193):1896-1900.
- Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to Meta-Analysis. Wiley; 2009.
- Ioannidis JPA. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 2016;94(3):485-514.