Meta-analysis for medical and health sciences

Evidence synthesis in medicine and health sciences

Medicine is the field in which systematic review and meta-analysis methods were first developed and are most formalized. Questions about treatments, tests, prognosis and risk are usually addressed by many studies of moderate size that do not agree, and decisions about care, policy and further research depend on a defensible summary of that evidence. Organizations such as Cochrane helped establish the methods that most clinical journals now expect, and the guidance in the Cochrane Handbook remains the main reference for reviews of interventions.

Several features distinguish health research from other disciplines. Outcomes are chosen for their importance to patients, so they are often events or times to events and not continuous measurements. Study designs range from randomized trials to registry analyses, each with its own threats to validity. Reporting is standardized through PRISMA 2020 and its extensions, protocols are expected to be registered in advance, and the certainty of the evidence is rated in a structured way using GRADE. The consequences of a wrong conclusion are also greater, which is why the conventions in this field are more prescriptive than in many others.

Medical and health sciences are the flagship area of this service. The same statistical framework described under meta-analysis and systematic review applies, with the design choices, outcome measures and appraisal tools specific to clinical and public health research described on this page.

Specialties and the evidence they rely on

Each specialty has its own typical study designs, outcome measures and recurring methodological problems. The summaries below indicate what distinguishes the evidence in each area, and each links to a page that develops the specialty in detail.

Study designs and what each contributes

A health-related synthesis usually draws on more than one kind of study, and the design determines both the appraisal tool and the way results are pooled.

Randomized trials

The primary evidence for the effects of an intervention. Risk of bias is assessed with RoB 2, and in GRADE the evidence starts at high certainty. Common problems are small trials, differences between comparators, and selective reporting of outcomes.

Cohort and case-control studies

Needed for harms, long-term outcomes and exposures that cannot be randomized. They are pooled as adjusted estimates and appraised with ROBINS-I or the Newcastle-Ottawa Scale, with reporting following MOOSE. Confounding is the central threat.

Diagnostic accuracy studies

Estimate sensitivity and specificity against a reference standard. Appraised with QUADAS-2 and pooled with bivariate models, as described under diagnostic accuracy meta-analysis.

Prognostic and prediction studies

Examine which factors or models predict outcomes. Prognostic factor studies are appraised with QUIPS and prediction model studies with PROBAST. See prognostic meta-analysis.

Prevalence and incidence studies

Estimate how common a condition is. The quality of the sampling frame and large between-study heterogeneity are the main issues. See prevalence meta-analysis.

Qualitative and mixed-methods studies

Describe the experiences and views of patients and clinicians. They are synthesized with methods such as thematic synthesis and reported with guidance such as ENTREQ. See qualitative evidence synthesis.

Common research questions

Most health-related reviews fall into a small number of question types. Each points to a different method.

• Does an intervention work compared with a control? Pairwise meta-analysis of randomized trials, usually with a random-effects model. See pairwise meta-analysis.
• Which of several treatments works best? Network meta-analysis, which combines direct and indirect comparisons and requires a plausible transitivity assumption.
• How accurate is a diagnostic or screening test? Meta-analysis of sensitivity and specificity, with attention to the threshold used and the patient spectrum.
• Which factors predict an outcome? Synthesis of prognostic factor or prediction model studies, typically using hazard ratios and discrimination measures.
• How common is a condition, and how does it vary? Meta-analysis of proportions or rates, with exploration of heterogeneity by region, period and population.
• Is a treatment or exposure safe? Synthesis of adverse events, often rare, from trials and observational studies, using methods suited to sparse data.
• Does risk change with the amount of an exposure? Dose-response meta-analysis, common in nutrition and environmental epidemiology.
• How do patients and clinicians experience care? Qualitative or mixed-methods synthesis, often in nursing and public health.

Methods commonly used in health research

The table links the common question types to the method, the usual effect measure and the reporting standard that journals look for.

Outcomes and effect measures in health research

Choosing and interpreting the outcome is often harder in health research than the statistics that follow. Five issues come up repeatedly.

Binary outcomes and baseline risk

Risk ratios and odds ratios are relative measures, and a relative effect that looks consistent across trials can correspond to very different absolute effects when the baseline risk differs. A useful report therefore converts the pooled relative effect into absolute terms for stated baseline risks, so that readers can see how many events would be expected to change in a population like their own. For common outcomes the odds ratio overstates the risk ratio, and it should not be described as if it were one.

Time-to-event outcomes

Survival and time-to-recurrence outcomes are pooled as hazard ratios on the log scale. Not every trial reports the hazard ratio and its standard error. When they are missing, they can sometimes be derived from reported confidence intervals, from log-rank statistics, or estimated from published survival curves, and the reliability of each route differs. The proportional hazards assumption should be examined when follow-up is long or curves cross.

Continuous outcomes and patient-reported measures

Symptom and function outcomes are measured on many different scales, which is why standardized mean differences are common in psychiatry, orthopedics and rehabilitation. A standardized effect is hard to interpret clinically, so results are usually re-expressed on a familiar scale and compared with the minimal important difference where one has been established. Mixing change-from-baseline and final-value data without care can distort results.

Composite and surrogate endpoints

Composite endpoints, such as major adverse cardiovascular events, combine outcomes of different importance and frequency, so a favorable composite can be driven by the least important component. The composite and its components should both be examined. Surrogate endpoints, such as progression-free survival or a laboratory marker, are acceptable substitutes only to the extent that they are shown to predict outcomes that matter to patients, and trial-level analyses in oncology have found the strength of that association to vary widely.

Adverse events and rare events

Harms are reported inconsistently and are often too rare for standard inverse-variance methods. Mantel-Haenszel or Peto approaches, careful handling of studies with zero events, and the use of observational data for long-term safety are part of any serious analysis of harms. Absence of reported events is not evidence of safety when follow-up is short or events were not systematically sought.

Methodological considerations specific to health evidence

Risk of bias and certainty of evidence

Appraisal tools are matched to design: RoB 2 for randomized trials, ROBINS-I for non-randomized studies of interventions, QUADAS-2 for diagnostic accuracy, and QUIPS or PROBAST for prognostic work. GRADE then rates the certainty of the body of evidence for each outcome. Five factors can lower certainty (risk of bias, inconsistency, indirectness, imprecision and publication bias), and three can raise it for observational evidence (a large effect, a dose-response gradient, and residual confounding that would reduce the observed effect). The service for risk-of-bias and certainty assessment covers this stage.

Protocol registration and selective reporting

A protocol written before the review begins, and registered where an appropriate registry exists, limits the freedom to change eligibility criteria or analyses once results are seen. PROSPERO accepts systematic reviews with a health-related outcome. Within the included studies, selective outcome reporting is a recognized source of bias, which is why reviewers compare the outcomes reported in a publication with those pre-specified in the trial registry or protocol whenever these are available.

Duplicate publications and overlapping data

One trial can generate several papers, and related reviews can include the same trials. Counting a trial twice inflates its weight and narrows the confidence interval. Linking reports to their parent study, and checking for overlap in umbrella reviews, is routine work in health evidence synthesis.

Populations, baseline risk and clinical heterogeneity

Differences in patient characteristics, severity, setting, co-interventions and follow-up often explain more of the variation between studies than chance does. Subgroup analysis and meta-regression can explore this, but only a limited number of study-level characteristics can be examined, and findings should be treated as hypotheses. Network meta-analysis adds the further requirement that the populations compared indirectly are sufficiently similar for the transitivity assumption to hold.

Unit of analysis

Cluster-randomized trials, crossover trials, split-mouth designs and trials with more than two arms cannot be entered as independent two-arm comparisons without adjustment. Unadjusted entry gives intervals that are too narrow, and the correction depends on information that is often not reported in the paper.

Existing reviews

Many topics already have several systematic reviews, some of them redundant or of low quality. Before a new review is planned, existing reviews are searched for, appraised and used to define what a new one would add. A well-justified update or a review with a new question is more useful than another review of the same trials.

How we support health research projects

Boundaries of this service

Evidence synthesis summarizes what published and registered studies show. It does not replace clinical judgment, and the results of a review do not tell an individual patient what to do. Developing clinical practice guidelines, which involves weighing benefits, harms, values and resources, is a separate process with its own methods and panels, and falls outside the scope described here. We also do not provide advice on the care of individual patients or on the interpretation of an individual's test results.

Where a question is better answered by a different design, such as a scoping review to map the evidence or a narrative synthesis because the studies cannot be pooled, we say so at the outset.

Frequently asked questions