Meta-analysis and evidence synthesis for climate change research

Evidence synthesis in climate change research

Climate change research draws on physics, ecology, economics, public health and social science. Synthesis takes many forms: the IPCC assessment reports are large expert syntheses; meta-analyses summarize, for example, how crop yields respond to warming, how mortality varies with temperature, or how species ranges have shifted; and systematic reviews examine whether adaptation and mitigation measures work. Evidence maps show where studies exist and where they do not, including the regions most vulnerable to impacts.

The evidence has particular features. Projections depend on emission scenarios and models. Impact studies often combine climate model output with a second model, so uncertainty accumulates. Observational studies of impacts compare years or regions with different weather, and the link to long-term climate change is an inference. Studies are concentrated in some regions. Our methods follow systematic review and meta-analysis practice, adapted to these features. This page builds on the general guidance for environment and sustainability. The service does not produce projections or climate advice.

Observations, projections and scenarios

A review should state which kind of evidence it covers and never combine them as if they were interchangeable. Projections are conditional statements: if emissions follow this scenario, then these changes are expected. They are not observations. Because climate models share components and data, an ensemble of 30 models is not 30 independent pieces of evidence, and averaging them does not give the uncertainty one would expect of independent estimates. Reviews of projections should describe the models and scenarios and avoid presenting a range as a probability interval unless the source does.

Response functions and warming levels

Many meta-analyses of impacts estimate the change in an outcome per degree of warming, such as the percentage change in yield of a crop per degree Celsius. Meta-regression on the amount of warming, with study type and region as moderators, can produce a response function. Non-linearity is common: effects of heat on crops and health are typically steeper above thresholds. Reviews should plot effects against the warming level, examine whether a linear approximation is adequate and report the range of warming covered by the studies. Extrapolating beyond that range is not supported by the data. Adaptation reduces impacts in some studies but not in others, and a review should separate studies that included adaptation from those that did not.

Health, agriculture and ecosystems

Studies of climate and health relate daily temperature to mortality or hospital admissions, with time-series and case-crossover methods, and show a J- or U-shaped relationship with higher risk at both ends. Meta-analyses of such location-specific curves use multilevel methods across cities and countries. Agricultural impact studies combine statistical and process models, and results differ by crop, region and treatment of carbon dioxide fertilization. Ecosystem impact studies report range shifts, phenological changes and extinction risk, with methods and data that vary. See the ecology and plant sciences pages for related methods. Reviews of health impacts connect with the public health area, and this service does not provide health advice.

Economic damages and the social cost of carbon

Estimates of the economic damage from climate change come from integrated assessment models and from empirical studies of weather and output. They differ widely, partly because of different assumptions about discount rates, damage functions and adaptation. Meta-analyses of the social cost of carbon have found large variation and evidence that results depend on modelling choices and on publication selection. A review should report the discount rate, the damage function and the treatment of uncertainty for each estimate. Differences in values reflect ethical choices as well as empirical ones, and a review should present them as such.

Adaptation and mitigation effectiveness

Evaluations of adaptation measures, such as early warning systems, cooling policies, drought-tolerant crops and flood defenses, and of mitigation measures, such as carbon pricing, efficiency standards and forest protection, vary in design and quality. Many adaptation studies are case studies with limited outcome data. Reviews of mitigation policy have found measurable but variable effects on emissions, depending on the design and the setting. A review should classify measures, outcomes and contexts, use evidence maps where studies are too few to pool, and report effects with their uncertainty. See the environmental economics and renewable energy and sustainability pages for related topics.

Geographic coverage and equity

Impact and adaptation research is concentrated in high-income countries and some large middle-income ones, while many of the most exposed regions have few studies. Evidence maps have documented this gap. A review should report the distribution of studies by region and avoid applying results from well-studied places to poorly studied ones. Vulnerability depends on income, governance and infrastructure, so the same warming can have different effects in different places, and averages across countries can hide the groups that are most affected.

Publication bias and attention

Studies reporting large or alarming impacts are more likely to be published in high-profile journals, and there is evidence of selective reporting in some literatures, such as the social cost of carbon. We use funnel-based and selection methods where they apply, compare high- and lower-profile journals and search grey literature, which includes many government and institutional assessments. The intense public interest in climate topics makes transparent methods especially important.

Detection, attribution and extreme events

Detection and attribution research asks whether observed changes lie outside natural variability and how much human influence contributed. Event attribution studies compare the likelihood or intensity of an extreme event in the present climate with a counterfactual climate without human influence, and report results as a change in probability or magnitude. These studies are conditional on models and on how the event is defined, such as the area and time window. A synthesis of attribution studies should note the event definition, the models and the method, and should be aware that events are chosen for study after they occur, often because they were severe, which affects the set of studies available. Pooling attribution results across events is possible only in a limited sense, and most reviews describe patterns by type of event, such as heat waves or heavy rainfall, where evidence is strongest.

Observational trend studies depend on the length and homogeneity of records. Changes in instruments, station location and urbanization can affect temperature series, and quality-controlled and homogenized datasets are used for this reason. Reviews should state the data source and the method of adjustment, and should not mix raw and adjusted series. Sea level, ice and ocean heat measurements come from satellites, tide gauges and floats, each with its own coverage and period, and a trend from one source is not interchangeable with a trend from another.

Common pitfalls we look for

• Treating projections as observations.
• Treating models in an ensemble as independent studies.
• Extrapolating response functions beyond the warming range studied.
• Mixing studies with and without adaptation.
• Applying results from high-income regions to vulnerable ones.
• Presenting economic damage figures without their assumptions.

Planning a climate change synthesis

We help define the exposure or measure, outcome and region, plan searches in Web of Science, Scopus, GreenFILE, CAB Abstracts, MEDLINE and EconLit, and set up coding of scenario, model, warming level, region, adaptation assumptions, outcome and study type. For mapping questions we follow guidance for systematic maps. See the systematic review service for scope and process.

An invented example of a response function

Suppose a meta-regression of 100 invented estimates gives an average yield change of minus 5 percent per degree of warming for a crop, with studies covering warming of 1 to 4 degrees. At 2 degrees the implied loss is 10 percent, and at 3 degrees 15 percent if the relationship is linear. If the data suggest steeper declines above 3 degrees, the linear extrapolation to 6 degrees (30 percent) would understate the loss, and in any case it is outside the studied range. If studies that include adaptation show half the loss, the average blends two different questions. The review would present response curves by adaptation status, state the range covered and avoid single numbers for a global yield change.

Coding and transparency

Coding frames record the type of evidence, scenario and model, period, warming level, region, outcome, adaptation assumptions, discount rate for economic studies, data source and funding. Two coders work independently on a sample, and the coded data and code are shared with the final report. Systematic maps are published with their full database so that others can query them.

How we support research projects in this area

Boundaries of this service

A climate change synthesis describes published evidence on climate, impacts, adaptation and mitigation. It does not produce climate projections for a location, adaptation plans, carbon accounting or policy recommendations. Projections depend on scenarios and models, and impact estimates depend on assumptions, so results should be read with their conditions.

Frequently asked questions