Meta-analysis and evidence synthesis for health economics

Evidence synthesis in health economics

Health economics studies how resources are used to produce health and how to compare the costs and effects of alternatives. Evidence syntheses in the field serve several purposes: summarizing published economic evaluations of an intervention, pooling inputs such as costs or health-state utilities for a model, reviewing the methods used in evaluations, and examining whether results transfer between countries. These tasks call for different approaches, and a review should state which it is doing.

Economic evidence is context-bound. Costs depend on prices, wages and organization of care, so a cost from one country cannot be applied in another without adjustment. Cost data are skewed and have a few very high values. Cost-effectiveness ratios are unstable when the effect difference is near zero. Our methods follow systematic review and meta-analysis practice, adapted to these features. This page builds on the general guidance for economics and should be read with the page on medical and health sciences.

Different questions, different syntheses

A common mistake is to pool incremental cost-effectiveness ratios from several studies. A ratio is a quotient of two uncertain quantities, can be negative in meaningful ways (a cheaper and more effective option is not the same as a more expensive and less effective one), and depends on the comparator and the setting. It is more informative to tabulate ratios with their context, or to pool costs and effects separately and compute net benefit.

Synthesizing costs

Costs are measured in currency units that differ by country and year. To combine them, reviews convert to a common currency and price year, using purchasing-power-parity exchange rates and a consumer or health-sector price index, and state the method. These adjustments assume that the relative prices of goods are similar, which is not always true. Costs also differ in what they include: drug acquisition, hospital stays, outpatient visits, productivity losses, informal care. A review should specify the perspective (health system, payer, societal) and code the cost components in each study, pooling like with like.

Cost distributions are right-skewed. Studies may report arithmetic means, medians or geometric means, and means are the relevant quantity for budget decisions. Methods for estimating means from medians are crude for skewed data. We prefer studies that report means with standard errors and consider ratio-of-means measures when costs differ in scale between settings.

Health-state utilities and quality-adjusted life years

Utilities express health-related quality of life on a scale where 1 is full health and 0 is death, and are used to compute quality-adjusted life years (QALYs). Values depend on the instrument and the set of preference weights that translate answers into scores, and on the population that provided the weights. EQ-5D values differ between countries because of different value sets. A review of utilities should record instrument, version, value set, mode of administration and respondent group, and should be cautious about pooling across instruments. Meta-regression can adjust for instrument, though few instruments per condition limit this. Pooled utilities often feed decision models, where their uncertainty should be passed on through probabilistic analysis.

Transferability across settings

Whether the result of an evaluation in one setting applies in another depends on differences in disease burden, practice patterns, prices, population and the comparators available. Checklists exist to help assess transferability, and models can be re-run with local inputs, which is usually more reliable than adjusting a ratio. A review should extract the information needed to judge transferability, such as the comparator, the country, the perspective and the price year, and should not recommend that a result be applied elsewhere. Meta-regression of cost-effectiveness across countries is limited by the small number of settings and by confounding with study methods.

Quality and reporting of economic evaluations

The CHEERS reporting guideline describes what an economic evaluation should report, and checklists such as the Drummond list and the Philips checklist for models are used to appraise quality. Common concerns are weak clinical inputs, incomplete costing, unstated perspective, inadequate handling of uncertainty and sponsor involvement. Industry-sponsored evaluations have been found, on average, to report more favorable ratios, though the evidence varies. We record sponsorship and examine whether it relates to results. Risk-of-bias tools for trials and observational studies are applied to the clinical effectiveness inputs, which determine much of the result.

Model-based evaluations

Many economic evaluations are decision-analytic models in which results depend on assumptions about transition probabilities, time horizon, discount rates and extrapolation beyond trial data. A review of models should tabulate structure, horizon, sources of inputs and uncertainty analysis, and compare how structural choices change conclusions. Models cannot be meta-analyzed like trials, since their outputs are not independent observations. Sensitivity of results to extrapolation is often the main driver of differences, and a review should flag it.

Policy evaluations and quasi-experimental evidence

Health policy studies, such as insurance expansions, price regulation or payment reforms, use designs like difference-in-differences, regression discontinuity or interrupted time series. They give effects on use, spending and outcomes, and can be pooled when they estimate the same quantity. Reviews should check the design assumptions, such as parallel trends, and report effects on a common scale such as percentage change in spending. Results from one country's system often differ from another's, so we report by setting.

Publication bias and selective reporting

Economic evaluations that find an intervention cost-effective may be more likely to be published, and the time-lag between trial and evaluation can also hide unfavorable results. Funnel plots of costs and effects can be examined, though ratios are poorly suited to them. Registries of economic analyses are rare, so the main safeguard is a wide search that includes gray literature and health technology assessment reports, which often contain more methodological detail than journal articles.

Handling uncertainty and distributions

Economic results carry uncertainty from sampling, from model structure and from the choice of inputs. In trial-based evaluations, uncertainty in costs and effects is usually summarized by bootstrapping or by a cost-effectiveness acceptability curve, which shows the probability that an option is cost-effective at different thresholds. These curves describe a single study and cannot be averaged across studies as if they were probabilities of the same event. In a review, the right step is to extract the information needed to judge the strength of each result: the sample size, the confidence interval for the cost difference, the confidence interval for the effect difference, and the threshold used.

When pooled inputs feed a decision model, the pooled mean and its standard error are used to define a distribution for probabilistic analysis. Costs are usually given gamma distributions, utilities beta distributions and probabilities beta or logistic-normal distributions, with parameters chosen to match the pooled mean and variance. The reviewer should describe the distributional assumption and check that the model results do not depend on it unduly. A random-effects pooled mean with a wide prediction interval is a better description of what the model might meet in a new setting than a narrow confidence interval for the average.

Reviews should also report missing data. Economic evaluations often have incomplete cost data, and the way it was handled, such as complete-case analysis or multiple imputation, can change results.

Common pitfalls we look for

• Pooling cost-effectiveness ratios across settings.
• Combining costs without common currency, price year and perspective.
• Mixing utilities from different instruments and value sets.
• Ignoring sponsorship of evaluations.
• Applying results from one country to another without local inputs.
• Treating model outputs as independent observations.

Planning a health economics synthesis

We help define the question (review of evaluations, pooling of cost or utility inputs, or a methods review), the perspective, the comparators and the price year. We plan searches in medical and economic databases, such as MEDLINE, Embase, EconLit and the NHS Economic Evaluation Database archive, plus health technology assessment sources, and set up data extraction for costs, resource use, utilities and methods. See the systematic review service for scope and process.

Converting and adjusting costs: an invented illustration

Suppose one study reports a hospital stay cost of 4,000 in local currency in 2015, and a second reports 6,500 in another currency in 2020. Both are converted to a common currency using a purchasing-power-parity rate and to a common price year using a price index. Assume, for illustration, that the first becomes 5,100 and the second 6,900 in the chosen units. The apparent difference of 2,500 shrinks to 1,800 after adjustment. The remaining difference could reflect real differences in practice or prices, or the cost components included. The review should show raw and adjusted values side by side and state which factors it could not adjust for.

Such adjustment is a modelling step, not a fact, and its uncertainty adds to that of the study estimates. Sensitivity analyses with alternative exchange rates and indices show whether conclusions depend on the choice.

How we support research projects in this area

Boundaries of this service

A health economics synthesis describes published evidence on costs, utilities and cost-effectiveness. It does not provide reimbursement or pricing recommendations, health technology assessment submissions or advice on any treatment, and it does not replace a decision-analytic model built for a specific setting. Economic results depend on local prices and practice, and clinical effectiveness inputs carry their own uncertainty.

Frequently asked questions