Bayesian meta-analysis service

What a Bayesian meta-analysis is

A Bayesian meta-analysis uses the same data as a conventional one, namely the effect estimates and their standard errors from each study, but it treats the unknown quantities, such as the average effect and the between-study variation, as having probability distributions. The analyst states a prior distribution for each unknown, which expresses what is believed about it before seeing the studies. The data update the prior through the likelihood, and the result is a posterior distribution that describes what is believed after seeing them.

The output is richer than a point estimate and an interval. The posterior distribution of the effect allows direct statements, such as the probability that the effect is beneficial, the probability that it exceeds a clinically meaningful threshold, or the probability that a treatment is the best of several. The interval reported is a credible interval, which, given the model and priors, contains the true effect with the stated probability. This is the interpretation many readers wrongly give to frequentist confidence intervals, and it is one reason the approach is attractive. The method page on Bayesian meta-analysis gives the general outline.

When a Bayesian approach is worth using

A Bayesian analysis is not better by default, and with many studies and weak priors it gives results close to a frequentist random-effects analysis. It earns its additional complexity in specific situations.

• Few studies. With two to five studies, the between-study variance cannot be estimated well from the data alone. An informative or weakly informative prior for it can stabilize the analysis and propagate the remaining uncertainty into the result.
• Rare events and sparse data, where conventional methods struggle with zero cells and an exact model can be fitted directly to the counts.
• Prior evidence that should count, such as earlier trials, evidence from a related population, or expert judgment, where its use is justified and transparent.
• Complex models, including network meta-analysis, multilevel and multivariate models, where Bayesian software handles the dependence structure flexibly.
• Decision-oriented questions, where probabilities of exceeding thresholds or of ranking are more useful than a significance test.

It is less suited to settings where the priors would be contested and no consensus on them can be reached, or where the audience will not accept prior assumptions. In those cases, a conventional analysis with a Bayesian one as a sensitivity analysis is often a sound compromise.

What the service includes

Choosing priors

The prior is the most debated part of a Bayesian analysis, and the right response is to make the choice explicit, justified and tested. For the average effect, a weakly informative prior, centered on no effect and wide enough to include any plausible value, lets the data dominate. For the between-study variation, the choice matters much more, particularly with few studies, because the data contain little information about it.

Conventional vague priors for the heterogeneity standard deviation, for example a very wide uniform or an inverse-gamma on the variance, can be unintentionally influential when studies are few. Weakly informative alternatives, such as half-normal or half-Cauchy distributions on the standard deviation, are widely recommended. Where there is relevant empirical information, such as distributions of heterogeneity found in collections of published meta-analyses for the type of outcome and comparison, it can be used to build an informative prior. Whatever is chosen, it is reported in full and the analysis is repeated with alternatives. See the guide on choosing priors in Bayesian meta-analysis.

Models and software

The standard model is the Bayesian counterpart of the random-effects model: the study effects are drawn from a distribution around an overall mean with a between-study standard deviation. Extensions cover binary data analyzed on the count scale with a binomial likelihood, which avoids continuity corrections, as well as network, multivariate and meta-regression models. Software commonly used for such models includes R packages such as bayesmeta and brms, and general-purpose engines such as Stan and JAGS. The software, version and settings are recorded and reported.

Fitting is by simulation, so the result must be checked to confirm that the chains have converged to the target distribution. Standard checks include running several chains from different starting values, examining trace plots, the potential scale reduction factor (R-hat), the effective sample size and, with Hamiltonian Monte Carlo, the absence of divergent transitions. A model that has not converged gives numbers that look plausible and are wrong, so these diagnostics are reported along with the results.

Sparse data and rare events

Rare events are a common reason to choose a Bayesian model. Conventional inverse-variance meta-analysis needs an effect estimate and variance from every study, so studies with zero events in one or both arms must be given a continuity correction or dropped, and both choices can change the answer. A Bayesian model can be fitted directly to the event counts with a binomial likelihood, which uses every study as it was reported, without adding artificial counts. The same logic applies to outcomes measured as counts or rates.

The price is that the result is more sensitive to the priors, because there is little information in the data. For that reason an analysis of sparse data should always be accompanied by the prior sensitivity analysis and by a comparison with an exact frequentist method such as a Mantel-Haenszel or Peto estimate where one applies. Agreement between approaches increases confidence in the result, and disagreement is reported and explained.

How long does a Bayesian analysis take?

The time depends on the number of studies, the complexity of the model, the number of outcomes, and how many prior scenarios are to be explored. Fitting itself can be quick for a standard model and slow for a large network, and the planning of priors and the checking of convergence take more time than the fitting. After a feasibility check, the schedule and its dependencies are agreed so that any fixed deadline can be planned for.

Reading the posterior

Results are reported as the posterior median or mean with a 95 percent credible interval. For the heterogeneity parameter, the posterior shows how much, or how little, the data can say about it, and with few studies the interval is often wide, which is itself an honest finding. The predictive distribution for the effect in a new study corresponds to a prediction interval in a frequentist analysis and is often more informative than the interval for the average.

Probability statements are among the most useful outputs and should be worded carefully. The probability that the effect exceeds zero depends on the prior as well as the data, so it is reported alongside the prior sensitivity analysis. Statements about exceeding a threshold of clinical importance, such as a minimal important difference, are particularly useful when the threshold is agreed in advance.

Sensitivity to the prior

A Bayesian analysis should show how much its conclusions depend on the prior. The analysis is repeated with several defensible priors, for example a weakly informative prior, a more diffuse one, and a skeptical or enthusiastic prior for the effect, and with different priors for heterogeneity. If the conclusions hold across them, the data are doing the work. If they change, the report states plainly that the result is prior-dependent and gives the range. This is not a weakness of the method but a feature: a conventional analysis hides the same dependence in its choice of model and estimator.

Reporting

A Bayesian meta-analysis is reported against PRISMA 2020 for the review as a whole and with additional detail on the Bayesian elements. The report gives the model and likelihood, every prior with its justification, the software and settings, the number of chains and iterations, the convergence diagnostics, the posterior summaries, and the results of the prior sensitivity analysis. Many journals are less familiar with Bayesian methods, so the methods section is written to be understood by a reader without specialist training, and where it helps a conventional analysis is shown alongside for comparison. Code and data are shared to allow reproduction. See reproducible meta-analysis.

Deliverables

Optional extension: full manuscript and submission

Limitations

The results of a Bayesian meta-analysis depend on its priors, and informative priors can be seen as injecting opinion into the analysis. The remedy is transparency and sensitivity analysis, not avoidance. With very few studies, no method can recover information that the data do not contain, and the posterior for heterogeneity will largely reflect the prior. Fitting is computationally more demanding and requires diagnostic checks that a conventional analysis does not. Reviewers and readers may be unfamiliar with the approach, so the explanation has to be clear. As with any meta-analysis, the quality of the included studies limits the conclusions, and the analysis does not correct bias shared by all of them.

Frequently asked questions