Software for meta-analysis

What to consider when choosing

No program is best in every case. The decision rests on five questions.

• Which methods do you need? A standard pairwise analysis of a handful of studies is within reach of every tool. Network meta-analysis, multivariate and multilevel models, dose-response, diagnostic accuracy, Bayesian models and robust variance estimation are available in fewer.
• How important is reproducibility? Scriptable tools allow the analysis to be rerun and checked. Menu-driven tools are easy to learn but leave no record of the steps unless you create one.
• What can the team use? A tool that nobody on the team understands produces results that nobody can check.
• What are the cost and licence conditions? Free and open-source tools lower barriers. Commercial software may be provided by institutions.
• Is the tool maintained and validated? Check for recent updates, documentation, a published description and comparisons with other programs.

The main options

The table lists commonly used tools. Features, licensing and maintenance change, so check the current documentation of each before choosing.

R

R is the most widely used environment among methodologists, because new methods are usually released as R packages first. The metafor package by Viechtbauer offers a very broad set of models: fixed-effect and random-effects, many estimators of between-study variance, the Knapp-Hartung adjustment, meta-regression, multilevel and multivariate models, influence diagnostics and a range of plots. The meta package by Schwarzer is oriented toward a simpler interface and produces common plots and summaries with short commands. The netmeta package performs frequentist network meta-analysis, and gemtc and others do Bayesian network meta-analysis through JAGS. Packages exist for diagnostic accuracy (mada, meta4diag), dose-response (dosresmeta), robust variance estimation (robumeta, clubSandwich) and publication-bias methods. The dmetar companion to a widely used online guide collects helper functions.

The cost of R is the learning curve, and the need to document the environment, because results can vary with package versions. The advantage is complete transparency and reproducibility, since the whole analysis is a script.

Stata, RevMan, CMA and the rest

Stata has a built-in meta suite, as well as long-standing user-written commands such as metan, metareg, mvmeta and the network suite. It is popular in epidemiology and health economics, produces publication-quality graphs and is scriptable.

RevMan is the software of Cochrane. It is designed for reviews of interventions in the Cochrane format and produces forest plots, risk-of-bias figures and summary of findings tables. Its range of models is limited, and it is not suited to network or multivariate analysis. RevMan Web is the current platform for Cochrane authors.

Comprehensive Meta-Analysis is commercial software with a spreadsheet-like interface, and it has been popular in psychology and education. It is easy to start with, but it offers fewer advanced options and does not keep a script by default.

JASP and jamovi are free graphical programs that wrap R packages, offering a path for people who do not code, with more transparency than many menu-driven tools because the underlying methods are documented.

Spreadsheet add-ins and web apps are convenient, but they vary in quality, and some are no longer updated. They should be checked against a trusted program before use for a published analysis.

Defaults that change results

Different programs can give different answers to the same data, and the cause is often a default that the user did not choose. Common sources of difference include the following.

• The estimator for between-study variance. DerSimonian-Laird is the default in some programs, REML in others.
• The confidence interval method. Wald-type intervals or the Knapp-Hartung adjustment.
• Continuity corrections. Whether, when and by how much a constant is added to zero cells.
• Handling of double-zero studies. Excluded or included with a correction.
• The effect scale. For example, whether odds ratios are pooled on the log scale and whether the Mantel-Haenszel or inverse-variance weights are used.
• Heterogeneity statistics. Formulas for I-squared and its interval.
• Handling of multi-arm trials. Whether the correlation is considered.

The practical lesson is to set every option explicitly, record it, and report it. Running the primary analysis in a second program, or with a second set of options, is a useful check that also reveals how sensitive the result is to the choices.

Validating your results

Programs have bugs, and users make mistakes. Several habits catch both. Reproduce a published meta-analysis with known results, such as the examples distributed with the packages or the textbooks, before running your own. Check a few study weights by hand: they are simple to calculate, and a mismatch shows a misunderstanding. Compare the output of two programs for the same model. Plot the data, and make sure the forest plot matches the extraction sheet. Examine the log or the summary output for warnings, such as non-convergence or dropped studies. Keep the check in the project record, so that a reviewer can see it was done.

AI coding assistants and software

Large language models can write analysis code, and many analysts use them. They can speed up routine tasks, but they also produce code that looks right and is wrong, such as an incorrect argument name, a wrong direction of effect or a default that differs from what was intended. Treat generated code as a draft to be tested, check each result against a manual calculation or a known example, and keep to the journal's policy on disclosing the use of AI tools. The responsibility for the analysis remains with the authors.

Reporting the software

A methods section should name the software and its version, the packages and their versions, the model and the options used (estimator, interval method, corrections), and the way the results can be reproduced, for example by sharing the script. A sentence such as "Analyses were conducted in R version 4.x using the metafor package version 4.x, with a random-effects model estimated by restricted maximum likelihood and the Knapp-Hartung adjustment" is the minimum. Cite the package, since developers of free software depend on citations. The PRISMA 2020 checklist asks for the software used in the synthesis.

Software by type of review

• Pairwise meta-analysis of trials. Any of the tools above. RevMan or meta in R give familiar outputs quickly, and metafor gives the most control.
• Network meta-analysis. netmeta (frequentist) or gemtc and multinma (Bayesian) in R, the network suite in Stata, or the MetaInsight web application for a first look. Check how multi-arm trials and consistency are handled.
• Diagnostic test accuracy. The bivariate and HSROC models, in R packages such as mada, in Stata's midas and metandi commands, or in the Bayesian approaches. RevMan supports a simpler analysis.
• Dose-response. dosresmeta in R, or the glst command in Stata, with restricted cubic splines for non-linear patterns.
• Multilevel and dependent effect sizes. metafor with random effects at several levels, robumeta and clubSandwich for robust variance estimation.
• Individual participant data. One-stage models in general-purpose software such as R packages for mixed models, or Stata, with careful attention to clustering by study.
• Prevalence and single-arm data. The metaprop function in meta, or generalized linear mixed models in metafor.
• Qualitative evidence. NVivo, ATLAS.ti and similar tools support coding in thematic synthesis, and EPPI-Reviewer supports mixed-methods work.

Figures and tools around the analysis

The analysis is only one step in a review. Screening tools such as Covidence, Rayyan and DistillerSR, reference managers such as Zotero and EndNote and data-extraction tools shape the quality of the data that goes into the program. Software for digitizing figures, such as WebPlotDigitizer, can recover numbers from graphs, with the usual caution. Forest plots, funnel plots and risk-of-bias plots can be produced directly by the analysis packages. Tools such as the robvis package create risk-of-bias figures, and the PRISMA2020 flow diagram generator creates the flow diagram. The point is to choose a chain of tools that passes data without retyping, since each manual transfer is a chance for error.

Learning and support

For people new to R, the online guide by Harrer and colleagues and the documentation of metafor are good starting points, and the Cochrane Handbook explains the methods behind the options. University statistics services and training from evidence-synthesis groups are useful, as are the user communities around each package. When a model is unfamiliar, work through a published example whose results are known before moving to your own data. If a team does not have the skills for the methods the question requires, bringing in a statistician early is better than discovering the gap at the peer-review stage, when fixing it means redoing the analysis.

How we can help

We run analyses in R, Stata and Bayesian software, choose methods suited to the data, check results across programs and deliver scripts that others can rerun. [OWNER VERIFICATION REQUIRED] The relevant services are statistical analysis, meta-analysis and network meta-analysis.

Frequently asked questions