The Cochrane Handbook

What the Handbook is

Cochrane is an international organization that produces systematic reviews of the effects of health care interventions. It was founded in the early 1990s, and named in honor of the epidemiologist Archie Cochrane, who argued that health services should be based on rigorous evidence from randomized trials, organized by specialty and kept up to date. Its reviews are published in the Cochrane Database of Systematic Reviews. To make the reviews consistent and rigorous, Cochrane developed detailed methods, and the Cochrane Handbook for Systematic Reviews of Interventions is where they are set out.

The Handbook is written by methodologists and review authors and is revised periodically. It covers every stage of a review: framing the question, searching, selecting studies, collecting data, assessing bias, analyzing data and synthesizing results, assessing the certainty of the evidence, and interpreting findings. Although it was written for Cochrane authors, it has become the standard methodological reference for systematic reviews of interventions in general, and many journals, guideline groups and universities expect authors to follow it. Much of the guidance in this site's pages on systematic review and meta-analysis is drawn from it.

How it is organized

The current edition is organized in two broad parts. The first, core methods, follows the sequence of a review and is the part most authors read from start to finish. It begins with starting a review and determining its scope, and goes on to defining the criteria for including studies, searching for and selecting studies, collecting data, choosing effect measures, considering bias, assessing risk of bias in randomized trials, summarizing study characteristics, analyzing data and undertaking meta-analyses, undertaking network meta-analyses, synthesizing findings by other methods, assessing risk of bias due to missing results, preparing summary of findings tables and grading the certainty of evidence, and interpreting results and drawing conclusions.

The second part addresses specific perspectives and types of evidence. It includes chapters on equity, complex interventions, patient-reported outcomes, adverse effects, economic evidence, qualitative evidence, approaches to accumulating evidence such as living reviews, variants of randomized trials, non-randomized studies, and individual participant data. Chapter numbers and titles have changed between editions, so the table of contents of the current version should be consulted. The guidance in each chapter is accompanied by boxes of key points, worked examples and references to the underlying literature.

Using the Handbook when planning a review

1. Read the early chapters before the protocol

   The chapters on scope, criteria and searching set out the decisions that should be made, and recorded, before the work starts.

2. Use the chapter that matches the task

   When extracting data, the chapter on collecting data; when choosing an effect measure, the chapter on effect measures; and so on. Each is self-contained.

3. Follow the recommendations on bias

   The chapters on bias set out the domain-based approach, with the tools described in the guide to risk-of-bias tools.

4. Plan the analysis with the analysis chapters

   The chapter on analysis covers effect measures, models, heterogeneity and sensitivity analysis, and the chapter on missing results covers reporting bias.

5. Use the certainty chapter for conclusions

   It explains how to prepare summary of findings tables and to word conclusions in line with the certainty of the evidence. See GRADE.

MECIR and the standards

The Handbook is guidance. For Cochrane reviews, a set of Methodological Expectations of Cochrane Intervention Reviews, known as MECIR, translates it into standards that authors must meet or justify not meeting. It lists conduct standards for each stage, from the protocol to the interpretation, marking each as mandatory or highly desirable, and reporting standards that parallel the PRISMA items. Authors of non-Cochrane reviews often use MECIR as a checklist for planning because it is concrete and specific. Cochrane editors use it in editorial checks. The reporting of reviews follows PRISMA 2020, and the two sets of expectations are broadly aligned.

How to cite it

The Handbook should be cited as the specific edition and version used, since the content changes. The usual form gives the editors, the title, the version number and its date, the publisher, and the web address. Where a particular chapter is the source of a statement, the chapter authors and title are cited, with the Handbook as the containing work, which is the practice followed in the references to this site's pages. Because there is also a print edition, which may differ from the current online version, authors should say which they used. Including the date of access for the online version is a further courtesy to readers when the version has been revised since. Statements attributed to the Handbook, such as the recommended thresholds, should be checked against the version cited.

Beyond Cochrane: other handbooks and manuals

Cochrane Handbook for Diagnostic Test Accuracy Reviews

The Cochrane guidance for reviews of diagnostic accuracy, with its own methods for bivariate models and QUADAS-2.

JBI Manual for Evidence Synthesis

Methods from the Joanna Briggs Institute for several review types, including scoping, qualitative, prevalence and umbrella reviews.

Campbell Collaboration guidance

Methods for reviews of social, educational and criminal justice interventions.

Guidance for undertaking reviews in health care

From the Centre for Reviews and Dissemination at the University of York, a long-standing source for reviews of effectiveness and others.

AHRQ Methods Guide

Guidance from the US Agency for Healthcare Research and Quality for comparative effectiveness reviews.

Collaboration for Environmental Evidence guidelines

Methods for systematic reviews and maps in environmental science.

These sources differ in emphasis but agree on the main principles. The choice depends on the type of review and the field, and the review should say which was followed. See choosing a review type.

The principles it teaches

Beyond the technical detail, the Handbook conveys a set of principles that run through every chapter. Plan before you look: the question, criteria and methods are specified in a protocol before the results are known. Be comprehensive and transparent: search widely, record what was done, and report in a way that lets others repeat it. Minimize error and bias in the process: use more than one person for the steps that involve judgment, and pre-specify the outcomes and analyses. Assess the studies critically: examine risk of bias by domain, outcome by outcome, and use the assessments in the synthesis. Combine results with care: choose a model for a reason, look at heterogeneity and do not hide it, and test the robustness of the findings. Interpret in line with the certainty of the evidence: say what is known, what is uncertain and what the implications are, without overreaching. These principles, more than any single recommendation, explain why a review that follows the Handbook can be trusted, and why reviewers of manuscripts look for them.

Editions and updates

The Handbook has been revised several times, and the changes matter. Edition 5 appeared in the late 2000s and edition 6 in 2019, with later updates to the online version, and the revisions have brought major changes: new risk-of-bias tools, guidance on network meta-analysis, summary of findings tables, prediction intervals, living reviews, and equity. Older advice, for example on thresholds for heterogeneity, or on assessing quality with the earlier tool, has been replaced, and some reviews and textbooks still repeat it. When a reviewer or a journal asks for adherence to the Handbook, the question is which version, and authors should use the current one unless they have a reason, stated in the protocol. A change log is available on the Handbook website, and checking it before finalizing the methods is a sensible step, as is checking that the software used implements the current guidance.

Training and support from Cochrane

Cochrane provides learning resources alongside the Handbook, including online modules and courses on the stages of a review, on specific tools such as the risk-of-bias tool for trials, and on reporting. These materials are freely accessible and are widely used by students and new reviewers. Cochrane review groups and national or regional centers offer support for those preparing Cochrane reviews. For reviews outside Cochrane, the Handbook and the learning resources are still valuable, with the proviso that Cochrane-specific editorial requirements do not apply. Authors who need hands-on help with a review in a particular field, with the planning, the search, the analysis or the reporting, can use the services described on this website, which follow the Handbook's principles. See the process page.

What the Handbook does not cover

The Handbook concerns systematic reviews of interventions, and mostly randomized trials. It is less complete for other questions, such as prognosis, prevalence, exposure and qualitative questions, although recent editions include chapters on some of these. It describes methods for reviews done by teams with considerable resources, and some recommendations, such as independent duplicate extraction of all data, are demanding. It is guidance and not a law, and authors may depart from it with reasons. It does not replace statistical advice, because the choice of method in a particular analysis depends on the data. And as methods develop, parts of it fall behind, so authors should read it alongside recent methodological literature. The guide on PRISMA 2020 covers the reporting standard that accompanies it.

RevMan and the review workflow

Cochrane reviews are prepared with software that supports the whole workflow, including a protocol and review template, structured data entry, meta-analysis and the production of forest plots and summary of findings tables. The meta-analysis software offers the standard methods described in the Handbook, with choices of effect measures and models, and creates the standard figures. Authors of non-Cochrane reviews can use it or other programs, such as the R packages described in the pages on methods. The software does not replace judgment: the Handbook's guidance on choosing models and interpreting results applies whichever program is used, and the settings should be reported. See meta-analysis.

Common mistakes in using the Handbook

• Citing it without a version, or quoting a threshold from an older edition.
• Treating its guidance as mandatory outside Cochrane, or as optional inside.
• Using it for review types it does not address, such as prevalence or qualitative reviews, without other guidance.
• Following the chapter on analysis without reading the one on interpreting results.
• Ignoring MECIR when planning, then being asked about it at peer review.
• Reading a single chapter in isolation and missing the guidance in the preceding ones.

Support

Frequently asked questions