Protocol development and registration

What a protocol is for

A review protocol is a written plan, made before the work begins, of what a review will do and how. It states the question, the criteria for including studies, the sources to be searched, how studies will be screened, what data will be extracted, how risk of bias will be assessed, and how the results will be synthesized. Its central purpose is to remove the freedom to change these decisions after the results are known. Without a protocol, it is easy, and often unintentional, to adjust criteria or analyses in ways that favor a preferred conclusion, and readers cannot tell whether that happened.

A protocol has practical benefits as well. It forces the review team to agree on the question and the methods at the start, when disagreement is cheapest to resolve. It shows editors and reviewers that the work was planned. It allows others to see that a review is under way and so to avoid duplicating it. And it provides the benchmark against which the finished review is compared, so that departures from the plan can be seen and explained. The systematic review service describes the stages that the protocol governs.

What the service includes

Defining the question and the criteria

The question is the core of the protocol, and the criteria that follow from it decide which studies the review will include. For an intervention question, the population, intervention, comparator and outcome are specified, together with the study designs that are acceptable. For other questions, such as prognosis, diagnostic accuracy or exposure, the frame changes, and the protocol uses the structure that fits. Each criterion is given a reason, since each can be challenged: why these ages, why this definition of the condition, why this follow-up period, why exclude studies in languages other than English.

Outcomes deserve particular care. The protocol should name the primary outcome or outcomes and the secondary ones, define how each is measured and at what time points, and say how multiple measures of the same outcome will be handled. Choosing the primary outcome in advance, and not after seeing which one gives a result, is among the most important protections against bias. Where outcomes are many, the protocol sets out which are most important to the users of the review, as the GRADE approach requires for the certainty of the evidence. See PICO and related frameworks.

Planning the methods

A complete protocol describes each stage with enough detail that another team could follow it. The search plan names the databases and registries, the date range and the supplementary methods, such as screening reference lists and contacting experts. The selection plan states whether two reviewers will screen independently and how a third will resolve disagreement. The extraction plan lists the data items and the form, which should be piloted. The appraisal plan names the risk-of-bias tool for each study design. The synthesis plan says whether a meta-analysis is intended, under what conditions, with which effect measure and model, and how heterogeneity will be examined.

The analysis plan within the protocol is where most of the protection against selective reporting lies. It lists the planned subgroup analyses, with the direction of any expected difference, the planned sensitivity analyses, the approach to missing data and dependent effect sizes, and the methods for assessing reporting bias. Analyses that are not in the plan can still be done, but they are reported as post hoc. The planned analyses are those readers can trust most.

Common weaknesses in protocols

Reviewers and editors who read protocols meet the same weaknesses repeatedly. Eligibility criteria are described in general terms and leave room to decide study by study, which defeats the purpose of having them. The primary outcome is not named, or several are named without saying which will drive the conclusions. The protocol says that a meta-analysis will be done "if appropriate" without saying what appropriate means, so the decision is left to the results. Subgroup analyses are listed without a rationale or a direction, which invites the search for any difference that appears. Dependent data, such as several outcomes or time points from one study, are not addressed. And the plan for assessing reporting bias is missing. Each of these can be fixed at the protocol stage at almost no cost, and each is much harder to fix once the data are in.

Protocols for different kinds of review

The same principles apply to reviews other than the classic intervention review, with differences in detail. A scoping review protocol sets out the objective and the concepts to be mapped and the sources searched, but it does not usually plan an appraisal of risk of bias or the pooling of effects, and guidance from the scoping review methodology groups and PRISMA-ScR applies. A rapid review protocol states which steps are shortened, such as a single screener with checking of a sample or a limited set of databases, so that the compromise is declared in advance. A living systematic review protocol adds the frequency of searches, the criteria for incorporating new evidence and the rules for retiring the review. A qualitative evidence synthesis protocol names the synthesis approach and the way the quality of studies and the confidence in findings will be assessed. An umbrella review protocol must say how overlap among the primary studies of the included reviews will be handled. The methods pages describe these designs.

Registration

Registration places the protocol in a public registry with a unique identifier. PROSPERO is the main international registry for systematic reviews with a health-related outcome, and it asks for a structured set of items drawn from the protocol. It does not accept every type of review, and its eligibility rules and timing requirements, such as how early in the review a record must be created, can change, so they are checked against the current guidance of the registry. Reviews outside its scope, including many in education, social science, management and the environment, can be registered on other platforms, such as open science repositories that support time-stamped registrations.

Registration serves several purposes. It makes the plan public and verifiable. It allows readers and reviewers to compare the published review with the plan. It reduces duplication of effort. And a growing number of journals ask for a registration number at submission. Changes after registration are normal but must be recorded as amendments with the date and the reason, and the final report states any departure from the protocol and why it was made. See the guide to PROSPERO registration.

Publishing the protocol

Some journals publish review protocols, which adds a peer-reviewed, citable record of the plan. Publication is optional, and registration alone satisfies the need for a public, time-stamped plan. Publishing makes sense when the journal considers protocols in scope, when the team wants the additional scrutiny, or when a funder or institution wants a peer-reviewed protocol. A protocol published in a journal should be consistent with the registered one, and the final paper cites both.

When to write the protocol

The best time is before any screening has begun, and ideally before the search, because decisions made after seeing the results are not protected by the plan. When a review is already under way, a protocol can still be written and registered where the registry allows it, but it must state truthfully at which point it was made and what had already been seen, and the report should be honest about this. A protocol written after the fact cannot offer the protection that a prospective one does, but it is still better than none, because it documents the logic of the methods.

The time needed to prepare a protocol depends on how well the question is already defined, how much scoping is needed, and how many team members must agree. A clear question with a straightforward design can be written up quickly, while a broad question that must be narrowed takes longer. After the first conversation, a schedule is agreed so that any deadline for registration, for example a funder's or a journal's, can be planned for.

Deliverables

Optional extension: from protocol to published review

Limitations and boundaries

A protocol does not guarantee a good review. It guards against one kind of problem, the unplanned and undisclosed change, and it does nothing about the quality of the search, the appraisal or the interpretation. A protocol that is too vague, or that leaves open the very decisions it should fix, offers little protection, and one that is rigid to the point of impracticality will have to be amended often. Registries have their own criteria and may decline a record, and a registration number is not an endorsement of the review's quality. Protocol registration does not make a review publishable, and no journal outcome is guaranteed.

Frequently asked questions