MOOSE: reporting meta-analyses of observational studies

What MOOSE is

MOOSE is a checklist of items to report in a meta-analysis of observational studies, developed by a group of researchers, editors and methodologists and published in the Journal of the American Medical Association in 2000. It was written at a time when meta-analyses of observational studies were increasingly common and often poorly reported, and when the QUOROM statement, which covered meta-analyses of randomized trials, did not address the particular concerns of observational evidence. MOOSE lists, under six headings, the information that a report should contain: background, search strategy, methods, results, discussion and conclusions.

Observational studies differ from trials in ways that matter to reviewers. People are not randomized to exposures, so confounding and selection bias are central. Designs vary: cohort, case-control and cross-sectional studies answer questions differently and are analyzed differently. Exposures are measured in many ways and studies adjust for different variables. These features make the synthesis more difficult and the reporting more important, and MOOSE was designed to capture them. It remains in use, especially in epidemiology, alongside PRISMA. The wider family of guidelines is described in the guide to reporting guidelines.

MOOSE and PRISMA 2020

PRISMA 2020 is the current standard for systematic reviews and applies to systematic reviews of observational studies as well as trials. It covers the systematic review process in more detail and more recently than MOOSE, including the flow diagram, the certainty of evidence, registration and the sharing of data and code. Many journals therefore ask for PRISMA, and some ask for both PRISMA and MOOSE for meta-analyses of observational studies, on the view that MOOSE adds items specific to such studies. Where an author has completed PRISMA 2020, the main overlap with MOOSE is in the search, the selection, the data collection and the synthesis, and the items that MOOSE emphasizes that PRISMA treats less specifically are those concerned with confounding, with the heterogeneity of observational designs and with the description of exposure.

The practical advice is to follow the guidelines that the target journal requires, to use PRISMA 2020 as the main checklist for a systematic review, and to add MOOSE where it is requested or where the observational focus makes its items relevant. A single table that cross-references each item to the manuscript can serve both. See PRISMA 2020.

What the checklist covers

The table paraphrases the checklist and is not a substitute for it. Authors should work from the published checklist and the accompanying paper, and check for any update to it.

An example of the issues in a real-style question

Suppose a review asks whether long-term exposure to a dietary factor is associated with the risk of a disease, based on cohort and case-control studies. The report must say which designs were included and the minimum follow-up, and how the exposure was defined, for instance as servings per day, with the categories in each study. It must describe how the search captured studies, including unpublished ones. It must say which confounders, such as age, sex, smoking and total energy intake, the team considered essential, and show whether the pooled association changes when only the studies that adjusted for them are included. It must give heterogeneity statistics, explore them by design and region, and present sensitivity analyses that drop studies at high risk of bias. It must assess small-study effects, and state the limits of causal interpretation. Each of these statements corresponds to a MOOSE heading, and a reader who finds them can judge how much weight to give the result. The example is illustrative and not a description of a real review.

Why it was written

In the late 1990s, meta-analyses of observational studies were proliferating in epidemiology, and several high-profile ones produced conflicting results on the same question, such as the effects of dietary and hormonal exposures. Critics showed that the conflicts often arose from differences in the studies chosen, in how confounding was handled and in how estimates were combined, and that the reports often did not give enough information to find out. A workshop of researchers, editors and methodologists, held in 1997, produced the reporting proposal that became MOOSE. It was explicitly a proposal and was intended to improve reporting, not to endorse particular methods. It has been widely cited and adopted by journals in epidemiology. The newer guidelines have taken over some of its functions, but the underlying concerns about the reporting of observational evidence are as relevant as they were.

Confounding and adjustment

The most important difference from reviews of trials is confounding. In an observational study, the groups being compared differ in other ways that affect the outcome, and the study may or may not have adjusted for them. A meta-analysis that pools estimates adjusted for different sets of variables, or that mixes unadjusted and adjusted results, can yield a meaningless average. The MOOSE items on the assessment of confounding ask authors to say how it was handled. Good practice includes extracting both adjusted and unadjusted estimates, specifying in advance which confounders are essential for the question, and analyzing separately or exploring by subgroup the studies that adjusted for them and those that did not. It also includes preferring the most fully adjusted estimates when pooling and stating this. Residual confounding is always possible, and the conclusions should recognize it. The tools for assessing risk of bias in non-randomized studies, described in risk-of-bias tools, help structure this assessment.

Design heterogeneity

Observational meta-analyses often include several designs, and they have to be handled thoughtfully. Cohort studies give risk ratios or hazard ratios, case-control studies give odds ratios, and cross-sectional studies give prevalence ratios or odds ratios. Odds ratios from case-control studies approximate risk ratios when the outcome is rare, but not otherwise, and pooling the measures together requires an assumption. Different designs are subject to different biases: case-control studies to recall and selection bias, cohort studies to loss to follow-up and confounding. A MOOSE-conforming report states which designs were included and why, presents results by design or tests for differences between them, and explains how effect measures were converted or combined. This is one place where the item on stratification or regression on predictors of study results is used in practice.

Exposure measurement and definition

Exposures in observational research are measured in diverse ways: self-report, records, biomarkers, questionnaires of differing validity. Categories and cut-points differ, and units differ. A report should describe the exposure and how it was defined and measured in the included studies, and explain how differences were handled. For continuous or graded exposures, a dose-response analysis may be appropriate, in which case the methods in the page on dose-response meta-analysis apply. The measurement error in exposure usually attenuates associations, and misclassification can differ between cases and non-cases in case-control studies. The report should consider this when interpreting the results and in the discussion of bias.

Bias, heterogeneity and sensitivity

As in any meta-analysis, MOOSE asks authors to report how heterogeneity was assessed, to present sensitivity analyses and to assess publication bias. For observational studies, heterogeneity is generally high, because of differences in population, design, exposure measurement and adjustment, and a report should explore it by subgroup analysis or meta-regression and interpret it, not simply report a statistic. Sensitivity analyses typically exclude studies at high risk of bias, restrict to the most fully adjusted estimates, or vary the model. Publication bias is probably a greater concern for observational research than for registered trials, since observational studies are not typically registered, so the plot and test described in publication bias, and a search for unpublished work, are important. The certainty of evidence is then judged, usually low at the outset for observational studies, as explained in GRADE.

Using MOOSE in practice

1. Check what the journal requires

   Some name PRISMA, some MOOSE, some both. The instructions for authors will say.

2. Use the checklist at the planning stage

   Decide in the protocol how confounding, design differences and exposure will be handled, so that the report can describe them.

3. Record what the checklist will ask

   Keep a log of the search, the exclusions with reasons, the contacts with authors and the coding decisions.

4. Complete it against the manuscript

   For each item, give the location. Add anything missing, and mark items that do not apply with the reason.

5. State which guidelines were followed

   Say in the methods which checklists were used, including PRISMA 2020 and MOOSE, and supply them if the journal asks.

Limitations of MOOSE

MOOSE was published in 2000 and has not been updated in the way PRISMA has. Some items reflect older practice, such as the qualifications of the person who searched, and it lacks items for current needs such as the certainty of evidence, the registration of protocols and the sharing of data and code, all of which PRISMA 2020 covers. It addresses reporting only: it does not tell authors how to deal with confounding or heterogeneity, and a report that follows it can still describe a poor analysis. It is also a general checklist for observational designs and does not address the specifics of particular designs, which have their own guidance, for instance for prognostic studies, genetic association studies and dose-response analyses. For these reasons, it is best seen as complementing PRISMA 2020 for observational evidence.

Common mistakes

• Using MOOSE alone for a systematic review, without PRISMA 2020.
• Pooling adjusted and unadjusted estimates without distinction.
• Combining case-control odds ratios with cohort risk ratios without comment.
• Not describing how exposure was defined and measured in each study.
• Completing the checklist without changing the manuscript.
• Describing a MOOSE-compliant report as a good review, when MOOSE concerns reporting only.

Support

Frequently asked questions