Meta-analysis and evidence synthesis for microbiology and immunology

Evidence synthesis in microbiology and immunology

Microbiology examines bacteria, viruses, fungi and other microorganisms; immunology examines how the body defends itself. Evidence syntheses in these fields serve different purposes: estimating the prevalence of resistance in a pathogen, comparing the immune response to vaccines, summarizing how an intervention changes the microbiome, reviewing animal studies before clinical trials, or assessing a candidate marker of immune protection. Some of this work overlaps with clinical research, covered under infectious diseases.

The evidence has features that call for care. Laboratory measurements depend on the assay, the laboratory and the reagent lot. Distributions of titers and microbial counts are skewed and often log-normal. Animal studies use small groups and may not predict human effects. Microbiome data are compositional, which means abundances are relative and not absolute. Our methods follow systematic review and meta-analysis practice, adapted to these laboratory and clinical features. This page builds on the general guidance for life sciences.

Assays, titers and measurement scales

Titers are analyzed as log values, and the ratio of geometric mean titers between groups is the usual effect measure, with pooling on the log scale. Values below the detection limit are censored, and simple replacement with half the limit can bias results when many values are censored. Where assays differ, standardizing to international units, when available, helps, and a review should record the assay, the laboratory and the units. Comparisons across assays should not be made without calibration.

Antimicrobial susceptibility and resistance prevalence

Prevalence of resistance is pooled with the methods in the prevalence meta-analysis page, using transformations that suit proportions and random-effects models. The key judgments concern the sampling frame. Isolates from hospital laboratories overrepresent severe cases and patients who have had previous treatment, which raises apparent resistance compared with community surveillance. Testing methods and breakpoint guidelines (for example from standards organizations) change over time, and the same isolate may be classed differently under different versions. A review should record the guideline and year, the specimen type, the setting and the country, and analyze resistance trends over time with care. Heterogeneity is usually extreme, and the pooled value is a summary of studies, not a figure for any particular hospital.

Vaccine immunogenicity and correlates of protection

Trials of vaccines report immune responses and, for some, efficacy against disease. A correlate of protection is a measurable immune marker that predicts protection from disease, and relatively few vaccines have established ones. Meta-analytic approaches relate treatment effects on a marker to treatment effects on a clinical outcome across trials, which is the stronger test of a surrogate than individual-level correlation. A review should state whether a marker is validated for the purpose and should not infer efficacy from immunogenicity alone. Differences by age, previous exposure, dosing schedule and vaccine type are examined as moderators. This service does not give vaccine recommendations or clinical advice.

Microbiome studies

Microbiome studies sequence the genes of microbial communities and report diversity and the abundance of taxa. Results depend on sampling, DNA extraction, sequencing target and depth, bioinformatic pipeline and reference database. Cross-study comparisons suffer from batch effects. Because the data are compositional, a rise in one taxon forces a fall in the relative abundance of others. Reviews can synthesize effect sizes for diversity measures, which are easier to compare, and for the direction of change in named taxa, while recognizing that agreement between studies on the taxa is modest. Reanalysis of raw sequences with a common pipeline is stronger than pooling published results, but it takes more time and requires available data. Causal claims from observational microbiome associations need caution, since disease, diet and medication all affect the microbiome.

Animal and in vitro studies

Systematic reviews of animal studies assess whether the findings support moving to human trials. They face known problems: small groups, lack of randomization and blinding in many reports, selective reporting, and models that imperfectly reflect human disease. Risk-of-bias tools designed for animal studies exist, and the reporting guideline ARRIVE sets expectations for primary studies. Effect sizes are usually standardized mean differences, pooled with random-effects models, and heterogeneity is large. We look for evidence of publication bias, which in preclinical research can inflate effects considerably, and we report how effects depend on study quality features such as randomization and blinding.

Publication bias and selective reporting

Positive laboratory results are more likely to be published, and many experiments are never reported. We search preprint servers, theses and conference abstracts, use funnel-based methods and compare studies with and without blinding or randomization. In vitro work has few formal reporting standards in many subfields, so reviews state which details were missing and how that affected inclusion.

Host factors and moderators

Immune responses vary with age, sex, nutrition, previous infection, co-infection and the drugs people take. Reviews of vaccine immunogenicity in older adults, in immunocompromised patients or in infants therefore code the characteristics of participants in detail, and analyze groups separately where numbers allow. Responses also change over time after vaccination, so the time of blood sampling is a basic moderator: a titer measured at four weeks and one measured at six months do not describe the same thing. Studies that follow antibody decline over time can be synthesized with meta-regression on time since vaccination, and the shape of the decline (linear on the log scale or not) is a modelling choice to be stated.

Pathogen factors matter as well. Strains differ between regions and years, and a response measured against one strain may not apply to another. We record strain and variant information when available and avoid pooling responses measured against different antigens without a moderator.

Surveillance and outbreak data

Surveillance reports and outbreak investigations provide counts and proportions that can be pooled, with the cautions that apply to routine data: reporting is incomplete, case definitions change, testing rates differ and the population at risk is not always known. Outbreak reports are written because something unusual happened, so they are not representative of infection in general. A review that pools attack rates or case fatality from outbreaks should say so, compute them with methods for proportions and examine heterogeneity by setting, age and period. Where surveillance data come from different countries, differences in health systems and testing explain a large share of the variation, and the review should show the numbers behind each country, together with the period covered and the case definition used by each source.

Genomic surveillance has become an important source, with sequences deposited in public databases. Syntheses of such data are mostly phylogenetic and epidemiological analyses by specialists rather than meta-analyses, and we describe their results without pooling them.

Common pitfalls we look for

• Pooling titers on the raw scale or across uncalibrated assays.
• Treating seroconversion as proof of protection.
• Combining hospital isolates and community surveillance in one resistance estimate.
• Ignoring breakpoint changes over time.
• Treating relative microbiome abundances as absolute.
• Extrapolating from animal models without noting their limits.

Planning a microbiology or immunology synthesis

We help define the organism or immune question, the measures and the comparison, plan searches in MEDLINE, Embase, Web of Science, bioRxiv and specialist sources, and set up coding of assay, laboratory, specimen type, breakpoint guideline, population and animal model features. See the systematic review service for scope and process.

An invented example of a titer ratio

Suppose a vaccine formulation produces a geometric mean titer of 320 and a comparator formulation 160 in an invented trial, a ratio of 2.0, with 95 percent confidence interval 1.4 to 2.9. On the log scale the difference is the natural log of 2, about 0.69, and the interval is symmetrical there. Pooling such ratios from several trials is done on the log scale, then converted back. A ratio of 2 means a doubling of the antibody level, which says nothing by itself about the clinical outcome unless there is a marker validated as predictive of protection, so the review would describe the finding as an immunogenicity difference and not as superior protection.

Coding and transparency

Coding frames record organism and strain, specimen, assay and laboratory, units, breakpoint guideline and year, population age and health status, vaccine or intervention details, schedule, time of sampling, animal species and strain, randomization and blinding, and funding. Two coders extract data independently on a sample, and when values are given only in figures they are extracted with digitizing software. The coded data and the analysis code are shared with the final report.

How we support research projects in this area

Boundaries of this service

A microbiology and immunology synthesis describes published laboratory, animal and clinical evidence. It does not provide diagnostic testing, clinical advice, treatment or vaccine recommendations, or laboratory protocols. Assays vary between laboratories and animal models may not predict human effects, so findings should be read with those limits in mind.

Frequently asked questions