Meta-analysis in pharmacy

Evidence synthesis in pharmacy

Pharmacy covers the science of medicines and the services that deliver them. Two kinds of research dominate. The first concerns drugs: efficacy in trials, safety in trials and after marketing, comparisons between agents in the same class, and the influence of dose, formulation and interactions. The second concerns services, such as medication review by pharmacists, adherence support, antimicrobial stewardship, smoking cessation and the management of chronic conditions in the community. Reviews in both areas use the same methods as other health sciences but face particular problems.

Drug evidence is shaped by regulation and sponsorship. Pivotal trials are run by manufacturers, regulators hold data that are not in journal articles, and the standard for approval differs from the standard for clinical decisions. Rare adverse events are not detected in trials of a few thousand people and need pharmacovigilance and large observational data. Service evidence is more heterogeneous, with complex interventions, variable delivery and outcomes that are indirectly linked to the pharmacist's action.

Our methods are described under meta-analysis and systematic review, with the adaptations below.

Rare adverse events and sparse data

Meta-analysis is often the only way to assess uncommon harms, but the data are sparse. Many trials report zero events in one or both arms. Consider a safety outcome with 3 events among 1200 patients, a proportion of 0.0025, or 0.25 percent. The numbers are invented. A single trial with such rates cannot show a difference between arms, and trials that report no events provide no information to methods that rely on a continuity correction, while other methods can handle them. Peto's odds ratio works well when events are rare and groups are balanced in size and the effect is small, but it is biased when groups are unbalanced or effects are large. Mantel-Haenszel methods without continuity correction include trials with an event in only one arm and drop trials with none, and beta-binomial and other generalized linear mixed models use all data. The guide on zero-event studies explains the options, and the choice should be stated in the protocol and tested in a sensitivity analysis.

Adverse events in trials are often collected without specific definitions, and published reports may list only those that exceed a frequency threshold. Reviews should compare the publication with the registry and, where possible, with regulatory documents. Serious adverse events, discontinuation because of adverse events and specific events of interest should be analyzed separately, and the exposure time should be taken into account when treatment durations differ.

Sponsorship, unpublished data and regulatory documents

Studies of drugs show a sponsorship effect: trials funded by manufacturers more often report favorable results and conclusions than independent trials, even when the quality of design appears similar. Reasons include the choice of comparator and dose, the selection of outcomes and analyses, and selective publication. Reviews should record the funding and the role of the sponsor, and test whether results differ. Clinical study reports and regulatory reviews, available from agencies such as the European Medicines Agency and the United States Food and Drug Administration, provide far more detail than journal articles and sometimes show different results for efficacy and harms. Their use takes time but improves the reliability of reviews of drugs with a history of selective reporting.

Registries of trials allow comparison between planned and reported outcomes. Where a primary outcome was changed, the review should note it and consider the risk of bias in the selection of the reported result.

Comparing drugs and doses

For most conditions, many drugs are available, and few have been compared directly. Network meta-analysis estimates relative effects and rankings, with the assumptions of transitivity and consistency. Differences in dose, in the duration of therapy and in the background treatment among trials threaten these assumptions. Where doses differ, dose-response models can show how efficacy and harms change with dose, allowing comparison at equivalent doses, see dose-response meta-analysis. Bioequivalence and noninferiority designs have their own requirements: bioequivalence is assessed by pharmacokinetic parameters with confidence intervals compared with accepted limits, and noninferiority requires a justified margin.

Pharmacokinetic and pharmacodynamic data can be pooled in population models, but these are outside the usual meta-analytic framework and need specialist methods. A review that includes them should identify them as a separate evidence stream.

Pharmacist-led services and complex interventions

Pharmacists deliver medication review, counseling, vaccination, adherence support, prescribing and monitoring. Trials of these services are often cluster randomized by pharmacy or practice and are open label. The intervention is complex and differs in content and intensity. Outcomes include medication appropriateness, adherence, blood pressure, glycemic control, hospital admissions and costs. The effect on distal outcomes such as hospital admission is often small and imprecise, while effects on proximal outcomes such as adherence or the number of medication problems are larger. Reviews should keep proximal and distal outcomes distinct and avoid inferring one from the other.

The TIDieR checklist is useful for describing services. Meta-regression on features such as the intensity of contact, the setting and the training of the pharmacist can help to identify what matters. Economic outcomes are variable and context dependent, and need a descriptive treatment unless methods are comparable.

Adherence and its measurement

Adherence is measured by self-report, pill counts, pharmacy refill records, electronic monitoring and drug levels. Self-report overestimates adherence, and methods differ in what they capture: initiation, implementation and persistence are different aspects, as the ABC taxonomy describes. Interventions aimed at adherence produce small average improvements in many reviews, with large variation. Pooling studies that use different measures requires standardization, and the review should state which measure was used. Studies of adherence interventions should be evaluated for whether improved adherence led to better health outcomes, since this link is not guaranteed.

Pharmacoepidemiology and real-world drug safety

After approval, drug safety is studied in cohorts, case-control studies, self-controlled designs and spontaneous report databases. These studies detect rare and delayed harms but are subject to bias. Confounding by indication arises when patients prescribed a drug differ in underlying risk from those not prescribed it. The new-user, active-comparator design reduces this by comparing patients starting one drug with patients starting another for the same indication. Immortal time bias arises when follow-up time before treatment starts is wrongly attributed to the treated group, and it can create a spurious protective effect. Protopathic bias arises when early symptoms of a disease lead to a drug prescription that is then blamed for the disease. Reviews of observational drug safety should extract the design and the approaches used to reduce these biases, and should compare results from designs with different strengths.

Spontaneous reporting systems record suspected adverse reactions voluntarily. They generate signals but cannot give rates, because the number exposed and the rate of reporting are unknown. Disproportionality statistics from such systems should not be pooled as if they were risk estimates. Reviews can summarize signals narratively, and when a signal is tested, it should be followed by studies with a denominator.

Antimicrobial stewardship and medication safety services

Pharmacists play a role in stewardship programs that aim to improve the use of antibiotics, and in programs to reduce medication errors, adverse drug events and potentially inappropriate prescribing in older adults. Evidence comes from interrupted time series, controlled before-after studies and cluster trials. Outcomes include prescribing rates, duration of therapy, resistance patterns, Clostridioides difficile rates, mortality and costs. Prescribing outcomes change quickly, while resistance and clinical outcomes change slowly and are affected by many other factors. Reviews should present these separately and avoid presenting a change in prescribing as evidence of better patient outcomes unless the latter are measured. For time series, the segmented regression model, the number of data points and the adjustment for autocorrelation determine the reliability of the estimate, and this information should be extracted.

Deprescribing, the planned reduction or stopping of medicines that may no longer benefit the patient, is another active field, where the outcomes include medicine count, falls, hospital admissions and mortality. Trials are small, and the effect on hard outcomes is uncertain. A review should state this plainly, and should report how many trials followed patients long enough to see any harm from stopping a medicine.

Common pitfalls we look for

• Dropping trials with zero events or adding a fixed correction without testing alternatives.
• Relying on journal articles alone for drugs with known selective reporting.
• Pooling different doses and durations as if they were the same treatment.
• Reporting benefit without absolute risks of harm.
• Ignoring sponsorship.
• Assuming that a better surrogate measure implies better health outcomes.

Planning and reporting

The protocol states the drug or service, dose, comparators, outcomes (including harms), the effect measures and the approach to sparse data, and plans for regulatory documents. Searches cover MEDLINE, Embase, CENTRAL, International Pharmaceutical Abstracts, registries and regulators' websites. Risk of bias uses RoB 2 for trials and ROBINS-I for observational comparisons. Certainty is rated with GRADE, with attention to imprecision for rare events. Reporting follows PRISMA 2020, including the extension for harms where relevant, and the protocol is registered in PROSPERO.

How we support research projects in this area

Boundaries of this service

A review of pharmaceutical studies describes evidence in groups. It does not provide prescribing, dosing or medicine advice for any person, and nobody should start, stop or change a medicine because of a research summary. Questions about a medicine belong with a pharmacist, doctor or other qualified health professional.

Frequently asked questions