Meta-analysis and evidence synthesis for criminology

Evidence synthesis in criminology

Criminology asks why crime occurs, how it is distributed and what reduces it. Evidence synthesis has an established role in the field: the Campbell Collaboration publishes systematic reviews of crime and justice interventions, and meta-analyses have addressed hot-spots policing, problem-oriented policing, cognitive-behavioral programs for offenders, restorative justice, correctional education, drug courts and violence prevention. These reviews are used by policymakers and have shaped practice.

The evidence has features that call for care. Randomized trials are less common than in medicine, so quasi-experimental designs form much of the evidence. Outcome data come from administrative records (arrests, convictions) that reflect enforcement as well as behavior. Interventions are adapted locally, and effects may be displaced to other places or times. Our methods follow systematic review and meta-analysis practice, adapted to these features. This page builds on the general guidance for social sciences. It is about research methods and does not offer legal advice or risk assessment of individuals.

Measuring crime and recidivism

Recidivism definitions vary by country, follow-up length and inclusion of technical violations, so comparison across studies requires care. A review should code the definition and follow-up period, analyze outcome types separately, and convert effects to a common scale such as an odds ratio only within comparable outcomes. Because enforcement activity can change recorded crime without changing offending, a review should use victimization data when available and discuss differences.

Designs and credibility

Evidence on place-based policing, community programs and legal reforms often comes from before-after comparisons with matched comparison areas, interrupted time series, and difference-in-differences. These designs need the assumption that comparison areas would have followed the same trend, which can fail if areas differ. Studies of programs for offenders often compare participants with non-participants who differ in risk and motivation, so selection into the program is a major concern. Reviews have therefore assessed methodological quality and found that effects tend to be smaller in stronger designs. The Maryland Scientific Methods Scale and similar ratings have been used to classify designs, and a review should show results by design level instead of pooling them blindly.

Randomized trials exist for some interventions, such as hot-spots policing and certain offender programs. Where they do, they are analyzed separately from quasi-experimental studies and compared.

Displacement and diffusion of benefits

Preventing crime in one place may push it to nearby places (displacement), or may reduce it nearby as well (diffusion of benefits). Studies of place-based interventions often measure outcomes in adjacent catchment areas. Reviews of hot-spots policing have found that diffusion is more common than displacement, with the number of studies still modest. A review should extract effects for target and adjacent areas, report the definition of the catchment area and examine whether displacement estimates depend on the design. Interventions aimed at people can have similar substitution effects, for example when offenders change the type of offense, which needs outcome measures covering several crime types.

Rehabilitation and offender programs

Reviews of programs for people who have offended, such as cognitive-behavioral therapy, education, vocational training and substance misuse treatment, generally find modest average reductions in re-offending, with large variation by program quality, risk level of participants and implementation. Principles from the risk-need-responsivity model are widely cited, and meta-analyses have tested whether programs consistent with those principles perform better. The studies supporting such conclusions are mostly quasi-experimental and often come from North America. A review should describe participants' risk level, program components, dosage and delivery, and flag developer-led evaluations. Program evaluations are sometimes authored by those who run them, and independent evaluations may show smaller effects.

Jurisdiction, period and portability

Legal systems, policing styles, prison conditions and social conditions differ across countries and change over time. An intervention evaluated in a United States city may not transfer to a European one. A review should present studies by country and period, use meta-regression sparingly and highlight limits to generalization. Reviews may also need to address ethical questions about the use of findings, such as whether the evidence reflects groups who are over-policed. Differential enforcement can affect recorded outcomes, and the review should say so when relevant.

Risk assessment instruments

Studies of tools that predict re-offending report discrimination (area under the curve) and calibration. Pooled AUC values for many instruments cluster around moderate discrimination, with variation across settings and groups, and performance is lower in some groups than others. Reviews of these tools should follow the methods for prediction models described on the prognostic meta-analysis page and discuss fairness, calibration across groups and the consequences of use. This service does not produce or evaluate risk assessments for any individual.

Publication bias and developer effects

Published evaluations of crime prevention programs may favor positive findings, and programs run by their developers often report larger effects. We search the grey literature, which is substantial in criminology (government and police reports, theses), compare published and unpublished results, and examine effects by evaluator independence.

Violence prevention, domestic abuse and youth programs

Evidence on violence prevention covers school-based programs, parenting support, early childhood interventions, mentoring and intervention for domestic abuse. Many outcomes are self-reported by participants or by teachers who know the program, and follow-up periods are often short. Reviews of programs for perpetrators of domestic abuse have found small average effects on re-offending with wide uncertainty, partly because outcome data, such as police reports and partner reports, capture different aspects. A review should record who reported outcomes and the follow-up, and be explicit that absence of a recorded incident is not the same as absence of abuse. Early childhood programs show long-term effects on crime in a few well-known trials, but these are few and may not carry over to other settings.

Youth justice research has the added need to look at harm: some programs that bring young people into contact with the justice system have been associated with more offending later, so reviews should look for adverse effects as well as benefits, and report them even when they are few and imprecise, because a missing harm signal is often a sign of missing data and not of safety.

Coding and transparency

Coding frames record the intervention and its components, setting, country, population and risk level, design and quality rating, outcome definition and source, follow-up length, attrition, evaluator independence and funding. Two coders extract data independently on a sample and agreement is reported. Where a study reports effects for several offense types or follow-up points, we specify in advance how to choose or model them. Data and code are shared with the final report so readers can verify results.

Common pitfalls we look for

• Treating recorded crime as a pure measure of offending.
• Pooling weak and strong designs without separating them.
• Ignoring displacement and diffusion.
• Combining different recidivism definitions and follow-up periods.
• Relying on developer-led evaluations alone.
• Transferring findings across jurisdictions without comment.

Planning a criminology synthesis

We help define the intervention, population, comparison and outcomes, plan searches in Criminal Justice Abstracts, NCJRS, Scopus, Web of Science, Campbell and government sources, and set up coding of design, quality level, outcome definition, follow-up, country and evaluator independence. Reporting follows PRISMA and Campbell Collaboration standards. See the systematic review service for scope and process.

An invented example of a recidivism effect

Suppose a program for adults leaving prison is associated with an odds ratio for reconviction within two years of 0.80 across 30 invented studies, with a 95 percent confidence interval from 0.70 to 0.92. If the baseline reconviction rate is 40 percent, an odds ratio of 0.80 corresponds to a rate of about 34.8 percent (odds of 0.667 times 0.80 equals 0.533, and 0.533 divided by 1.533 is 0.348). That is a reduction of about five percentage points. The same studies, divided by design, might show an odds ratio near 0.70 in weak designs and 0.90 in randomized trials. A reader would rightly take the smaller figure from the stronger designs as the better guide, and the review should present both.

Ethics, sensitivity and use

Criminology research concerns victims, offenders and communities. Reviews treat the data as aggregate published results and avoid identifying individuals or places in sensitive ways. The language used should be precise and avoid labels that stigmatize. Reports note funding sources, since government or police funding may influence evaluation, and include limitations clearly because findings may be used in policy debates and in public discussion that can go beyond what the data support.

How we support research projects in this area

Boundaries of this service

A criminology synthesis describes average effects across published evaluations and studies. It does not provide legal advice, expert testimony, risk assessment of any person, policing strategy or program design for a particular jurisdiction. Crime data reflect enforcement as well as behavior, and findings may not transfer across places and periods.

Frequently asked questions