Meta-analysis and evidence synthesis for marine biology

Evidence synthesis in marine biology

Marine biology addresses questions with large public interest: how warming and ocean acidification affect marine animals, whether marine protected areas increase fish abundance, how fishing changes ecosystems, and how coral reefs respond to stress. Meta-analysis combines many small experiments across species, and surveys across regions, to estimate general patterns. Some syntheses have changed the debate: reviews of acidification experiments found average negative effects on calcification, survival and growth, with large variation among taxa, while later work found that reported effects on fish behavior were much smaller in larger and more recent studies.

That last finding illustrates the main point of this page. Marine evidence can be affected by small-study effects and decline effects, by experiments whose conditions differ from natural ones and by sampling that is unevenly distributed. Our methods follow meta-analysis and systematic review practice, adapted to these features. This page builds on the general guidance for life sciences and shares methods with the ecology page.

Experimental conditions and realism

A review should code the treatment magnitude, the control condition, the duration and the life stage, and use meta-regression to see whether effects change with the size of the perturbation. It should also compare single-species laboratory experiments with mesocosm and field studies, where they exist, and be cautious about extrapolating from the former to ecosystems. Pseudo-replication, where several animals in one tank are treated as independent replicates, was common in older experiments and artificially inflates sample sizes, so reviews should extract the number of true replicates (tanks) where it can be identified.

Decline effects and replication

In some marine literatures, the earliest studies report large effects and later, larger studies report much smaller ones. Possible reasons include publication bias favoring striking findings, small-sample exaggeration, differences in methods, and in some cases questions about data. A review can test for this by including publication year and sample size as moderators, comparing effects by study size, and looking at preregistered or multi-laboratory replications. If a few influential studies drive the pooled estimate, leave-one-out and influence analyses show it. Concerns about the reliability of particular datasets are matters for the relevant journals and institutions, and a review reports what the evidence shows and what checks it performed without making allegations.

Behavioral and physiological outcomes

Behavioral endpoints, such as preference for a predator cue or activity level, are noisy and depend on the test design. Physiological measures, such as metabolic rate, depend on handling and temperature. Effects of environmental change on these outcomes are often small relative to individual variation. A review should record the apparatus and protocol, whether observers were blind to treatment, and the number of individuals per tank. Blinding of observers is a known moderator in behavioral studies and has been found to reduce effects in some analyses.

Marine protected areas and fisheries

Studies of marine protected areas compare fish biomass and abundance inside and outside, sometimes before and after protection. On average, protected areas have higher biomass of target species, with variation related to age, size, level of enforcement and isolation. Comparisons are confounded when areas are placed in places that differ in habitat or in fishing pressure, so designs with before-after and control-impact elements are stronger. Fisheries science uses stock assessment models, whose outputs are estimates produced by models and are not pooled like experimental effects. A review of fisheries evidence should describe model structure and data quality, and avoid presenting assessments as direct observations. This service does not provide stock assessments or management advice.

Geographic and taxonomic coverage

Marine research is concentrated on charismatic or economically important groups (corals, fish, bivalves), in accessible waters, and in a few countries. Deep-sea, polar and tropical developing-country systems are under-studied. Evidence maps are useful for showing this coverage, and a review should report the distribution of studies by taxon, habitat and region. Generalizing from shallow temperate species to the whole ocean would be unjustified.

Publication bias and selective reporting

We examine funnel asymmetry in multilevel data, the relation between effect size and publication year, and differences between journals and unpublished sources. Because many marine experiments report many outcomes, selective reporting within papers is a further concern. We code which outcomes were prespecified when a registration or protocol exists, which is rare, and compare effects for primary and secondary outcomes.

Coral reefs, bleaching and community responses

Coral reef research combines experiments on single species with surveys of reef communities over time. Bleaching, mortality and cover change are measured by divers, by photographic transects and by remote sensing, and the methods differ in resolution and in the groups they can distinguish. A synthesis of cover change must say how cover was measured, whether the same transects were revisited and how thermal history was accounted for. Coral species vary strongly in tolerance, and a pooled response averages over winners and losers. Community-level change, such as a shift from coral to algae, is a different outcome from change in cover of a single species, and reviews should keep them apart. Recovery after disturbance depends on local pressures such as nutrient inflow and fishing, so site-level covariates are coded as moderators.

Long-term monitoring datasets are an important source and are often analyzed with specialized models by the groups that collected them. Reviews of this evidence generally describe results from such datasets in tables, since they are not designed as independent studies.

Survey methods and sampling bias

Field surveys differ in gear, depth, season, observer experience and effort. Catch per unit effort depends on gear and on how the fishery operates, so it is an index of abundance that can be biased, for example when fishers target aggregations that persist as stocks decline. Visual census by divers misses cryptic and wary species and is affected by observer skill. When survey data are pooled, a review should record the method for each, include method as a moderator and explain how effort was standardized. Studies that use environmental DNA or acoustic methods offer new data but have their own calibration issues, and reviews should be open about the early stage of work in these areas.

Sampling is also uneven in space and time, with more data from places that are easy to reach and from recent decades. A baseline problem arises when the earliest surveys already come after decades of exploitation, so comparisons with the past may understate change. Reviews should state the earliest year of data for each system and say whether the baseline is a pristine condition or an already altered one, because the size of a reported decline depends on where the comparison starts.

Common pitfalls we look for

• Counting animals in one tank as independent replicates.
• Extrapolating from short laboratory exposures to ecosystems.
• Pooling treatments far beyond realistic projections with realistic ones.
• Ignoring decline effects and sample size trends.
• Treating stock assessment outputs as observations.
• Generalizing from well-studied taxa and regions.

Planning a marine biology synthesis

We help define the organism group, the stressor or intervention and the outcome, plan searches in Web of Science, Scopus, ASFA and CAB Abstracts, and set up coding of species, life stage, treatment level, duration, setting, replicate unit and observer blinding. See the meta-analysis service for scope and process.

An invented example of a decline effect

Suppose ten invented studies of a behavioral effect of acidification are ordered by publication year. The first three, with a mean of 12 animals per group, report standardized effects around 1.5. The last seven, with a mean of 40 animals per group, report effects around 0.2. A pooled estimate across all ten is about 0.6, and a model with sample size as a moderator suggests that the effect at large sample size is close to 0.2. The review would report both the pooled figure and the size-adjusted figure, say that early small studies appear to overestimate the effect, and judge the evidence to be of low certainty. Readers would be misled if only the first figure was quoted.

Coding and transparency

Coding frames record species and its taxonomic placement, life stage, origin (wild or hatchery), treatment and control values with measurement method, duration, tank or replicate structure, observer blinding, outcome and measurement, location and year. Two coders extract data from a sample independently, figures are digitized with software and checked, and the coded data and code are shared with the final report.

How we support research projects in this area

Boundaries of this service

A marine biology synthesis describes average effects across published experiments and surveys. It does not provide fisheries management advice, stock assessment, conservation planning or species identification. Many experiments are short and use conditions that differ from natural ones, and effect sizes in some literatures have declined as larger studies appeared, so conclusions should be read with caution.

Frequently asked questions