Meta-analysis and evidence synthesis for marketing research

Evidence synthesis in marketing research

Marketing research asks how people and organizations respond to what firms do. Typical questions are whether advertising raises sales, how much demand changes when price changes, whether satisfaction predicts loyalty, whether online reviews influence purchase, and whether a promotion has lasting effects or only moves purchases forward in time. The field has a long record of quantitative synthesis, especially on elasticities of sales to price and advertising, because firms and researchers wanted generalizable numbers rather than results from one brand.

The evidence has features that need care. Studies use different products, markets and periods. Laboratory experiments measure intentions, while market data measure behavior. A single paper often reports many effects, for several brands or conditions. Many results come from convenience samples, such as students or online panels. Our methods follow meta-analysis and systematic review practice, extended for these features. This page builds on the general guidance for management and business.

Effect sizes, elasticities and what they mean

Elasticities are already unit-free, so they can be pooled directly, but only when their model specifications are comparable. A log-log model gives a constant elasticity, whereas a linear model gives an elasticity that changes along the demand curve. A review should code the functional form, the data level (store, brand, household), the time horizon and whether endogeneity of price was addressed, and then test whether any of these moderates the result. Reviews of price elasticity have found wide variation across product categories and studies, and have attributed part of it to method choices, which is the reason to code them.

When both elasticities and standardized effects appear in the same review, they answer different questions and should be analyzed separately. Converting one to the other requires assumptions that are rarely justified.

Laboratory versus market evidence

Experiments give clean causal comparisons but measure responses such as stated intention or attitude. Market data measure real purchase but come from settings where firms choose prices and advertising in response to demand, so the estimates can be biased unless the study handles that endogeneity. The two kinds of evidence therefore complement each other and should not be pooled into a single estimate without a moderator for data source.

Intentions are known to overstate behavior. A review that mixes outcomes such as intention, trial and repeat purchase should analyze them as separate outcomes. It should also report whether the experimental stimuli were real brands, fictitious brands or generic products, since familiarity changes how people respond.

Field experiments and randomized online tests are valuable because they combine random assignment with real behavior. They also have limits: results are specific to a platform and period, and firms tend to run, and to publish, tests that worked.

Samples, context and generalizability

Many consumer experiments use university students or paid online panel workers. Whether results generalize depends on the topic. Responses to a product claim may differ little between groups, while responses to price or financial risk may differ a lot. A review should extract sample type and test it as a moderator.

Culture and market development are additional moderators. Studies from the United States and Western Europe dominate the literature, and effects of advertising appeals, loyalty programs or brand origin can differ elsewhere. Meta-regression on country-level indicators is possible but limited by the small number of countries and by confounding with product category and period. Reviews should provide a table of countries, product categories and periods covered, and say where evidence is thin.

Time matters too. Digital advertising, search and social media have changed within a decade, so a pooled effect across twenty years may describe no current setting. Period can be examined as a moderator, but the review should be modest about extrapolating.

Dependent effect sizes within papers

Marketing papers commonly report many effects: several brands, several product categories, several experimental conditions or several outcome measures. Treating each as independent gives too much weight to papers with many effects and understates uncertainty. Options include selecting one effect per paper by a stated rule, averaging within papers, using multilevel models with effects nested in papers, or using robust variance estimation to correct standard errors. Each has trade-offs, and the choice should be stated in advance.

Multilevel models are well suited because they separate within-paper from between-paper variance and allow a moderator to act at either level. We report the variance at each level and run sensitivity analyses with alternative handling of dependence. We also check whether results hinge on papers that contribute a large share of effects.

Satisfaction, loyalty and brand evidence

Customer satisfaction, loyalty, trust and brand attitude are measured with survey scales, and the correlations among them are often large partly because the same respondents answer all items at one time. A review of these relationships should record whether the data came from a single survey, whether items overlapped in content and whether behavioral loyalty data, such as repeat purchase, was available. Analyses using behavioral outcomes tend to find smaller associations than those using attitudinal outcomes.

Meta-analytic structural equation modelling can test a model of several constructs, such as satisfaction, trust, commitment and loyalty, by pooling the correlation matrices from many studies and fitting the model to the pooled matrix. This needs studies that report the full matrix or that can be supplemented by author contact. We describe the assumptions, including how sample size is defined for the pooled matrix, and the fit indices used with the caution they need.

Digital marketing, reviews and online data

Online reviews, social media engagement, search advertising and email campaigns generate large data sets and many studies. Large samples give precise estimates of small effects, so a statistically significant result may be commercially trivial, and heterogeneity across platforms can be large. A review should describe the platform, the period and the metric (click, view, purchase, review rating), and should note that metrics are defined differently by different companies and change over time.

Studies that use company data may be selected: firms share successful campaigns. Studies based on reviews face self-selection, since people who write reviews are not representative. We look for these issues in risk-of-bias judgment, and we flag results from unpublished industry reports as a separate category, since those reports are not peer reviewed and may not describe their methods.

Publication bias and the file drawer in marketing

Marketing journals favor results that are statistically significant and theoretically interesting. Funnel-plot asymmetry, selection models and p-curve analysis can show whether the literature is likely to overestimate effects. Because the field uses many different effect measures, these tools are applied within a measure, not across the whole data set. We search conference proceedings, working papers, dissertations and industry reports to reduce the problem, and we report what was found in each source.

Replication of consumer experiments, in marketing and in related fields, has shown that some effects are smaller than first reported. A review can include replications and test whether published original studies give larger effects than replications.

Searching marketing and business literature

Relevant studies are spread across marketing, psychology, economics, information systems and communication. Core databases include Business Source Complete, ABI/INFORM, Scopus, Web of Science, PsycINFO and EconLit, plus preprint servers such as SSRN. Search terms must handle variants such as "price sensitivity", "price elasticity" and "demand response". Citation tracking from key papers and prior meta-analyses is productive. We record the search string for each database, and the date, so the search can be repeated.

Common pitfalls we look for

• Pooling elasticities from different model forms without coding the form.
• Mixing intentions and behavior in one outcome.
• Counting every effect from a paper as independent.
• Relying on student samples for claims about the general consumer.
• Ignoring period in fast-changing digital settings.
• Treating a statistically significant lift in a huge sample as a large effect.
• Using unpublished company reports without describing their methods.

Planning a marketing evidence synthesis

A useful marketing review starts with a precise question: which action, for which consumers, in which market, compared with what, and measured how. We help define inclusion criteria, choose effect measures, build the coding sheet (including data source, sample type, product category and model form), plan the handling of dependence and write the protocol. Where the aim is a general elasticity benchmark, the plan emphasizes coding of model characteristics; where the aim is to test a theory, it emphasizes construct definitions and moderators. See the meta-analysis service for scope and process.

How we support research projects in this area

Boundaries of this service

A marketing evidence synthesis describes average effects across studies. It does not provide marketing strategy, campaign design, pricing advice or a forecast for any firm, product or market. Commercial decisions depend on costs, competition and context that a review cannot see. Results from consumer studies describe the populations sampled and may not apply to other groups, countries or periods.

Frequently asked questions