Qualitative evidence synthesis

What qualitative evidence synthesis is

Qualitative studies explore how people understand and experience events: why patients stop taking a medicine, how teachers adapt to a new curriculum, what makes a screening program acceptable to a community. Each study usually involves a small, purposively chosen sample and rich description. A single study is limited in scope, and the findings of many studies on the same topic are scattered across journals.

Qualitative evidence synthesis is the systematic integration of such findings. It has several names, including qualitative systematic review, qualitative meta-synthesis and, in some traditions, qualitative meta-analysis, but the last term is misleading, because the synthesis does not pool numerical effects. The aim is to build a deeper or broader interpretation than any single study provides. Reviews of this kind are used to inform guidelines, to design and interpret trials of complex interventions and to explain why interventions work in some settings and not in others.

The Cochrane Qualitative and Implementation Methods Group has issued guidance for such syntheses alongside intervention reviews. The methods differ from meta-analysis in nature as well as in technique. They work with words and concepts, they do not seek a single average and they accept that the researcher's interpretation shapes the result. Rigor comes from transparency, not from statistics.

Choosing a synthesis method

More than a dozen methods exist. The choice depends on the question, on the number and richness of the studies and on whether a framework is available. The table lists five that are widely used.

Thematic synthesis, described by Thomas and Harden in 2008, was developed for health-care questions about the views of people receiving an intervention. Meta-ethnography, introduced by Noblit and Hare in 1988, is the most interpretive and is aimed at generating new concepts and theory. Framework synthesis is more structured and is useful where a policy or logic model is available. The method should be named in the protocol and its steps followed and described, instead of labelling a loosely done review with a method name.

Framing the question and sampling studies

Quantitative reviews often use PICO. For qualitative questions, other structures fit better, such as SPIDER (sample, phenomenon of interest, design, evaluation, research type) or PICo (population, phenomenon of interest, context). Whatever the structure, the question should be about a phenomenon and context, such as the experiences of carers of people with dementia during hospital discharge, not about an effect size.

Sampling in a qualitative synthesis does not always mean including every study. Because the aim is conceptual saturation and not statistical estimation, some syntheses sample purposively from a large pool, for example by selecting studies with the richest data. If a review does this, it must state the rule and the reason. Exhaustive sampling remains the default for reviews that inform guidelines, where completeness supports confidence.

Searching qualitative studies is harder than searching for trials. Indexing is inconsistent, and the study design may not be mentioned in the title or abstract. Searches should combine subject terms with design terms (interview, focus group, thematic analysis, grounded theory, ethnography) in several databases, supplemented by citation chasing and, where relevant, grey literature such as theses. A librarian with experience in qualitative searching improves the yield.

Appraising qualitative studies

Appraisal of qualitative research is debated. Some argue that quality criteria cannot be applied across paradigms; others hold that reviewers need a structured way to judge credibility. Common tools include the CASP qualitative checklist and the JBI checklist for qualitative research. Their items cover the clarity of aims, the fit of the design, recruitment, data collection, reflexivity, ethics, rigor of analysis and the clarity of findings.

Reviewers decide what to do with the result. Some exclude studies of the lowest quality, some include all and consider quality when interpreting, and others test the effect of excluding studies on the synthesis through sensitivity analysis. The important points are that the approach is chosen in advance, applied by more than one reviewer and reported. Judging a study as weak because the report was brief, when the study itself may have been sound, is an error to avoid. Many published reports are cut by word limits.

Context also matters. The synthesis should record the setting, the population and the time of each study, since themes may not transfer across contexts. A summary table of study characteristics is as necessary here as in a quantitative review.

How a synthesis is done

The process varies by method, but several steps recur. The reviewers read the studies repeatedly. They extract the findings as the authors presented them, including participant quotations and the authors' interpretations, because both can be data. They code the findings, group the codes and develop themes. In thematic synthesis, descriptive themes stay close to the original studies and analytical themes go beyond them to answer the review question. In meta-ethnography, reviewers list the key concepts of each study, determine how the studies are related (reciprocal, refutational or forming a line of argument), translate concepts into one another and synthesize the translations.

Two or more reviewers should take part, and the team should record how disagreements were settled. Reflexivity is expected: the team should state what assumptions and backgrounds they bring, since these influence interpretation. Software such as NVivo, ATLAS.ti or a spreadsheet can organize the coding. The software does not do the analysis; it holds the data and the audit trail.

The result should be more than a list of themes. A good synthesis explains relationships between themes, shows where studies agree and differ and answers the review question. A conceptual diagram or a line of argument can express this. The team should keep an audit trail, showing how each theme was derived from the codes and the extracts that support it, so that another reader can follow the path from the primary studies to the final statement. Thick description of the contexts in which themes arose also helps readers judge whether the findings apply to their own setting.

Confidence in the findings and reporting

Review findings need a confidence rating, as effect estimates do. GRADE-CERQual (Confidence in the Evidence from Reviews of Qualitative research) assesses each finding on four components: methodological limitations of the contributing studies, coherence of the finding with the underlying data, adequacy of the data supporting it and relevance of the contributing studies to the review question. The result is a judgment of high, moderate, low or very low confidence for each finding, and the reasoning is shown in an evidence profile. Lewin and colleagues described the approach in 2015, and the GRADE-CERQual group has since published component guidance.

For reporting, ENTREQ (Enhancing Transparency in Reporting the Synthesis of Qualitative Research) offers 21 items covering the aims, the search, the selection, the appraisal and the synthesis. For meta-ethnography, the eMERGe guidance gives 19 reporting items that are specific to the method. PRISMA 2020 can also guide the reporting of the search and selection. A reviewer should select the guidelines that match the method and say so.

Findings should be written for the intended reader. For a guideline panel, findings are linked to the recommendation they inform. For a trial team, they point to implementation barriers and outcomes to be measured. The limits of transferability, given the sampling of the primary studies, should be stated.

Using qualitative synthesis with other evidence

A qualitative synthesis is often conducted alongside a quantitative review of effects. The two can be combined by mapping qualitative themes against the components or outcomes of an intervention, by comparing the subgroups in trials with the conditions described in the qualitative evidence, or by using the qualitative findings to explain differences between trials. Where an integrated design is planned from the start, a mixed-methods review is the appropriate frame. Where the aim is to explain how programs work in context, a realist review may be better.

Qualitative findings should not be forced into numeric form. Counting how many studies mention a theme can be useful for description, but frequency does not equal importance, and a theme found in one rich study can matter more than one mentioned in several brief ones.

Limitations

Qualitative synthesis rests on interpretation, so different teams may reach different themes from the same studies. Results depend on the quality and depth of the primary reports, which are often shortened by word limits. Sampling of the primary studies was purposive, so findings may not transfer to other settings. Appraisal of qualitative research is debated and no tool is universally accepted. The synthesis does not estimate how common an experience is. Confidence ratings help but cannot remove the effect of subjective judgment. Findings describe experiences and contexts and are not clinical advice.

How we can help

Frequently asked questions