What a mixed-methods review is
Some questions have two halves. Does a school-based program reduce bullying? And how do pupils and teachers experience the program, and what makes it work or fail? Trials answer the first half. Interview and observation studies answer the second. A mixed-methods review treats these as parts of one question and combines the evidence.
Pluye and Hong describe a mixed studies review as a literature review that combines qualitative, quantitative and mixed-methods studies, and a mixed-methods review in the narrower sense as one that deliberately integrates the results of separate quantitative and qualitative syntheses. Sandelowski, Voils and Barroso, writing on mixed research synthesis, stress that integration is the point: the review should say more as a whole than the quantitative and qualitative parts say alone.
The Joanna Briggs Institute has published methodological guidance for mixed-methods systematic reviews, and Cochrane's work on complex interventions uses similar logic when it links trials with process evaluations. The review type is therefore established, although the terminology varies between fields, and the review should define it in the protocol.
Review designs
Three designs are commonly described, following the convergent and sequential structures used in primary mixed-methods research. The choice depends on how closely the two kinds of evidence match.
| Design | How it works | Suited to |
|---|---|---|
| Convergent segregated | Quantitative and qualitative evidence are synthesized separately, then brought together | Independent syntheses answering related questions; integration at the interpretation stage |
| Convergent integrated | Data are transformed (qualitized or quantitized) and synthesized together | Evidence is similar enough in form to be combined in one synthesis |
| Convergent contingent (sequential) | One synthesis shapes the question or the sampling of the next | A first synthesis identifies a question, such as barriers, that a second synthesis addresses |
In the convergent segregated design, the quantitative evidence is synthesized, perhaps by meta-analysis, and the qualitative evidence is synthesized, perhaps by thematic synthesis, in separate steps with their own methods and quality criteria. The results are then integrated, for example by comparing themes with effect estimates. This design keeps the integrity of each method and is the most often recommended. Convergent integrated designs transform one kind of data into the other, by turning themes into counts or findings into narrative categories, so that a single synthesis can be done. That is simpler to describe but can lose information and needs justification.
Framing the question
The review question should call for both kinds of evidence. Mixed-methods review questions are often split into sub-questions: one about effectiveness or magnitude and one about experience, acceptability or context. Each sub-question has its own eligibility criteria, search terms and synthesis method, and the protocol should explain how the answers will be linked.
A common mistake is to add qualitative studies to a trial review as an afterthought. Another is to include both kinds of evidence without a plan for bringing them together, which produces two reviews in one document. The integration plan should be written before analysis: what will be compared, by what method and what the integration could show, such as agreement, disagreement or complementary information.
Searching, selecting and appraising
The search must reach several literatures. Quantitative studies are found with the usual design filters, while qualitative studies need the terms described for qualitative synthesis. Using a single search for all designs is efficient but yields many records, and some teams run separate searches for each sub-question. The strategy, the sources and the dates should be reported in full.
Appraisal is where mixed-methods reviews differ most from single-method reviews. One option is to use a design-specific tool for each study type: RoB 2 or ROBINS-I for quantitative studies and CASP or JBI checklists for qualitative studies. Another is to use the Mixed Methods Appraisal Tool, which provides a common framework for five categories of study. The MMAT authors advise against computing an overall score and recommend describing the ratings for each criterion. Whichever is used, the review should explain how quality affects the synthesis.
Screening and extraction should be done by two reviewers. Extraction forms need fields for both kinds of study, and for mixed-methods primary studies the team must decide whether to treat the quantitative and qualitative parts as separate sources of evidence.
Integrating the evidence
Integration is the central task and the part most often done poorly. Several techniques are available. In a matrix or joint display, the rows are the outcomes or the components of the intervention, and the columns hold the quantitative findings, the qualitative findings and the points of agreement or conflict. Seeing the two sets of results side by side makes patterns clear. In a narrative weaving approach, the text moves back and forth between the two types of evidence, one theme or outcome at a time.
Qualitative findings can be used to explain quantitative results. If trials show that an intervention works in some settings but not others, qualitative studies about those settings may suggest reasons such as lack of staff time or a mismatch with local priorities. Quantitative results can in turn be used to test whether themes matter: does a trial of a program that addressed a barrier found in interviews show better results than a trial that did not? Such comparisons are exploratory and should be presented as hypotheses unless the analysis was planned and powered.
Where the two bodies of evidence disagree, say so. Disagreement is a finding, and it may point to a gap between what people say and what the measures capture.
A worked outline
Imagine a review of text-message reminders to improve attendance at clinic appointments. The quantitative sub-question asks whether reminders increase attendance compared with no reminder. The qualitative sub-question asks how patients and clinic staff view reminders and what affects whether a reminder helps. The design is convergent segregated.
The quantitative strand includes randomized trials, appraised with RoB 2 and pooled by random-effects meta-analysis, with a prediction interval and a GRADE rating. The qualitative strand includes interview and focus-group studies, appraised with CASP and synthesized by thematic synthesis, with a CERQual rating for each finding. The two strands are then set in a joint display. The rows list themes such as timing of the message, language of the message, trust in the clinic and access to a phone. The columns show what the trials say about subgroups or intervention features related to each theme and what the interviews say.
If the trials show smaller benefits where reminders were sent the day before, and interview studies report that patients need more time to arrange transport, the two strands point to the same explanation. If the trials show no difference by language but interviews stress the importance of language, the review reports the mismatch and suggests reasons, such as trials not measuring it. This outline illustrates the logic of integration and does not report results from a real review.
Reporting and quality
PRISMA 2020 covers the reporting of searches, selection and results, and the review should add a description of the mixed-methods design, the integration method and any transformation of data. The Joanna Briggs guidance and the work of Hong and colleagues on convergent and sequential designs give detail on what to describe. The report should also give the confidence in the findings from each strand, using GRADE for quantitative outcomes and GRADE-CERQual for qualitative findings.
A strong mixed-methods review shows a clear rationale for mixing, a design matched to the question, explicit integration, appraisal of every included study and conclusions that draw on both strands. Weaknesses to avoid include separate reviews placed side by side, unjustified transformation of data and unequal attention, where one strand is detailed and the other is thin.
Team and time
A mixed-methods review needs a team with quantitative and qualitative skills and a shared language. The project is larger than a single-method review because there are two syntheses and an integration step, so time and cost estimates should reflect that. Early agreement on terms, such as what counts as a theme, a finding or a result, avoids confusion. Decisions about authorship roles, data management and the order of work, for example whether the qualitative strand starts after the quantitative search or in parallel, should be settled in the protocol. A shared extraction spreadsheet with separate tabs for each strand, and a log of decisions made during integration, give the team an audit trail and make the report easier to write. Reviewers should also agree in advance how to handle primary studies that themselves use mixed methods, since their quantitative and qualitative parts may be extracted into the matching strands. Regular meetings during integration are useful because this stage depends on discussion between team members who read the evidence differently.
Limitations
Mixed-methods reviews are demanding. They need expertise in two traditions, take longer than single-method reviews and have no single agreed standard for integration. Transforming data from one form to the other can lose information. Quantitative and qualitative studies are rarely matched in setting, population or time, so comparisons between strands are indirect. Integration involves interpretation, and different teams could draw different conclusions. Findings from the qualitative strand are not estimates of effect. The review describes patterns in the evidence and does not give advice on an individual's care.
Evidence synthesis informs decisions about groups and policy. It does not replace professional judgment about an individual.
How we can help
Support for a mixed-methods review
The work can be supported at different depths. Choose what you need, and the scope is agreed in writing before work begins.
Protocol and registration
A protocol and analysis plan written before the search, with registration prepared where appropriate.
Search and screening
Documented search strategies, screening and the flow diagram.
Appraisal and synthesis
Data extraction, risk of bias, certainty of evidence and the synthesis.
Manuscript and submission
Optional: the full paper, the reporting checklist and the submission materials.
Frequently asked questions
What is a mixed-methods review?
A review that combines quantitative and qualitative evidence, usually by synthesizing each and then integrating the results.
How is it different from a systematic review of trials?
A trial review answers questions about effects. A mixed-methods review adds qualitative evidence to answer questions about experience, context and implementation as well.
Do I have to transform the data?
No. The segregated design synthesizes each strand separately and integrates the results, which avoids loss of information.
How do I appraise studies of different designs?
Use design-specific tools or the MMAT, report ratings by criterion and explain how quality was considered in the synthesis.
What is a joint display?
A table or figure that places quantitative and qualitative findings side by side to show where they agree, disagree or complement each other.
Which reporting guideline should I use?
PRISMA 2020 for the review, plus a description of the design and integration method as recommended in JBI and related guidance.
References
- Pluye P, Hong QN. Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews. Annu Rev Public Health. 2014;35:29-45.
- Sandelowski M, Voils CI, Barroso J. Defining and designing mixed research synthesis studies. Res Sch. 2006;13(1):29-44.
- Stern C, Lizarondo L, Carrier J, et al. Methodological guidance for the conduct of mixed methods systematic reviews. JBI Evid Synth. 2020;18(10):2108-2118.
- Hong QN, Pluye P, Bujold M, Wassef M. Convergent and sequential synthesis designs: implications for conducting and reporting systematic reviews of qualitative and quantitative evidence. Syst Rev. 2017;6:61.
- Hong QN, Fabregues S, Bartlett G, et al. The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Educ Inf. 2018;34(4):285-291.
- Lewin S, Glenton C, Munthe-Kaas H, et al. Using qualitative evidence in decision making for health and social interventions (GRADE-CERQual). PLoS Med. 2015;12(10):e1001895.