Building a search strategy

What a good search must do

A search for a systematic review has two aims that pull in opposite directions. It should be sensitive, finding as many as possible of the studies that meet the eligibility criteria, because any study missed is lost to the review. And it should be specific enough that the number of irrelevant records is manageable, because every record has to be screened. A search that is highly sensitive retrieves thousands of records of which a few dozen are relevant, and one that is highly specific retrieves a short list that may miss important studies. Systematic reviews favor sensitivity, and accept a high volume of irrelevant results as the price.

The search must also be reproducible. Another searcher, given the strategy and the date, should be able to run it and retrieve the same set. That is why the strategy is written down exactly, the databases and platforms are named, and every limit is recorded. These requirements are summarized in PRISMA-S, the extension of PRISMA for searches. The service is described on the page for search strategy development.

Start with concepts, not words

The question is broken into its main concepts, usually two or three. For an intervention question these are often the population or condition and the intervention, sometimes with the study design. The comparator and the outcome are generally left out of the search because they are inconsistently reported and indexed, and adding them would drop relevant studies. See PICO and related frameworks.

For each concept, the searcher lists the words that authors and indexers might use: synonyms, plural and singular forms, alternative spellings, abbreviations and acronyms, older and newer terminology, and related terms. A good source of candidate terms is a handful of studies known to be relevant: their titles, abstracts and index terms show how the concept is expressed. A thesaurus in the database, and a scan of the results of a first test search, add more. The aim is a list that is wide enough to catch the variation in language, with each concept treated as a block.

Boolean operators

The structure of a typical strategy is therefore a set of parenthesized groups of synonyms, each joined internally by OR, and the groups joined by AND. Parentheses matter: without them, the database may combine terms in an order that gives the wrong logic. A misplaced bracket is one of the most common and most damaging errors in search strategies, which is why strategies are peer reviewed.

Truncation, wildcards, phrases and proximity

Several devices increase sensitivity without listing every word form. Truncation uses a symbol, often an asterisk, to find all words that begin with a stem: a search for exercis* finds exercise, exercises, exercising and exercised. Wildcards stand for a single character, for spelling variants such as wom?n. Phrase searching, with quotation marks in many databases, finds words together in order. Proximity operators find words within a stated distance of each other, regardless of order, which is useful when the same idea can be written in different word orders, and is more sensitive than a phrase while more specific than AND.

The symbols differ between platforms. The asterisk is common, but the symbol for a single-character wildcard, the form of proximity operators and the treatment of phrases are not the same in PubMed, Ovid, EBSCO and others, and some databases do not support all of them. The searcher checks the help pages of each platform, and a strategy is translated, not copied. Truncating too early, for example at three letters, can retrieve thousands of irrelevant words, so the stem is chosen with care.

Controlled vocabulary and free text

Most large databases index records with a controlled vocabulary of subject headings: MeSH in MEDLINE and PubMed, Emtree in Embase, and other thesauri in other databases. An indexer assigns headings according to the content of the paper, whatever words the authors used, so a heading search can catch records that never use the expected word. Headings can usually be exploded, to include narrower terms under a broader one, and restricted to the major focus of the paper. Their weakness is inconsistency: indexing is not perfect, is applied unevenly across records, and is absent from the newest records that have not yet been indexed.

Free-text searching of the title and abstract catches those records and the words authors actually used, but misses papers that use different words. A strong strategy therefore combines both for each concept, joining the heading and the free-text terms with OR. The searcher also checks the thesaurus entry for each heading, because its scope note, the heading's history and its entry terms suggest further free-text words. The guide to databases compares what the main sources offer.

A worked structure

Suppose the question is whether exercise programs reduce depressive symptoms in adolescents. There are three concepts: the population, the intervention and the condition. A strategy for PubMed, with illustrative terms, might be structured as follows. It is an outline to show the logic and not a validated strategy.

#1  "Adolescent"[Mesh] OR adolescen*[tiab] OR teen*[tiab] OR "young people"[tiab]
#2  "Exercise"[Mesh] OR "Exercise Therapy"[Mesh] OR exercis*[tiab] OR "physical activity"[tiab]
#3  "Depression"[Mesh] OR "Depressive Disorder"[Mesh] OR depress*[tiab]
#4  #1 AND #2 AND #3

In PubMed, [Mesh] searches the subject headings and [tiab] searches titles and abstracts. In Ovid MEDLINE, the same ideas are written with different symbols, for example exp for exploding a heading and .ti,ab. for the title and abstract fields, and in Embase the headings are Emtree terms. The structure is the same in each: a group of headings and free-text words for each concept, joined by OR, and the groups joined by AND. In practice the strategy would be longer, with more synonyms, and would be built and tested iteratively, as described next.

Testing and refining

A strategy is a hypothesis about how to find the relevant studies, and it is tested. The usual test uses a set of studies already known to be relevant, found from earlier reviews, reference lists or expert suggestions. The strategy is run, and the searcher checks how many of the known studies it retrieved. If any are missing, the reason is traced, for example a term that was not included or a heading that was not assigned, and the strategy is revised. The proportion of known studies retrieved is a rough estimate of sensitivity, and is reported. The searcher also scans samples of the results, to see which irrelevant terms generate the most noise, and tightens them if it can be done without losing known studies.

This iterative development is normal, and the final strategy is the one that is recorded. The record of the process can be kept as a log, which is useful for reporting and for peer review.

Peer review of the strategy

Errors in strategies are easy to make and hard to see, so a second searcher reviews the strategy. The PRESS guideline gives a checklist covering six areas: whether the search translates the question accurately, whether Boolean and proximity operators are correct, whether subject headings are suitable and exploded appropriately, whether text words are adequate with spelling variants and truncation, whether spelling, syntax and line numbers are correct, and whether limits and filters are justified. Evidence shows that peer review finds errors that change the results of searches. The reviewer's comments, and the changes made, are documented. See search strategy development.

Filters and limits

Search filters, also called hedges, are prepared strategies that retrieve a type of study, such as randomized trials, or a topic, such as a population. They save time and are useful when they have been validated, with published sensitivity and precision, and when they are used as designed and for the platform they were written for. The searcher chooses a filter whose performance has been evaluated, and cites it. Limits, such as language, date or publication type, are used sparingly and reported, because each can remove relevant records. A restriction to English-language publications, for instance, is common and defensible in some reviews and a source of bias in others. Design filters should not be applied for topics where the relevant studies are poorly indexed by design, as with diagnostic accuracy studies, where filters perform poorly. See diagnostic accuracy meta-analysis.

Documenting the search

For every source, the record includes the database name and the platform or interface, the date of the search, the full strategy exactly as it was run, any filters and limits, and the number of records retrieved. PRISMA-S sets out what to report, and journals usually expect the full strategies as a supplement. The records are exported in a consistent format, merged and de-duplicated, with the number removed recorded for the flow diagram. Because the literature grows, a search should be updated before submission when a long time has passed, and the update is documented in the same way. See the guide on de-duplication and screening.

Common mistakes

• Copying a strategy between databases without translating the headings and syntax.
• Searching only free-text words, or only subject headings, and so missing records that the other would catch.
• Including the outcome or comparator as a concept and losing studies that do not mention it in the abstract.
• Missing parentheses, which change the logic without warning.
• Truncating too early, flooding the results with irrelevant words.
• Not testing against known studies.
• Not recording the strategy and the date, so the search cannot be reported or repeated.

Support

Frequently asked questions