Literature reviews and systematic reviews: a glossary

6S pyramid

Also called: 6S model, 6S hierarchy, 6S hierarchy of pre-appraised evidence

The 6S pyramid is a hierarchy of sources of evidence for practitioners, rising from individual studies at the base through synopses of studies, syntheses, synopses of syntheses and summaries to systems at the top. It ranks sources by how ready they are to use rather than by study design, since the lower levels take more time and expertise to find, appraise and apply.

Adequacy of data (CERQual)

Also called: adequacy (CERQual), data adequacy (CERQual)

Adequacy of data is the GRADE-CERQual component that judges how rich and how plentiful the data supporting a qualitative review finding are. A finding resting on thin data from a few studies warrants less confidence than one supported by detailed accounts, and adequacy is weighed with methodological limitations, coherence and relevance.

Aggregate data meta-analysis

Also called: AD meta-analysis, summary data meta-analysis, study-level meta-analysis

An aggregate data meta-analysis is one that combines the summary results reported for each study, such as a mean difference or an odds ratio with its confidence interval, rather than the records of individual participants. It is the usual kind of meta-analysis, quicker and cheaper than an individual participant data meta-analysis but limited to the analyses the studies chose to report.

Aggregative synthesis

Also called: aggregative review, aggregative logic, aggregating review

An aggregative synthesis is a review that adds up the findings of studies on concepts defined in advance, to test a theory or estimate an effect, as a meta-analysis of trials does. Gough, Thomas and Oliver contrast it with configurative synthesis: aggregative reviews favour exhaustive searching, methods fixed beforehand and similar studies, though many reviews mix both logics.

Albatross plot

An albatross plot is a graph that places each study by its sample size and P value, with contour lines showing the effect sizes those combinations imply. It needs only a two-sided P value, the sample size and the direction of effect, so it can include studies that report too little for a meta-analysis, and Cochrane lists it among displays for synthesis without meta-analysis.

AMSTAR 2

Also called: AMSTAR-2, AMSTAR, A MeaSurement Tool to Assess systematic Reviews

AMSTAR 2 is a 16-item checklist, published in 2017, for appraising the methodological quality of systematic reviews of healthcare interventions that include randomised or non-randomised studies, or both. It replaced the 11-item AMSTAR of 2007, rates overall confidence in a review as high, moderate, low or critically low from weaknesses in critical domains, and is not meant to produce a score.

Backward citation searching

Also called: backward citation tracking, backward snowballing, reference list checking, reference list searching, checking reference lists, footnote chasing

Backward citation searching is the checking of the reference lists of studies already found, and of related reviews, for further eligible studies. Cochrane makes checking the reference lists of included studies mandatory, and because authors may cite selectively, it supplements a database search rather than replacing it.

References from a file

Bayesian meta-analysis

Also called: Bayesian random-effects meta-analysis

A Bayesian meta-analysis is one that combines prior distributions for the effect, and often for the between-study variance, with the study data to give posterior distributions for them. The Cochrane Handbook notes that it can carry the full uncertainty about heterogeneity, which matters most when there are few studies, and it allows direct probability statements about the effect.

Best-fit framework synthesis

Also called: best fit framework synthesis, BFFS

Best-fit framework synthesis is a qualitative evidence synthesis that begins with an existing published model or theory, identified systematically as the closest fit to the review question, and codes the included studies' findings against it. Data that do not fit are analysed thematically to form new themes, and the result is a revised model for the new context.

Bibliographic coupling

Also called: bibliographic coupling analysis

Bibliographic coupling is a link between two documents that cite one or more of the same earlier works, and the more references they share, the more strongly they are coupled. Used in bibliometric analysis to group current research, it looks back through reference lists, whereas co-citation analysis links older works that later papers cite together.

Bibliographic database

Also called: academic database, research database, scholarly database, abstract and citation database

A bibliographic database is an indexed collection of records of published research, each giving the citation, usually the abstract and often subject headings, such as MEDLINE, Embase, APA PsycInfo, ERIC, Scopus or Web of Science. Systematic reviews search several because each covers different journals and indexes them differently, and Cochrane requires CENTRAL, MEDLINE and Embase where available.

Bibliometric analysis

Also called: bibliometrics, bibliometric review, bibliometric study

Bibliometric analysis is the statistical study of a body of publications and their metadata, such as authors, journals, keywords and citations, to describe a field's output or map the relationships within it. Unlike a systematic review it does not synthesise what the studies found. It may be evaluative, measuring output and impact, or relational, mapping networks such as co-citation or co-authorship.

Boolean operators

Also called: Boolean operator, Boolean logic, Boolean searching, AND OR NOT

Boolean operators are the words AND, OR and NOT used to combine terms in a database search: OR joins synonyms within one concept, AND joins different concepts so that every record contains each, and NOT removes records containing a term. Search guidance warns that NOT can silently discard relevant records, and brackets are used to nest terms correctly.

How to find sources for a literature review fast

Campbell Collaboration

Also called: Campbell, The Campbell Collaboration

The Campbell Collaboration is an international network that produces and publishes systematic reviews and evidence and gap maps on social and economic questions, in areas such as social welfare, disability, ageing, climate and management. It works through specialist coordinating groups, and its guide to searching for studies is derived from the Cochrane Handbook's chapter on searching.

CASP checklists

Also called: CASP, CASP checklist, CASP tools, Critical Appraisal Skills Programme

The CASP checklists are free critical appraisal checklists from the Critical Appraisal Skills Programme, a UK organisation, with a version for each common design, including randomised trials, cohort, case-control, diagnostic, qualitative and economic studies and systematic reviews. Each asks a set of questions about a study's methods and findings, and Cochrane's qualitative guidance cites the qualitative checklist.

CENTRAL

Also called: Cochrane Central Register of Controlled Trials, Cochrane CENTRAL

CENTRAL, the Cochrane Central Register of Controlled Trials, is a database of reports of randomised trials, built from systematic searches of MEDLINE, Embase and other databases, trials registers and other sources, and published in the Cochrane Library. Cochrane requires every intervention review to search it, and it includes trial reports that MEDLINE, Embase and other databases do not.

Certainty of evidence

Also called: quality of evidence, quality of the evidence, certainty of the evidence, certainty in the evidence

Certainty of evidence is the degree of confidence that the true effect lies close to the estimate a body of evidence gives, rated in GRADE as high, moderate, low or very low for each outcome. GRADE formerly called it quality of evidence, and it is a judgement about the whole body of evidence on an outcome, not about any single study.

CIMO

Also called: CIMO framework, Context, Intervention, Mechanism, Outcome

CIMO is a question framework standing for Context, Intervention, Mechanism and Outcome, used mainly in management and organisational research and in realist or mixed methods reviews. Its mechanism element asks why an intervention produces its outcome in a given context, which suits questions about what works, for whom and in what circumstances.

CINeMA

Also called: Confidence in Network Meta-Analysis, CINeMA framework

CINeMA, Confidence in Network Meta-Analysis, is a framework and online application for rating certainty in the results of a network meta-analysis across GRADE-like domains. It uses a contribution matrix, showing how much each direct comparison feeds into each network estimate, to weigh judgements such as risk of bias across the whole network.

Citation bias

Citation bias is the tendency for studies with positive or statistically significant results to be cited more often than studies with null or negative results on the same question. It matters to reviewers because a search that follows references can over-represent favourable studies, one reason citation searching supplements, and never replaces, database searching.

Citation searching

Also called: citation tracking, citation chasing, citation chaining, citation tracing, snowballing, snowball searching

Citation searching is the finding of further studies through their citation links with studies already known to be relevant, either by checking what those studies cite or by finding later work that cites them. Campbell's searching guide treats tracking, chasing, chaining and snowballing as names for the same method, and the TARCiS statement of 2024 gives guidance on its terms, use and reporting.

How to find sources for a literature review fast

Clinical heterogeneity

Also called: clinical diversity

Clinical heterogeneity is variation among the studies in a review in their participants, interventions and outcomes, such as different ages, doses or ways of measuring pain. The Cochrane Handbook calls it clinical diversity and distinguishes it from statistical heterogeneity, which it can cause, and from methodological diversity in study design and risk of bias.

Clinical study report

Also called: CSR, clinical study reports

A clinical study report is the full account of a clinical trial's design, conduct and results that a company submits to regulators when seeking approval for a medicine, often running to thousands of pages. Reviewers can obtain some from bodies such as the European Medicines Agency, and they often reveal outcomes that were measured but never published.

Co-citation analysis

Also called: cocitation analysis, co-citation

Co-citation analysis is a bibliometric method that links two works whenever a later paper cites both, treating works that are often cited together as related. It is used to map the intellectual structure and landmark works of a field, and it differs from bibliographic coupling, which links citing papers through the references they share.

Cochran's Q (heterogeneity test)

Also called: Cochran's Q statistic, Q statistic, Q test for heterogeneity, chi-squared test for heterogeneity, test of heterogeneity

Cochran's Q is the meta-analysis statistic that sums each study's weighted squared distance from the pooled estimate, tested against a chi-squared distribution with one fewer degrees of freedom than there are studies. It asks whether results vary more than chance allows, has low power when studies are few, and is distinct from Cochran's Q test for matched binary data.

Cochrane

Also called: Cochrane Collaboration, The Cochrane Collaboration

Cochrane is an international, independent, not-for-profit network of researchers, health professionals, patients and carers that produces systematic reviews of health interventions and the methods for doing them. It was founded in Oxford in 1993, is named after the health researcher Archie Cochrane, and does not accept funding for its reviews from commercial bodies with an interest in the results.

Cochrane Handbook

Also called: Cochrane Handbook for Systematic Reviews of Interventions, Cochrane Handbook version 6

The Cochrane Handbook for Systematic Reviews of Interventions is Cochrane's official guide to planning, conducting and reporting reviews of the effects of health interventions, from framing the question to meta-analysis and GRADE. Version 6.5 appeared in 2024 under senior editors Julian Higgins and James Thomas, and the Campbell Collaboration's searching guide is derived from it.

Cochrane Library

Also called: The Cochrane Library

The Cochrane Library is Cochrane's online collection of databases, which includes the Cochrane Database of Systematic Reviews, where every Cochrane protocol and review is published, and CENTRAL, the register of reports of controlled trials. Reviewers search it both for existing reviews on their question and, through CENTRAL, for trials.

Cochrane review

Also called: Cochrane systematic review, Cochrane Reviews

A Cochrane review is a systematic review prepared to Cochrane's methodological standards and published, after its protocol, in the Cochrane Database of Systematic Reviews. Most summarise the benefits and harms of health interventions and grade the certainty of the evidence, and Cochrane will not publish a review by a single author or funded by a company with an interest in the result.

Coherence (CERQual)

Also called: coherence (GRADE-CERQual)

Coherence is the GRADE-CERQual component that judges how well the underlying study data support a qualitative review finding, that is, whether the finding follows clearly and convincingly from them. A finding that some of the data contradict, or support only ambiguously, warrants less confidence, and coherence is weighed with methodological limitations, adequacy of data and relevance.

Combining P values

Also called: P value combination, Fisher's method, Fisher's combined probability test, combining probabilities

Combining P values is a synthesis method that merges the one-sided P values of several studies, for example with Fisher's method, to test whether any of the studies shows an effect. Cochrane accepts it when studies report only P values and directions of effect, but it says nothing about how large the effect is.

Configurative synthesis

Also called: configurative review, configuring review, configurative logic

A configurative synthesis is a review that arranges findings from varied studies into a new pattern or theory, as meta-ethnography and other interpretive syntheses do, rather than adding them up. Gough, Thomas and Oliver describe such reviews as exploratory and iterative, seeking enough variety of studies rather than every study, and interested in heterogeneity rather than homogeneity.

Consistency (network meta-analysis)

Also called: coherence (network meta-analysis), consistency assumption, network consistency

Consistency, or coherence, is the agreement between direct and indirect evidence about the same comparison in a network meta-analysis, the statistical counterpart of the transitivity assumption. Disagreement is called incoherence or inconsistency and is examined with methods such as node splitting, but the tests have low power, so finding none does not prove transitivity holds.

Context-mechanism-outcome configuration

Also called: CMO configuration, CMOC, CMO, context mechanism outcome

A context-mechanism-outcome configuration is a statement, used in realist reviews and evaluations, of how a mechanism triggered in a particular context produces a particular outcome. Realist reviewers build and test such configurations from the evidence they gather, in order to explain what works, for whom and in what circumstances rather than to estimate an average effect.

Contour-enhanced funnel plot

Also called: contour enhanced funnel plot

A contour-enhanced funnel plot is a funnel plot shaded to show where results would reach conventional levels of statistical significance, such as P below 0.1, 0.05 and 0.01. If the gaps in an asymmetric plot fall in the non-significant regions, publication bias is a more plausible cause, while gaps in significant regions point to other explanations.

Controlled vocabulary

Also called: controlled vocabularies, indexing vocabulary, database thesaurus

A controlled vocabulary is a fixed, usually hierarchical set of preferred terms that a database's indexers assign to records to describe their subjects, whatever words the authors used, such as MeSH in MEDLINE, Emtree in Embase or the ERIC Thesaurus. Searching it retrieves records on a concept consistently, and systematic searches combine it with free-text terms.

Corrected covered area

Also called: CCA

The corrected covered area is a measure of how much the systematic reviews in an overview of reviews overlap, calculated from a citation matrix that records which primary studies appear in which reviews. Overlap matters because the same trial counted in several reviews can be double-counted, so overview authors measure it and deal with it in their synthesis.

Critical appraisal

Also called: quality appraisal, quality assessment, methodological quality assessment, study appraisal

Critical appraisal is the systematic judgement of a study's trustworthiness, relevance and results, usually with a checklist suited to its design, such as the CASP or JBI tools. In a review it informs how much weight each study carries, and Cochrane prefers to assess risk of bias specifically rather than general quality, which mixes bias with reporting and precision.

Critical interpretive synthesis

Also called: CIS

Critical interpretive synthesis is a review method, developed by Dixon-Woods and colleagues in 2006 by adapting meta-ethnography, that builds a new theoretical account from a large and methodologically varied literature. It samples papers purposively and then theoretically, puts relevance ahead of methodological standards, and questions the literature's assumptions as well as summarising it, producing what its authors call a synthesising argument.

Critical review

Also called: critical literature review

A critical review is a review that goes beyond describing the literature to evaluate its quality and argue a position, often ending in a new hypothesis or model, in Grant and Booth's typology of reviews. Its search and appraisal are seldom systematic, so its value lies in its conceptual contribution rather than in methods another reader could repeat.

Cumulative meta-analysis

Also called: cumulative meta analysis

A cumulative meta-analysis is a meta-analysis repeated each time a new study is added, usually in order of publication, to show how the pooled estimate and its confidence interval changed over time. Lau and colleagues used it in 1992 to show that streptokinase's benefit after a heart attack was clear by 1973, years before the largest trials were run.

Data extraction

Also called: data collection (systematic review), data abstraction, study data extraction

Data extraction is the step in a systematic review in which reviewers record, from each included study, the details they need, such as its methods, participants, interventions, outcomes, results and funding. It is done on a planned and piloted form, and Cochrane requires at least two people to extract outcome data independently.

Data extraction form

Also called: extraction form, data collection form, extraction template, coding form

A data extraction form is the template, on paper, in a spreadsheet or in review software, on which reviewers record the same set of details from every included study. Cochrane advises piloting it on several studies with more than one extractor before use, and revising any question that proves unclear.

Deduplication

Also called: de-duplication, duplicate removal, removing duplicates, dedupe

Deduplication is the removal of repeated records of the same report retrieved from several databases, done in a reference manager or review software before screening. PRISMA 2020 asks reviewers to report how many duplicates were removed, and care is needed not to delete a separate report of the same study, which should be linked to it instead.

Thesis literature review, keeping hundreds of sources straight

DerSimonian and Laird method

Also called: DerSimonian-Laird, DerSimonian Laird, DL method, DL estimator

The DerSimonian and Laird method is a random-effects approach to meta-analysis, published in 1986, that estimates the between-study variance, tau squared, by a simple method of moments. It became so widely used that it is often called the standard approach, but the current Cochrane Handbook defaults to the restricted maximum likelihood estimator and recommends the Hartung-Knapp-Sidik-Jonkman method for confidence intervals.

Diagnostic test accuracy review

Also called: DTA review, DTA systematic review, diagnostic accuracy review, systematic review of diagnostic test accuracy

A diagnostic test accuracy review is a systematic review of studies that compare an index test with a reference standard in people who may have a target condition, summarising how well the test detects it, usually as sensitivity and specificity. Its studies are appraised with the QUADAS tools, and Cochrane publishes a separate handbook for these reviews.

Double data extraction

Also called: dual data extraction, duplicate data extraction, independent data extraction

Double data extraction is the independent recording of each study's data by two reviewers, whose forms are then compared and disagreements resolved by discussion or by a third person. Cochrane makes it mandatory for outcome data because it reduces both errors and the influence of one person's judgement, while rapid reviews often let a second reviewer check one extractor's work.

Dual screening

Also called: double screening, independent screening, duplicate screening, double independent screening

Dual screening is the practice of having two reviewers judge every record against the eligibility criteria independently, then compare their decisions and resolve disagreements, at both the title and abstract stage and the full-text stage. It reduces missed studies and single-person bias, and rapid reviews may relax it, for example by having a second reviewer check only the exclusions.

Duplicate publication bias

Also called: multiple publication bias

Duplicate publication bias is the distortion that arises when the same study's results appear in more than one publication, often favourable ones, so that a review counts them twice. Reviewers guard against it by linking every report to its underlying study, since the study rather than the report is the unit of a systematic review.

ECLIPSE

Also called: ECLIPSE framework, ECLIPSE question framework

ECLIPSE is a question framework for reviews about services and policies, standing for Expectation, Client group, Location, Impact, Professionals and Service. Published by Wildridge and Bell in 2002, it suits questions about how a service is organised or delivered and what it achieves, where a clinical framework such as PICO fits poorly.

Effect direction plot

An effect direction plot is a table-like display that shows, for each study and outcome or outcome domain, whether the effect favoured the intervention, favoured the comparator or was unclear, using arrows whose size or colour can carry further information such as sample size. Cochrane lists it among visual displays for syntheses that cannot use meta-analysis.

Egger's test

Also called: Egger test, Egger's regression test, Egger regression, Egger's regression asymmetry test

Egger's test is a regression test for funnel plot asymmetry, published in 1997, that regresses each study's standardised effect on its precision and asks whether the intercept differs from zero. A significant result signals small-study effects rather than proving publication bias, it has low power with fewer than about ten studies, and Cochrane advises other tests for odds ratios.

Eligibility criteria (systematic review)

Also called: study eligibility criteria, inclusion and exclusion criteria (review), review inclusion criteria, selection criteria (review)

Eligibility criteria in a systematic review are the rules, set in the protocol, that decide which studies are included, usually built from the review's population, interventions, comparisons and study designs. Cochrane advises against making reported outcomes a criterion, since excluding studies that do not report an outcome can bias a review, and the criteria concern studies rather than individual participants.

eMERGe

Also called: eMERGe reporting guidance, eMERGe guidance

eMERGe is the reporting guidance for meta-ethnography, published in 2019, with 19 criteria organised around the method's seven phases. It asks authors to explain, among other things, how studies were chosen, how reciprocal and refutational translation were done and how the line-of-argument synthesis was reached.

Emtree

Also called: Emtree thesaurus, Emtree terms, Embase subject headings

Emtree is Elsevier's hierarchical thesaurus of life science terms, used to index the records in Embase, with more than 100,000 preferred terms, including every MeSH term, and many synonyms. An Embase search combines exploded Emtree terms with free-text words, and a MEDLINE strategy has to be translated for it because the two vocabularies differ.

ENTREQ

Also called: ENTREQ statement, Enhancing Transparency in Reporting the Synthesis of Qualitative Research

ENTREQ, Enhancing Transparency in Reporting the Synthesis of Qualitative Research, is a 21-item reporting guideline for qualitative evidence syntheses, published by Tong and colleagues in 2012. Its items cover the introduction, methods and methodology, searching and selection, appraisal and the synthesis of findings, whichever synthesis method was used.

Evidence and gap map

Also called: evidence gap map, EGM, gap map, evidence and gap maps

An evidence and gap map is a systematic, usually interactive, display of all the evidence of a given kind in a field, typically a matrix of interventions against outcomes whose cells show the studies and reviews available. The Campbell Collaboration uses them to show where evidence is plentiful and where it is missing, not what the evidence says.

Evidence map

Also called: evidence mapping

An evidence map is the product of a systematic search across a wide field that shows what research exists and where the gaps are, in a user-friendly form such as a figure, a table or a searchable database. A 2016 review of published evidence maps found the term used inconsistently, and the evidence and gap map is one standardised kind.

Evidence profile

Also called: GRADE evidence profile, GRADE profile

An evidence profile is a detailed GRADE table that sets out, for each outcome, the judgement on every domain of certainty alongside the study results. It supports the shorter summary of findings table by giving the fuller reasoning behind each certainty rating.

Evidence synthesis

Also called: research synthesis, knowledge synthesis, synthesis of evidence

Evidence synthesis is the family of methods that bring together the findings of many studies in a transparent way to answer a question, including systematic reviews, meta-analyses, scoping reviews, qualitative evidence syntheses and evidence maps. The methods differ in breadth and depth, and in whether they aggregate results or configure them into new understanding.

Evidence-based medicine

Also called: EBM, evidence based medicine

Evidence-based medicine is the practice of basing decisions about individual patients on the best current research evidence, weighed together with clinical expertise and the patient's circumstances. The term entered the medical literature in 1991, and its methods, including critical appraisal and the use of systematic reviews, were later widened into evidence-based practice across the health professions.

Evidence-based practice

Also called: EBP, evidence based practice

Evidence-based practice is the making of professional decisions from the best available, current and relevant research, combined with practitioners' knowledge and the choices of the people receiving care, within the resources available. The Sicily statement of 2005 set out its five steps: ask an answerable question, find the evidence, appraise it, apply it and evaluate the result.

Explode (subject heading)

Also called: explode, exploding a subject heading, exploded MeSH term, exploded search

To explode a subject heading is to search for it together with all the narrower headings beneath it in the thesaurus hierarchy, so that a search on a broad MeSH or Emtree term also retrieves records indexed under its more specific terms. The Cochrane Handbook stresses exploding headings where appropriate so that relevant records are not missed.

Fail-safe N

Also called: failsafe N, fail-safe number, Rosenthal's fail-safe N

Fail-safe N is the number of unpublished studies with null results that would have to exist to make a meta-analysis's combined result no longer statistically significant, proposed by Rosenthal in 1979 in response to the file drawer problem. It is now heavily criticised, because different versions give very different numbers, there is no benchmark for judging it, and Rosenthal's version addresses significance rather than effect size.

Field searching

Also called: field search, searching by field, field codes, field tags

Field searching is restricting a database search term to particular parts of each record, such as the title, abstract, author keywords or journal name, using the database's field codes. Limiting free-text terms to the title and abstract makes a search more precise, but every database has its own syntax, so the codes must be adapted for each source.

Fixed-effect model (meta-analysis)

Also called: fixed-effect meta-analysis, fixed effect model, common-effect model, common effect model, equal-effect model

A fixed-effect model is a meta-analysis model that assumes every study estimates one identical true effect, so that differences between results are due to chance alone. The Cochrane Handbook also calls it a common-effect model, and when heterogeneity is present it gives a narrower confidence interval than a random-effects model, which can overstate certainty.

Forest plot

Also called: blobbogram, forest diagram

A forest plot is the standard graph of a meta-analysis, showing each study's effect estimate as a square sized by its weight with a horizontal line for its confidence interval, and the pooled result as a diamond at the bottom. A vertical line marks no effect, so readers can see at a glance which studies favour which side and how much they vary.

Forward citation searching

Also called: forward citation tracking, forward snowballing, cited reference searching, cited-by searching, forward chaining

Forward citation searching is the finding of later publications that cite a study already known to be relevant, using citation indexes such as Web of Science and Scopus or a Cited by link. It finds newer work that a database search may have missed, and it complements backward citation searching of reference lists.

Framework synthesis

Also called: framework-based synthesis

Framework synthesis is a qualitative evidence synthesis that organises and charts the findings of included studies against a framework of themes chosen at the start, drawn from theory or earlier work, and refined as the data are coded. It is more deductive than thematic synthesis, and best-fit framework synthesis is a variant that chooses its starting framework systematically.

Free-text searching

Also called: free text searching, free-text terms, keyword searching, text word searching, textword searching, natural language searching

Free-text searching is searching for the words authors actually use, in the title, abstract and other fields of each record, as opposed to the subject headings a database assigns. Systematic searches use both, adding synonyms, spelling variants, truncation and proximity operators to the free-text terms, because either method alone misses relevant records.

Full-text screening

Also called: full text screening, full-text review, full-text assessment, second-stage screening

Full-text screening is the second stage of study selection in a systematic review, in which the complete reports of records that passed title and abstract screening are read and judged against the eligibility criteria. PRISMA 2020 asks reviewers to report how many full texts were assessed and the reasons for excluding those that were rejected.

Funnel plot

Also called: funnel graph

A funnel plot is a scatter plot of the studies in a meta-analysis, with each study's effect estimate against its standard error, drawn so that large precise studies sit at the top and small ones spread out below. Without bias the points form a symmetrical inverted funnel, and asymmetry suggests small-study effects, of which publication bias is only one possible cause.

GRADE

Also called: GRADE approach, Grading of Recommendations Assessment, Development and Evaluation, GRADE system, GRADE framework

GRADE is a system for rating the certainty of a body of evidence and the strength of recommendations, developed by the GRADE Working Group. For each outcome, evidence from randomised trials starts as high certainty and from observational studies usually as low, then is rated down for risk of bias, inconsistency, indirectness, imprecision or publication bias, and occasionally rated up.

GRADE-CERQual

Also called: CERQual, Confidence in the Evidence from Reviews of Qualitative research

GRADE-CERQual is an approach for judging how much confidence to place in each finding of a qualitative evidence synthesis, rating it high, moderate, low or very low. It considers four components, methodological limitations, coherence, adequacy of data and relevance, and complements GRADE, which rates evidence about effects.

Grey literature searching

Also called: grey literature search, gray literature search, gray literature searching, searching grey literature

Grey literature searching is the part of a review's search that looks beyond commercially published journals and books for reports, theses, conference abstracts, preprints and documents from government and other organisations. Studies published outside journals tend to show smaller effects, so it reduces publication bias, but it has no natural end, and reviewers plan and justify the sources they use.

Hand searching

Also called: handsearching, hand-searching, manual searching

Hand searching is the page-by-page examination of every issue of chosen journals, or of conference proceedings, over a set period, to find eligible studies that database searches miss. It is labour-intensive, so reviewers usually limit it to journals where many included studies have appeared, and it can now be done by scrolling electronic tables of contents.

Hartung-Knapp-Sidik-Jonkman method

Also called: HKSJ, HKSJ adjustment, Hartung-Knapp adjustment, Knapp-Hartung adjustment, Hartung-Knapp method

The Hartung-Knapp-Sidik-Jonkman method is a way of calculating the confidence interval for the pooled result of a random-effects meta-analysis that better reflects uncertainty in the estimated between-study variance. The current Cochrane Handbook recommends it when there are more than two studies and the estimated heterogeneity is above zero.

Harvest plot

A harvest plot is a chart that groups the studies in a synthesis by whether their results suggest benefit, harm or no effect, drawing each study as a bar whose height or shading can show features such as risk of bias or study size. It extends vote counting visually and is used when a meta-analysis is not possible.

Heterogeneity (meta-analysis)

Also called: statistical heterogeneity, heterogeneity, between-study heterogeneity, between-study variation

Heterogeneity in a meta-analysis is variation between studies' results greater than chance alone would produce, suggesting that the true effect differs between studies. It is measured with Cochran's Q, I squared, tau squared and the prediction interval, arises from clinical or methodological diversity, and should be explored, for example by subgroup analysis or meta-regression, rather than ignored.

I squared (I²)

Also called: I², I2, I-squared, I2 statistic, I-squared statistic, Higgins I2

I squared is a statistic that estimates the percentage of the variation between study results in a meta-analysis that reflects real differences rather than chance, which Cochrane reviews began reporting around 2003. The Cochrane Handbook's rough guide reads 0 to 40 per cent as possibly unimportant and 75 to 100 as considerable, with overlapping bands in between, and its value rises as studies grow more precise.

Imprecision (GRADE)

Also called: imprecision

Imprecision, in GRADE, is a reason for lowering certainty when a body of evidence has few participants or events, so that the confidence interval is wide enough to include effects that would lead to different decisions. It is one of the five GRADE domains for rating down, alongside risk of bias, inconsistency, indirectness and publication bias.

Inconsistency (GRADE)

Also called: inconsistency of results

Inconsistency, in GRADE, is a reason for lowering certainty when studies' effect estimates differ widely and the differences cannot be explained, shown for example by confidence intervals that barely overlap or a high I squared. It concerns heterogeneity within a body of evidence and differs from inconsistency, or incoherence, between direct and indirect evidence in a network meta-analysis.

Indirect comparison

Also called: adjusted indirect comparison, indirect treatment comparison, ITC, Bucher method

An indirect comparison estimates the relative effect of two interventions that have never been compared head to head, by combining each one's effect against a common comparator, such as B against A and C against A. The adjusted method of Bucher keeps the benefit of randomisation within each trial, and its validity rests on the transitivity assumption.

Indirectness (GRADE)

Also called: indirectness, indirectness of evidence

Indirectness, in GRADE, is a reason for lowering certainty when the available evidence answers a somewhat different question from the one asked, for example because the studies used another population, a related intervention, a different comparator or a surrogate outcome. It asks how directly the evidence applies to the review question and the decision it will inform.

Individual participant data meta-analysis

Also called: IPD meta-analysis, individual patient data meta-analysis, IPD review, IPD-MA, individual participant data review

An individual participant data meta-analysis is one that collects, checks and reanalyses the original records of each participant in each study, usually obtained from the investigators, rather than published summaries. It allows common definitions, time-to-event analyses and tests of whether participant characteristics change the effect, but typically takes two years or more and dedicated staff.

Information specialist

Also called: health sciences librarian, medical librarian, search specialist, review librarian

An information specialist is a librarian or information professional who designs, runs and documents the searches for a systematic review and advises on sources, syntax and reporting. Cochrane advises review teams to work with one from the protocol onwards and to have search strategies peer reviewed by another before they are run.

Integrative review

Also called: integrative literature review

An integrative review is a review that summarises past empirical and theoretical literature, drawing experimental and non-experimental studies together, to give a fuller understanding of a phenomenon or problem. Associated with Whittemore and Knafl's 2005 method and common in nursing, it suits broad aims such as defining concepts or reviewing theories, and it has no reporting guideline of its own.

Inverse-variance method

Also called: inverse variance method, inverse-variance weighting, inverse variance weighting, IV method

The inverse-variance method is the standard way of pooling studies in a meta-analysis, in which each study's weight is one divided by the square of its standard error. Larger, more precise studies therefore count for more, and the method is used in both fixed-effect and random-effects models, the latter adding the between-study variance to each study's own variance.

JBI

Also called: Joanna Briggs Institute

JBI, formerly the Joanna Briggs Institute, is an evidence-based healthcare organisation at Adelaide University in Australia that develops methods for evidence synthesis and publishes the JBI Manual for Evidence Synthesis. Its guidance covers systematic, scoping, umbrella, mixed methods and qualitative reviews, the last by meta-aggregation, and it leads an international collaboration of universities and hospitals.

JBI critical appraisal tools

Also called: JBI checklists, JBI appraisal checklists, Joanna Briggs Institute checklists

The JBI critical appraisal tools are free checklists, one for each study design, for judging the trustworthiness, relevance and results of published research, from randomised trials and cohort studies to qualitative research, case reports and expert opinion. Several were revised from 2023 onwards, and the revised trial tool is framed as an assessment of risk of bias.

Language bias

Also called: English language bias, English-language bias

Language bias is the distortion that arises because studies published in English, which are more likely to report positive results, are easier to find and more often included than studies in other languages. Reviews reduce it by not restricting searches by language, and the Cochrane Handbook advises applying any language restriction at screening rather than in the search.

Leave-one-out analysis

Also called: leave-one-out meta-analysis, leave one out analysis, one-study-removed analysis

A leave-one-out analysis is a form of influence analysis that repeats a meta-analysis as many times as there are studies, leaving a different study out each time, to see how much any single study changes the pooled result or the heterogeneity. It is often reported as a sensitivity analysis, and a result that hinges on one study deserves caution.

Line-of-argument synthesis

Also called: line of argument synthesis, lines-of-argument synthesis, line of argument

A line-of-argument synthesis is the stage of a meta-ethnography that puts the translated concepts from the studies together into a new overarching interpretation, or storyline, of the phenomenon. It goes beyond reciprocal and refutational translation, and eMERGe asks authors to say whether one was produced and, if not, why not.

Literature review

Also called: lit review, review of the literature, review of literature

A literature review is a written account of what published research says about a topic, which summarises, compares and evaluates sources to show the state of knowledge and where the gaps lie. The term covers everything from a short narrative overview to a full systematic review, so a reader should check which methods were actually used.

Literature review chapter

Also called: dissertation literature review, thesis literature review, literature review section, lit review chapter

A literature review chapter is the part of a dissertation or thesis that sets out, critically and in organised themes, what existing research has found about the topic, and shows the gap the new study will fill. University guides stress synthesis rather than a list of summaries, often moving from broad themes to the specific question, and the chapter also frames the chosen theory and methods.

Managing dissertation references from proposal to submission

Literature search

Also called: literature searching, lit search, database searching

A literature search is the planned search for publications relevant to a question, across databases, citation links, reference lists and other sources, with the terms, dates and results recorded so that it can be repeated. In a systematic review it is designed to be comprehensive and reported in full, while a shorter documented search may suffice for an essay.

Find sources

Living systematic review

Also called: LSR, living review, living evidence synthesis

A living systematic review is a systematic review that is kept continually up to date, with searches run at frequent intervals and new evidence incorporated as it appears. Elliott and colleagues proposed the approach in 2017 for fields where research is emerging quickly, current evidence is uncertain and new results could change policy or practice.

Location bias

Also called: journal location bias

Location bias is the distortion that arises when studies are easier or harder to find because of where they were published, for example because some journals are indexed in the main databases and others are not. If studies with favourable results tend to appear in the more accessible journals, a review searching only major databases will overstate effects.

Lumping and splitting

Also called: lumping versus splitting, lumpers and splitters

Lumping and splitting are the two ends of a choice about a review's scope: lumping asks one broad question that combines varied interventions or populations, while splitting asks narrow questions about each separately. The Cochrane Handbook notes that broad reviews can explore what explains differing effects, while narrow ones are easier to manage.

Mantel-Haenszel method (meta-analysis)

Also called: Mantel-Haenszel meta-analysis, Mantel-Haenszel method, M-H method

The Mantel-Haenszel method is a fixed-effect way of pooling dichotomous outcomes, such as risk ratios or odds ratios, in a meta-analysis, using its own weighting scheme rather than inverse variance. The Cochrane Handbook prefers it when events are sparse or studies small, because it then has better statistical properties than the inverse-variance method.

Mapping review

Also called: systematic map, systematic mapping review, systematic mapping study, mapping study

A mapping review is a systematic review that describes and categorises the research on a topic, for example by design, population or setting, to show what has been studied and where the gaps are, without synthesising the findings. Gough and colleagues note that a systematic map can be a product in its own right or a first step towards narrower syntheses.

MECIR

Also called: Methodological Expectations of Cochrane Intervention Reviews, MECIR standards

MECIR, the Methodological Expectations of Cochrane Intervention Reviews, is Cochrane's set of standards for conducting and reporting its intervention reviews, each marked mandatory or highly desirable. The Cochrane Handbook cites them throughout, for example making searches of CENTRAL, MEDLINE and Embase mandatory, and the Campbell Collaboration keeps a parallel set called MECCIR.

MeSH

Also called: Medical Subject Headings, MeSH terms, MeSH term, MeSH headings

MeSH, the Medical Subject Headings, is the controlled and hierarchical vocabulary produced by the US National Library of Medicine to index, catalogue and search records in MEDLINE, PubMed and other NLM databases, and it is updated every year. A systematic search combines exploded MeSH terms with free-text words, because either alone misses relevant records.

Meta-aggregation

Also called: meta-aggregative review, JBI meta-aggregation, meta aggregation

Meta-aggregation is JBI's method of qualitative evidence synthesis, in which reviewers present the findings of included studies as their authors intended, without reinterpreting them, grouping similar findings into categories and the categories into synthesised findings. Grounded in pragmatism, it aims to produce statements that can guide action in practice or policy.

Meta-analysis

Also called: meta analysis, metaanalysis, meta-analyses

A meta-analysis is the statistical combination of results from two or more separate studies of the same question into a single pooled estimate with a confidence interval. It is usually one part of a systematic review, and it is only as sound as the review behind it: the search, the choice of studies and the judgement that they are similar enough to combine.

Meta-ethnography

Also called: meta ethnography, metaethnography

Meta-ethnography is an interpretive method of qualitative evidence synthesis, set out by Noblit and Hare in 1988, that compares and translates concepts across qualitative studies to build interpretations that go beyond any single study. It has seven phases, including reciprocal and refutational translation and a line-of-argument synthesis, and is among the most used synthesis methods in health research.

Meta-narrative review

Also called: meta-narrative synthesis, metanarrative review

A meta-narrative review is a systematic review, developed by Greenhalgh and colleagues in 2005, for topics that different research traditions have conceptualised and studied in different ways. Drawing on Kuhn's idea of paradigms, it traces the storyline of each tradition over time and then compares them, and the RAMESES project published reporting standards for it in 2013.

Meta-regression

Also called: meta regression, metaregression

Meta-regression is a regression analysis, within a meta-analysis, of whether study characteristics such as dose, duration or risk of bias explain differences between the studies' effect estimates. Its associations are observational across studies rather than randomised, so they cannot show cause, and the Cochrane Handbook advises against it with fewer than about ten studies.

Meta-synthesis

Also called: metasynthesis, qualitative meta-synthesis, qualitative metasynthesis

A meta-synthesis is a synthesis of findings from qualitative studies that aims at an interpretation greater than the sum of its parts rather than a summary. Usage varies: some authors use the word for any qualitative evidence synthesis, while others mean a particular interpretive approach, such as that associated with Margarete Sandelowski, so a paper's stated method matters more than its label.

Methodological heterogeneity

Also called: methodological diversity

Methodological heterogeneity is variation in how the studies in a review were designed and conducted, such as differences in blinding, follow-up or the instruments used to measure an outcome. The Cochrane Handbook calls it methodological diversity, and like clinical diversity it can produce statistical heterogeneity, so it is examined before deciding whether studies can sensibly be pooled.

Methodological limitations (CERQual)

Also called: methodological limitations (GRADE-CERQual)

Methodological limitations is the GRADE-CERQual component that judges how much flaws in the way the contributing primary studies were designed or carried out should lower confidence in a qualitative review finding. It is often informed by an appraisal tool such as the CASP qualitative checklist, and it is weighed with coherence, adequacy of data and relevance.

Mixed-methods review

Also called: mixed methods systematic review, mixed-methods systematic review, mixed studies review, MMSR

A mixed-methods review is a systematic review that includes quantitative, qualitative and mixed methods studies and brings their findings together to answer a complex question more fully than either kind could alone. JBI's guidance takes a convergent approach, either transforming data so both kinds can be combined, or synthesising them separately and then integrating the two sets of findings.

MMAT

Also called: Mixed Methods Appraisal Tool, MMAT 2018, MMAT version 2018

The Mixed Methods Appraisal Tool is a critical appraisal checklist, developed by a team at McGill University, for reviews that include studies of different designs, with five criteria each for qualitative, randomised, non-randomised, quantitative descriptive and mixed methods studies. Its 2018 version adds two screening questions, and its authors discourage an overall numerical score.

MOOSE

Also called: MOOSE guidelines, MOOSE checklist, Meta-analysis Of Observational Studies in Epidemiology

MOOSE, Meta-analysis Of Observational Studies in Epidemiology, is a reporting checklist for meta-analyses of observational studies, published by Stroup and colleagues in JAMA in 2000 after a workshop held in 1997. It covers the background, search strategy, methods, results, discussion and conclusions of such a review.

Narrative review

Also called: traditional literature review, traditional review, narrative literature review, non-systematic review

A narrative review is a conventional literature review in which an author selects and discusses studies on a topic without a pre-specified, reproducible method for searching, selecting or appraising them. It can give a readable overview and expert interpretation, but because its choices are not made explicit, it is more open to bias than a systematic review.

Narrative synthesis

Also called: narrative summary

Narrative synthesis is a way of synthesising the studies in a systematic review mainly in words and tables rather than statistics, used when a meta-analysis is not possible. Guidance by Popay and colleagues in 2006 gives it four elements, from developing a theory to assessing robustness, but Cochrane warns that the label alone describes nothing and asks authors to name the actual methods.

Network diagram

Also called: network plot, network graph, network geometry

A network diagram is a figure used in network meta-analysis in which each intervention is a node and each line joins two interventions that have been compared directly in trials. The thickness of the lines or the size of the nodes often shows the number of studies or participants, so gaps and weak links in the evidence are visible at once.

Network meta-analysis

Also called: NMA, mixed treatment comparison, MTC, multiple treatments meta-analysis, multiple-treatments meta-analysis

A network meta-analysis is a meta-analysis that compares three or more interventions at once by combining direct evidence from head-to-head trials with indirect evidence through common comparators. It can estimate effects for pairs never compared directly and rank the options, but its results are trustworthy only if the transitivity assumption holds and direct and indirect evidence agree.

New evidence pyramid

Also called: revised evidence pyramid, modified evidence pyramid

The new evidence pyramid is a revision of the traditional hierarchy of evidence, proposed by Murad and colleagues in 2016, with wavy lines between study designs and systematic reviews removed from the top. The wavy lines reflect GRADE's rating up and down, since design alone does not settle certainty, and reviews become a lens through which the rest of the evidence is appraised and applied.

Newcastle-Ottawa Scale

Also called: NOS, Newcastle Ottawa Scale, Newcastle-Ottawa quality assessment scale

The Newcastle-Ottawa Scale is a tool, developed by the universities of Newcastle in Australia and Ottawa in Canada, for assessing the quality of non-randomised studies, in versions for cohort and case-control studies. It awards stars for the selection of groups, their comparability and the ascertainment of exposure or outcome, though AMSTAR 2's authors caution against using its totals as a score.

Non-reporting bias

Also called: bias due to missing results, bias due to missing evidence, missing evidence bias

Non-reporting bias is the Cochrane Handbook's term for bias that arises when decisions about whether, when or where to report a study's results depend on their P value, size or direction, leaving results missing from a meta-analysis. It covers whole studies left unpublished and results left out of published studies, and the Handbook prefers it to the looser term reporting bias.

OSF Registries

Also called: OSF registration, Open Science Framework registration, OSF preregistration, OSF Registry

OSF Registries is the registration service of the Open Science Framework, where researchers can publicly time-stamp a protocol, including a review protocol of any type or discipline, using a general-purpose template for systematic reviews. Unlike PROSPERO it has no editorial check before registration and accepts scoping reviews, so it is used for reviews that PROSPERO does not take.

Outcome reporting bias

Also called: selective outcome reporting, selective reporting, selective reporting bias, outcome switching, selective non-reporting

Outcome reporting bias is the selective reporting of a study's outcomes, or of analyses of them, according to whether the results were favourable or statistically significant, for example by dropping or demoting a pre-specified primary outcome. Reviewers detect it by comparing protocols and trial register entries with the published reports.

P-curve analysis

Also called: p-curve, p curve

P-curve analysis is a method that examines the distribution of the statistically significant P values across a set of studies to judge whether the findings reflect a true effect. A curve bunched towards very small P values suggests evidential value, while a flat curve suggests no effect or practices such as P-hacking, which funnel plot methods cannot detect.

PCC

Also called: Population, Concept, Context, PCC framework, PCC mnemonic

PCC is the question framework JBI recommends for scoping reviews, standing for Population, Concept and Context. It needs no separate intervention, comparison or outcome, since a scoping review maps what exists rather than testing an effect, and any of these can sit within the concept, while the context may be a setting, a culture or a place.

Pearl growing

Also called: citation pearl growing, pearl-growing, traditional pearl growing

Pearl growing is a search technique that starts from one highly relevant source and grows the search from it, by borrowing its subject headings and keywords for new searches and by following its references and the works that cite it. It is useful early in a search or on a new topic, and it overlaps with citation searching, though it also mines the source's terms.

PECO

Also called: PECOS, Population, Exposure, Comparator, Outcome

PECO is a question framework for reviews of exposures rather than interventions, standing for Population, Exposure, Comparator and Outcome, with PECOS adding Study design. It is used in environmental and occupational health and other questions about risk, where the exposure is something people experience rather than something a researcher assigns.

PEO

Also called: Population, Exposure, Outcome

PEO is a question framework standing for Population, Exposure and Outcome, used for questions about causes, risks or experiences where there is no obvious comparison group. Library guides recommend it for aetiology questions and some qualitative reviews, and it is a simpler relative of PECO.

PerSPEcTiF

Also called: PerSPEcTiF framework

PerSPEcTiF is a question framework for qualitative evidence syntheses of complex interventions, standing for Perspective, Setting, Phenomenon of interest, Environment, Comparison (optional), Time or timing, and Findings. Proposed by Booth and colleagues in 2019, it adds the viewpoint and wider context that questions about complex interventions need.

PET-PEESE

Also called: PET PEESE, precision-effect test, precision-effect estimate with standard error

PET-PEESE is a regression method for publication bias that estimates the effect expected in a study of perfect precision, using the precision-effect test and, if that finds an effect, the precision-effect estimate with standard error. It has been criticised for overcorrecting and for performing poorly with few studies or high heterogeneity.

Peto method

Also called: Peto odds ratio, Peto one-step method, Peto's method

The Peto method is a way of pooling odds ratios in a meta-analysis using an approximation that works well when events are very rare, below about one in a hundred. The Cochrane Handbook advises it only when effects are not very large and trial arms are of similar size, and advises against it for common events.

Phrase searching

Also called: phrase search, exact phrase searching

Phrase searching is enclosing two or more words in quotation marks so that a database retrieves them only as that exact phrase, in that order, such as "parental involvement". It is more precise than joining the words with AND but less sensitive than a proximity operator, which also finds the words close together in another order.

PICO

Also called: PICO framework, PICO question, Population, Intervention, Comparison, Outcome

PICO is the standard framework for a focused question about an intervention, standing for Population, Intervention, Comparison and Outcome, as in adults with depression, exercise, usual care and symptom scores. Cochrane builds a review's eligibility criteria from its PICO elements plus the types of study, and distinguishes the review's PICO from the PICO of each synthesis.

PICo (qualitative)

Also called: PICo framework, Population, phenomenon of Interest, Context

PICo is a question framework for qualitative reviews, standing for Population, phenomenon of Interest and Context, written with a lower-case o to set it apart from PICO. It suits reviews of experiences and meanings, and it drops the comparison and outcome, which rarely fit qualitative questions.

PICOS

Also called: PICOS framework, PICOS question

PICOS is the PICO framework with S for study design added, so that the question also states which kinds of study will be included, such as randomised trials only. It is common in the eligibility criteria of systematic reviews, and the Campbell Collaboration's guidance on evidence and gap maps relates a map's rows and columns to it.

PICOT

Also called: PICOT question, PICOT format

PICOT is the PICO framework with T for time added, meaning the period over which the intervention is given or the outcome measured, as in symptom scores at six months. The Sicily statement on evidence-based practice lists such five-part questions among the formats used to teach question framing.

Pooled estimate

Also called: summary estimate, summary effect, pooled effect, pooled effect size, combined estimate, overall effect estimate

A pooled estimate is the single combined result of a meta-analysis, a weighted average of the studies' effect estimates with its own confidence interval, shown as the diamond on a forest plot. Under a fixed-effect model it estimates one common effect, and under a random-effects model the average of a distribution of effects.

Precision (of a search)

Also called: search precision, precision of a search

The precision of a search is the proportion of the records it retrieves that are relevant, the number of relevant reports found divided by all reports found. Systematic review searches trade precision for sensitivity, accepting many irrelevant records so as not to miss studies, since abstracts can be screened fairly quickly.

Prediction interval (meta-analysis)

Also called: prediction interval, 95% prediction interval

A prediction interval in a meta-analysis is the range within which the true effect in a new, similar study is expected to lie, reflecting the spread of effects between studies as well as uncertainty in their average. It is wider than the confidence interval whenever there is heterogeneity, and can show that an effect which is beneficial on average may be absent or harmful in some settings.

PRESS

Also called: Peer Review of Electronic Search Strategies, PRESS checklist, PRESS 2015, search peer review

PRESS, Peer Review of Electronic Search Strategies, is an evidence-based guideline and checklist, updated in 2015, for having a second information specialist check a systematic review's search strategy before it is run. It covers translating the question, Boolean and proximity operators, subject headings, text words, spelling, syntax and line numbers, and limits and filters.

PRISMA 2020

Also called: PRISMA, PRISMA statement, PRISMA 2020 statement, Preferred Reporting Items for Systematic Reviews and Meta-Analyses, PRISMA checklist, PRISMA guidelines

PRISMA 2020 is the current reporting guideline for systematic reviews, Preferred Reporting Items for Systematic reviews and Meta-Analyses, published by Page and colleagues in 2021 to replace the 2009 statement. It has a 27-item checklist, a 12-item checklist for abstracts and a flow diagram, and extensions adapt it for particular kinds of review.

PRISMA flow diagram

Also called: PRISMA flowchart, PRISMA flow chart, PRISMA diagram, study flow diagram

A PRISMA flow diagram is the figure in a systematic review that counts records and reports through each stage of selection, from those identified in databases, registers and other sources, through duplicates removed and screening, to the studies finally included, with reasons for exclusions. The 2020 version shows database searches and other methods, such as citation searching, in separate columns.

PRISMA-P

Also called: PRISMA-P 2015, PRISMA for protocols, Preferred Reporting Items for Systematic review and Meta-Analysis Protocols

PRISMA-P is the reporting guideline for systematic review protocols, published by Moher and colleagues in 2015, with 17 numbered items, or 26 counting sub-items, grouped as administrative information, introduction and methods. It sets out the minimum a protocol should contain, and is meant to be used when preparing a protocol before registering the review, for example in PROSPERO.

PRISMA-S

Also called: PRISMA-S checklist, PRISMA literature search extension, PRISMA Search

PRISMA-S is the extension of PRISMA for reporting the literature searches of systematic reviews, published by Rethlefsen and colleagues in 2021, with 16 items. It asks for every database and other source, the full search strategies, limits, filters, peer review of the search and how records were managed and deduplicated, so that the searches can be reproduced.

PRISMA-ScR

Also called: PRISMA extension for scoping reviews, PRISMA ScR

PRISMA-ScR is the extension of PRISMA for scoping reviews, published by Tricco and colleagues in 2018, with 20 essential reporting items and 2 optional ones. It adapts the systematic review checklist to reviews that map a body of literature, and the optional items concern critical appraisal of the included sources, which scoping reviews often omit.

Problematic study

Also called: potentially problematic study, untrustworthy study, study with integrity concerns

A problematic study, in Cochrane's terms, is a study whose trustworthiness is in doubt because of suspected misconduct, such as fabricated or falsified data, or because it has been retracted. Cochrane requires reviewers to examine retraction notices and errata for included studies, to set suspect studies aside while they are investigated, and to exclude those shown to be unreliable.

Retracted papers in your reference list

Prospective meta-analysis

Also called: PMA

A prospective meta-analysis is a systematic review and meta-analysis of studies identified and judged eligible before any of their results are known, so that its questions, criteria and analyses are fixed without knowledge of the findings. Most collect individual participant data, and Cochrane requires each to be registered in PROSPERO with a public protocol.

PROSPERO

Also called: International Prospective Register of Systematic Reviews, PROSPERO registration

PROSPERO is the International Prospective Register of Systematic Reviews, run by the Centre for Reviews and Dissemination at the University of York, where authors record a planned review's question and methods before carrying it out. It accepts systematic, rapid and umbrella reviews in health, social care and related fields, but not scoping reviews, which are often registered on OSF instead.

Protocol registration (systematic review)

Also called: review registration, systematic review registration, registering a review, prospective registration of a review

Protocol registration is the public, dated recording of a planned review's question, eligibility criteria and methods in a register such as PROSPERO or OSF Registries before the review is carried out. It reduces duplicated effort and lets readers compare the finished review with its plan, exposing changes made after the results were known.

Proximity operators

Also called: proximity operator, adjacency operators, adjacency searching, proximity searching, NEAR operator, ADJ operator

Proximity operators are search commands, such as NEAR, ADJ or N3 depending on the database, that find two terms within a set number of words of each other, in any order or a fixed one. They are more sensitive than phrase searching and more precise than AND, and the Cochrane Handbook recommends them for linking related free-text terms.

Publication bias

Also called: file drawer problem, file-drawer problem, file drawer effect

Publication bias is the distortion that arises when whether a study is published depends on its results, so that studies with positive or significant findings are more likely to appear than those with null or negative ones. It makes the published literature, and any review limited to it, look more favourable than the full evidence, and Rosenthal called it the file drawer problem.

QUADAS-2

Also called: QUADAS 2, Quality Assessment of Diagnostic Accuracy Studies 2

QUADAS-2 is a tool, published in 2011, for assessing risk of bias and applicability in primary studies of diagnostic test accuracy included in systematic reviews. Its four domains, patient selection, index test, reference standard, and flow and timing, are each judged for risk of bias, the first three also for concerns about applicability, with signalling questions to guide the judgements.

QUADAS-3

Also called: QUADAS 3, Quality Assessment of Diagnostic Accuracy Studies 3

QUADAS-3 is the revised tool for assessing risk of bias and applicability in diagnostic test accuracy studies, published in 2026 as the successor to QUADAS-2. It judges each accuracy estimate rather than each study against an ideal test accuracy trial, across four domains, participants, index test, target condition and analysis, and adds an overall judgement.

Qualitative evidence synthesis

Also called: QES, qualitative systematic review, systematic review of qualitative studies, qualitative research synthesis

A qualitative evidence synthesis is a systematic review that brings together findings from qualitative studies, such as interview and focus group research, to understand people's experiences and views and the context and implementation of interventions. It uses methods such as thematic synthesis, framework synthesis or meta-ethnography, and GRADE-CERQual can rate confidence in each of its findings.

Quality score

Also called: quality scale, summary quality score, quality scoring

A quality score is a single number that adds up a study's ratings on a checklist or scale, meant to summarise its methodological quality. Cochrane, the authors of AMSTAR 2 and the authors of the MMAT all advise against it, because a total mixes bias with reporting and precision, is hard to interpret and can hide the one flaw that matters most.

Question framework

Also called: research question framework, review question framework, question formulation framework

A question framework is a mnemonic that breaks a review question into its key elements, such as PICO for interventions, PCC for scoping reviews or SPIDER for qualitative research. It helps make the question precise, set the eligibility criteria and structure the search, though not every question fits a framework and none is compulsory.

RAMESES

Also called: RAMESES publication standards, Realist And MEta-narrative Evidence Syntheses: Evolving Standards, RAMESES standards

RAMESES, Realist And MEta-narrative Evidence Syntheses: Evolving Standards, is the project that published reporting standards in 2013 for realist syntheses, with 19 items, and for meta-narrative reviews, with 20 items. The standards were agreed by an international Delphi panel and ask authors to explain and justify any changes to the methods as first described.

Random-effects model (meta-analysis)

Also called: random-effects meta-analysis, random effects meta-analysis, random effects model

A random-effects model is a meta-analysis model that assumes the studies estimate different but related true effects, following a distribution, so that its pooled result is the average effect rather than one common effect. It adds the between-study variance to each study's own variance, which gives smaller studies relatively more weight and widens the confidence interval when heterogeneity exists.

Rapid review

Also called: rapid evidence assessment, REA, rapid evidence synthesis, rapid systematic review

A rapid review is an evidence synthesis produced in less time than a full systematic review, by simplifying or skipping some of its steps, so that decision-makers get evidence when they need it. Cochrane's rapid reviews guidance of 2021 allows shortcuts such as a single screener with a second checking the exclusions, and asks that changes to the protocol be documented.

Rating down (GRADE)

Also called: downgrading, rating down, downgrading the evidence

Rating down, in GRADE, is lowering the certainty of a body of evidence for serious concerns in any of five domains: risk of bias, inconsistency, indirectness, imprecision and publication bias. Evidence from randomised trials starts at high certainty and can be rated down to moderate, low or very low, with the reason for each step given in a footnote.

Rating up (GRADE)

Also called: upgrading, rating up, upgrading the evidence

Rating up, in GRADE, is raising the certainty of a body of evidence, mainly from non-randomised studies, for one of three reasons: a large effect, a dose-response gradient, or plausible confounding that would have reduced the apparent effect rather than inflated it. It lets strong observational evidence start low and still reach moderate or high certainty.

Realist review

Also called: realist synthesis, realist systematic review

A realist review is a theory-driven review of complex interventions, first described by Pawson, that asks what works, for whom, in what circumstances and how, rather than how well something works on average. It makes the programme theory explicit, uses varied evidence to test and refine it as context-mechanism-outcome configurations, and is reported to the RAMESES standards.

Reciprocal translation

Also called: reciprocal translational analysis, reciprocal synthesis

Reciprocal translation is the step in meta-ethnography in which concepts from different qualitative studies that describe similar things are translated into one another, recognising the same idea even when the studies use different words. It aims to keep each concept's context and meaning, and it sits beside refutational translation, which handles concepts that contradict one another.

Record, report and study (PRISMA)

Also called: records, reports and studies, study versus report, record versus report

In PRISMA 2020, a record is the title or abstract of a report indexed in a database, a report is any document, such as an article, preprint or register entry, that gives information about a study, and a study is the investigation itself. Reviews count records and reports during selection but include studies, since one study may have several reports.

Refutational translation

Also called: refutational synthesis, refutational analysis

Refutational translation is the step in meta-ethnography that examines concepts or whole studies which contradict one another, in order to explain the differences, exceptions and inconsistencies rather than smooth them over. eMERGe notes that it is often overlooked, and a single meta-ethnography can include both reciprocal and refutational translation.

Relevance (CERQual)

Also called: relevance (GRADE-CERQual)

Relevance is the GRADE-CERQual component that judges how far the body of evidence behind a qualitative review finding applies to the context in the review question, such as its population, phenomenon of interest and setting. Evidence gathered in a different setting or group from the one asked about warrants less confidence in the finding.

Review protocol

Also called: systematic review protocol, protocol (systematic review)

A review protocol is the document, written before a systematic review begins, that sets out its question, eligibility criteria, search, selection, appraisal and synthesis methods, including any planned subgroup analyses. Writing it without knowledge of the studies limits the influence of the authors' biases, and PRISMA-P lists what it should contain.

Risk of bias assessment

Also called: risk-of-bias assessment, RoB assessment, assessment of risk of bias, risk of bias appraisal

A risk of bias assessment is the judgement, for each included study or result, of how likely its design or conduct is to have distorted its estimate of effect, made domain by domain with a tool such as RoB 2 or ROBINS-I. Cochrane prefers it to rating general quality, because it targets systematic error rather than reporting or precision.

RoB 1

Also called: original Cochrane risk of bias tool, Cochrane risk-of-bias tool 2011, RoB 1.0

RoB 1 is the earlier Cochrane tool for assessing risk of bias in randomised trials, in its 2011 form, which judged each study as at low, high or unclear risk in domains such as random sequence generation, allocation concealment, blinding, incomplete outcome data and selective reporting. The Cochrane Handbook now recommends RoB 2, which assesses results rather than whole studies.

RoB 2

Also called: RoB 2.0, RoB2, Risk of Bias 2, Cochrane RoB 2, revised Cochrane risk-of-bias tool

RoB 2 is Cochrane's current tool for assessing risk of bias in a specific result of a randomised trial, with five domains: the randomisation process, deviations from intended interventions, missing outcome data, measurement of the outcome and selection of the reported result. Signalling questions lead through an algorithm to low risk, some concerns or high risk for each domain and overall.

ROB-ME

Also called: ROB-ME tool, Risk Of Bias due to Missing Evidence

ROB-ME is a tool for assessing the risk that a meta-analysis is biased because whole studies or particular results are missing owing to their P values, size or direction. The Cochrane Handbook describes it as the first structured approach to this problem, and it leads to a judgement of low risk, some concerns or high risk.

ROBINS-E

Also called: ROBINS-E tool, Risk Of Bias In Non-randomized Studies - of Exposure

ROBINS-E is a tool for assessing risk of bias in a result from an observational study of the effect of an exposure, such as an environmental, occupational or behavioural exposure, on health. Designed mainly for use in systematic reviews, it covers seven domains and asks assessors to judge both the risk of bias and its likely direction.

ROBINS-I

Also called: ROBINS-I tool, ROBINS-I V2, Risk Of Bias In Non-randomised Studies of Interventions

ROBINS-I is the Cochrane-recommended tool for assessing risk of bias in a result from a non-randomised study of an intervention, by comparing the study with a hypothetical target trial that would answer the same question without bias. The original 2016 version rates risk as low, moderate, serious or critical across seven domains, beginning with confounding, and a second version adds algorithms.

ROBIS

Also called: ROBIS tool, Risk Of Bias In Systematic reviews

ROBIS is a tool, published in 2016, for assessing the risk of bias in a systematic review itself rather than in its primary studies, aimed at authors of overviews, guideline developers and review authors. It works in three phases: an optional check of relevance, concerns in four domains of the review process, and an overall judgement of risk of bias.

Scoping review

Also called: scoping study, scoping literature review, ScR

A scoping review is a review, done with systematic methods, that maps the extent, range and nature of the evidence on a broad topic to identify key concepts, kinds of evidence and gaps, rather than answering a narrow question about effects. It usually neither assesses risk of bias nor pools results, builds on Arksey and O'Malley's 2005 framework, and is reported with PRISMA-ScR.

Scoping search

Also called: preliminary search, exploratory search, pilot search, scoping searches

A scoping search is a quick preliminary search, done before a review is planned in detail, to gauge how much literature exists, check for existing or ongoing reviews and refine the question and scope. It is exploratory, unlike the full documented search run for the review itself, and it is not the same as a scoping review, which is a whole type of review.

Search filter

Also called: search hedge, methodological search filter, methodological filter, study design filter

A search filter is a pre-tested block of search terms added to a strategy to retrieve a particular kind of record, such as studies of one design, as with the Cochrane Highly Sensitive Search Strategy for randomised trials in MEDLINE. Filters differ from a database's built-in limits, and Cochrane advises against trial filters in CENTRAL, which is already a register of trials.

Search limits

Also called: database limits, limiting commands, search restrictions

Search limits are a database's built-in options for narrowing results, for example by date, language, publication type or human studies. Review guidance asks authors to justify any restriction, and warns that language limits in particular can introduce bias and are better applied as an eligibility criterion at screening.

Search strategy

Also called: search strategies, literature search strategy, systematic search strategy, search plan

A search strategy is the full, planned set of searches for a review: the sources, the concepts, the subject headings and free-text terms for each, the operators and limits that combine them, and the dates run. For a systematic review it is written in advance, peer reviewed, adapted for each database and reported in full so that others can repeat it.

Search string

Also called: search query, search statement, query string

A search string is a single line of search terms and operators entered into a database, such as (exercise OR "physical activity") AND depress*. A search strategy is usually built from several numbered strings, one for each concept, which are then combined, making each part easier to check and adapt.

How to find sources for a literature review fast

Seed article

Also called: seed articles, benchmark study, benchmark articles, known relevant studies

A seed article is a study already known to meet a review's eligibility criteria, used to harvest search terms and then to test whether the finished search strategy retrieves it. Seed articles, also called benchmark studies, may come from earlier reviews, experts or exploratory searching, and a strategy that misses them needs revising.

Selection model (publication bias)

Also called: selection models, weight-function model

A selection model is a statistical method for publication bias that models the probability that a result was published as a function of its P value, size or direction, and estimates the effect after allowing for the results presumed missing. The Cochrane Handbook cautions that such models assume non-reporting is the only cause of small-study effects.

Sensitivity (of a search)

Also called: search sensitivity, sensitivity of a search, search recall, recall (searching)

The sensitivity of a search is the proportion of all the relevant reports in a source that the search actually retrieves, also called recall. Systematic review searches aim for high sensitivity with reasonable precision, because missing eligible studies is worse than screening extra records, and known relevant seed articles can be used to test it.

Sensitivity analysis (meta-analysis)

Also called: sensitivity analysis (systematic review)

A sensitivity analysis in a systematic review repeats the analysis with a different but defensible choice, such as excluding studies at high risk of bias, using another effect measure or model, or changing an eligibility criterion, to see whether the conclusions hold. Unlike a subgroup analysis, it tests robustness rather than whether the effect differs between groups.

Signalling question

Also called: signalling questions, signaling question, signaling questions

A signalling question is one of the factual questions in a risk-of-bias tool such as RoB 2, ROBINS-I or QUADAS that prompts the assessor to look for a specific feature of a study, answered in RoB 2 as yes, probably yes, probably no, no or no information. The answers feed, often through an algorithm, into the judgement for each domain.

Small-study effects

Also called: small study effects, small-study effect, small-study bias

Small-study effects are the tendency in some meta-analyses for smaller studies to show larger effects than bigger ones, seen as asymmetry in a funnel plot. The Cochrane Handbook lists several possible causes, including non-reporting bias, poorer methods in small studies, genuine differences between populations, artefacts of the effect measure and chance, so asymmetry alone does not prove publication bias.

SPICE

Also called: SPICE framework, Setting, Perspective, Intervention, Comparison, Evaluation

SPICE is a question framework, proposed by Booth in 2006, standing for Setting, Perspective, Intervention or phenomenon of interest, Comparison and Evaluation. It suits qualitative, service and policy questions in the social sciences and health services, where the setting and whose viewpoint counts matter as much as the intervention.

SPIDER

Also called: SPIDER tool, SPIDER framework, Sample, Phenomenon of Interest, Design, Evaluation, Research type

SPIDER is a question and search framework for qualitative and mixed methods reviews, standing for Sample, Phenomenon of Interest, Design, Evaluation and Research type, published by Cooke, Smith and Booth in 2012. It replaces PICO's population and intervention with a sample and a phenomenon, though its authors said it needed testing on a wider range of topics.

State-of-the-art review

Also called: state of the art review, state-of-the-art literature review

A state-of-the-art review is a review that concentrates on the most current research on a topic, rather than its history, to describe where the field stands now. In Grant and Booth's typology it may offer new perspectives on an issue or point out areas for further research, which distinguishes it from reviews that combine retrospective and current coverage.

Study selection

Also called: screening (systematic review), study screening, selection of studies

Study selection is the process of deciding which records found by the search are eligible for a systematic review, usually in two stages, screening titles and abstracts and then reading full texts against the eligibility criteria. It is ideally done by two reviewers independently, recorded in a PRISMA flow diagram, and often managed in software such as Covidence or EPPI-Reviewer.

Study weight

Also called: weighting (meta-analysis), weights in meta-analysis, meta-analytic weight, percentage weight

A study weight is the share of influence each study has on the pooled result of a meta-analysis, usually based on the inverse of its variance so that larger, more precise studies count for more, and it is shown as a percentage on a forest plot. Random-effects models spread the weights more evenly across studies than fixed-effect models do.

Subgroup analysis (meta-analysis)

Also called: subgroup meta-analysis, subgroup analysis (systematic review)

A subgroup analysis in a meta-analysis splits the studies, or their participants, into groups by a characteristic such as age, dose or setting and compares the pooled effects to see whether the effect differs. The Cochrane Handbook advises specifying subgroups in the protocol and treating unplanned ones as generating hypotheses only, since many comparisons produce spurious differences.

Subject heading

Also called: subject headings, index term, descriptor, thesaurus term, subject term

A subject heading is a single preferred term from a database's controlled vocabulary, assigned by indexers to records about that subject, such as the MeSH heading Depressive Disorder. Headings sit in a hierarchy of broader and narrower terms, can be exploded to include the narrower ones, and differ from one database to another.

SUCRA

Also called: surface under the cumulative ranking curve, SUCRA value, SUCRA score

SUCRA, the surface under the cumulative ranking curve, is a single number from 0 to 100 per cent that summarises how highly an intervention ranks across all possible positions in a network meta-analysis, 100 meaning it is always best. The Cochrane Handbook warns that rankings mislead unless they are read with the effect estimates and their uncertainty.

Summary of findings table

Also called: SoF table, summary of findings, GRADE summary of findings table

A summary of findings table is a table in a systematic review that gives, for up to seven important outcomes, the numbers of participants and studies, the relative and absolute effects and the GRADE certainty of the evidence, with footnotes explaining each rating. Cochrane reviews use it to present the main results to decision-makers in one place.

SWiM

Also called: Synthesis Without Meta-analysis, SWiM guideline, SWiM reporting guideline, synthesis without meta-analysis guideline

SWiM, Synthesis Without Meta-analysis, is a nine-item reporting guideline, published in 2020, for systematic reviews that synthesise quantitative effects without a meta-analysis. It asks how studies were grouped, which standardised metric and synthesis method were used, how heterogeneity and certainty were assessed and how data were presented, and it deliberately avoids the vague label narrative synthesis.

Synthesis matrix

Also called: literature review matrix, literature matrix, review matrix, source matrix, synthesis grid

A synthesis matrix is a table used when writing a literature review, with the sources along one axis and the themes or questions along the other, so that each cell records what a source says on a theme. Reading across and down shows where sources agree, differ or are silent, which turns summaries of single papers into synthesis.

Thesis literature review, keeping hundreds of sources straight

Systematic review

Also called: systematic literature review, SLR, systematic reviews, systematic review of the literature

A systematic review is a review that answers a specific question by finding, appraising and synthesising all the studies that meet eligibility criteria set in advance, using explicit and reproducible methods designed to limit bias. It is planned in a protocol, involves a comprehensive search, screening, appraisal and synthesis with or without meta-analysis, and is reported in full, for example to PRISMA 2020.

Systematised review

Also called: systematized review, systematic-style review

A systematised review is a review that uses some elements of systematic review methods, such as a structured search, without meeting all of them, for example because one person does the screening or there is no formal appraisal. Grant and Booth described it as typically a postgraduate student assignment, done where the resources for a full systematic review are not available.

Tau squared

Also called: τ², tau², tau2, between-study variance

Tau squared is the estimated variance of the true effects across studies in a random-effects meta-analysis, a measure of how much real effects differ between settings. Unlike I squared it does not depend on the number or precision of the studies, and its square root, tau, is on the scale of the effect and underlies the prediction interval.

Thematic synthesis

Also called: thematic synthesis (Thomas and Harden)

Thematic synthesis is a method of qualitative evidence synthesis, described by Thomas and Harden in 2008, with three stages: coding the primary studies' findings line by line, grouping those codes into descriptive themes, and generating analytical themes that go beyond the studies to answer the review question. It keeps an explicit link between conclusions and the primary texts.

Third-order constructs

Also called: third order constructs, third-order interpretations, first-, second- and third-order constructs

Third-order constructs are the reviewers' own interpretations in a meta-ethnography or similar synthesis, built from second-order constructs, the primary authors' interpretations, which rest in turn on first-order constructs, the participants' own accounts. Keeping the three levels distinct shows readers whose interpretation each finding represents.

Time-lag bias

Also called: time lag bias, lag time bias

Time-lag bias is the distortion that arises when studies with positive or significant results are published sooner than those with null or negative results, so that an early review sees a more favourable picture than the evidence will eventually show. Updating reviews and searching trials registers for completed but unpublished studies reduce it.

Title and abstract screening

Also called: title/abstract screening, title-abstract screening, abstract screening, first-stage screening, TiAb screening

Title and abstract screening is the first stage of study selection in a systematic review, in which reviewers read the title and abstract of every record found and exclude those clearly ineligible, passing the rest to full-text screening. The Cochrane Handbook notes that 60 to 120 abstracts can be screened an hour, which is why sensitive searches remain practical.

Transitivity

Also called: transitivity assumption, similarity assumption

Transitivity is the assumption behind indirect comparisons and network meta-analysis that the sets of trials making different comparisons are similar, on average, in everything that could modify the effect apart from the interventions compared. In practice it means participants could in principle have been randomised to any of the interventions, and it is checked by comparing effect modifiers across comparisons.

Trim and fill

Also called: trim-and-fill, trim and fill method, Duval and Tweedie trim and fill

Trim and fill is a funnel-plot-based method, published by Duval and Tweedie in 2000, that estimates how many studies are missing from one side of an asymmetric plot, imputes their mirror images and recalculates the pooled effect. It assumes the asymmetry is caused by publication bias and performs poorly when heterogeneity is high, so its adjusted estimate should be read cautiously.

Truncation

Also called: truncation symbol, stemming, right truncation

Truncation is the use of a symbol, usually an asterisk, at the end of a word stem to retrieve every ending in one search, so that child* finds child, children and childhood. The symbol differs between databases, and a stem cut too short retrieves unrelated words, so each truncated term is worth checking.

Umbrella review

Also called: overview of reviews, overview of systematic reviews, review of reviews, reviews of reviews, meta-review, Cochrane overview

An umbrella review is a review that gathers and compares the findings of existing systematic reviews and meta-analyses on a broad question, rather than of primary studies. Cochrane calls it an overview of reviews, and its main hazards are overlapping reviews that count the same studies twice and reviews of uneven quality, appraised with tools such as AMSTAR 2 or ROBIS.

Unit-of-analysis error

Also called: unit of analysis error, unit-of-analysis issue, unit of analysis problem

A unit-of-analysis error, in a meta-analysis, is an analysis in which the number of observations does not match the units that were randomised, for example a cluster-randomised trial analysed as if individuals were randomised, or one control group counted twice against two intervention arms. It gives wrongly narrow or wide confidence intervals and distorts the study's weight.

Vote counting

Also called: vote-counting, vote counting based on statistical significance, vote counting based on direction of effect

Vote counting is a way of summarising studies by tallying how many found a positive, negative or no effect. Counting by statistical significance is discouraged, because small underpowered studies are scored as showing no effect and the tally can point to the wrong answer, whereas Cochrane accepts counting by direction of effect, tested with a sign test, when better methods are impossible.

Wildcard

Also called: wildcards, wild card, wildcard symbol, wildcard character

A wildcard is a symbol, often a question mark, that stands for one or more characters inside a word in a database search, so that wom?n finds woman and women and colo?r finds color and colour. It captures spelling variants, including British and American forms, and the symbols vary from one database to another.