Week 6 - Assignment: Analyze Qualitative Data AND Week 7 - Assignment: Signature Assignment: Design a Qualitative Study

Journal of Clinical Epidemiology 129 (2021) 74e85

REVIEW

Defining Rapid Reviews: a systematic scoping review and thematic analysis of definitions and defining characteristics of rapid reviews

Candyce Hamela,b,*, Alan Michauda, Micere Thukua, Becky Skidmorea, Adrienne Stevensa, Barbara Nussbaumer-Streitc, Chantelle Garrittya,b

aOttawa Hospital Research Institute, Knowledge Synthesis Group, Ottawa, ON K1H 8L6, Canada bUniversity of Split, School of Medicine, Split, Croatia 21000

cCochrane Austria, Department for Evidence-based Medicine and Evaluation, Danube University Krems, Krems, Austria

Accepted 29 September 2020; Published online 8 October 2020

Abstract

Background and Objective: Rapid reviews were first mentioned in the literature in 1997, when Best et al. described the rapid health technology assessment program in the south and west regions of England but did not provide a formal definition. More recently, the only consensus around a rapid review definition is that a formal definition does not exist. The primary aim of this work is to create a repository of existing definitions and to identify key themes, which may help the knowledge synthesis community in defining rapid review products.

Methods: A systematic scoping review was performed to identify definitions used in journal-published rapid reviews written in English between 2017 and January 2019. We searched Medline, Embase Classic þ Embase, PsycINFO, ERIC, Cochrane Library, CINAHL, and Web of Science on December 21, 2018. Two reviewers performed study selection and data extraction using a prioriedefined methods pub- lished in a protocol. Definitions from rapid review methods articles (published from 1997 onward) identified in another scoping review were added to the results, and all definitions were thematically analyzed using NVivo. A quantitative analysis was also performed around studies cited.

Results: Definitions from 216 rapid reviews and 90 rapid review methods articles were included in the thematic analysis. Eight key themes were identified: accelerated/rapid process or approach, variation in methods shortcuts, focus/depth/breadth of scope, compare and contrast to a full traditional systematic review, stakeholder rationale, resource efficiency rationale, systematic approach, bias/limita- tions. Secondary referencing was a common occurrence.

Conclusion: Thematic analysis performed in this systematic scoping review has allowed for the creation of a suggested definition for rapid reviews that can be used to inform the systematic review community. � 2020 Elsevier Inc. All rights reserved.

Keywords:: Scoping review; Rapid reviews; Definition; Thematic analysis

1. Introduction

A rapid review (RR) was originally mentioned in the literature in 1997, when Best et al. described the rapid health technology assessment program in the south and

Funding: This work is supported in part by funds provided by Cochrane

and in part from a Canadian Institutes of Health Research grant (funding

research number 142310). The funders had no role in the development

of this manuscript and its protocol (registered on OSF: https://os-

f.io/y5f2m/). They did not have a role in the data collection, analyses,

interpretation of the data, and publication of the findings.

Declaration of interest: None.

* Corresponding author. Ottawa Hospital Research Institute, 501

Smyth Road, Ottawa, Ontario, Canada K1H 8L6.

E-mail address: cahamel@ohri.ca (C. Hamel).

https://doi.org/10.1016/j.jclinepi.2020.09.041

0895-4356/� 2020 Elsevier Inc. All rights reserved.

west regions of England [1]. Although they did not provide a definition of an RR, they described a service which pro- duces reports within two person months. The key features of the service were to produce reports that were accurate, timely, and accessible to decision makers. More recently, the only consensus around an RR definition is that a formal definition does not exist [2e4]. Several definitions have been used in publications about RR methods, RR programs, and RRs themselves. In 2016, Kelly et al. performed a modified Delphi consensus approach and came up with a set of statements defining the characteristics of an RR [4], but did not provide a formal definition or a systematic evaluation of existing definitions.

The popularity of RRs has been increasing over the past 20 years, with various organizations developing RRs, including the World Health Organization (WHO) [5], the

75ical Epidemiology 129 (2021) 74e85

What is new?

Key findings � A repository of definitions from 216 rapid reviews

and 90 rapid review methods articles was created (158 rapid reviews and 73 rapid review methods ar- ticles provided a definition).

� Among the rapid reviews, 59 unique references were cited 275 times. The top four cited authors were referenced 135 times. Among rapid review methods articles, 50 unique references were cited 179 times.

� A thematic analysis identified eight key themes in defining rapid reviews.

� Secondary referencing was common among cited articles.

What this adds to what was known? � There is currently no consensus on what defines a

rapid review.

� The four most commonly reported themes (used in ~�50% of definitions) were used to create a pre- liminary definition of a rapid review. Suggestions are included on how users might tailor this defini- tion to best meet their individual remit and man- dates for producing rapid reviews.

What is the implication and what should change now? � The preliminary definition, with caveats, presented

can help the systematic review community define their review with consistency, regardless of the la- bel used to describe it.

Samueli Institute’s Rapid Evidence Assessment of the Literature (REAL�) program [6], and the Canadian Agency for Drugs and Technologies in Health Rapid Response Service [7]. The number of RRs published in the last 5 years has steadily increased. In 2013, 15 journal-published RRs were identified, growing to 52 by 2016, and 108 in 2018. Although these numbers are small, most RRs are not published in journals. For example, in 2016, 52 published RRs and over 250 unpublished RRs were identified from various health care organizations. In 2017, the Knowledge Synthesis Group at the Ottawa Hos- pital Research Institute identified 148 organizations glob- ally who produced RRs [RR workshop presentation, November 27, 2019 Ottawa, Canada, with data derived from internal projects].

Some of the problems with lacking a common definition for RRs are that it makes it difficult

C. Hamel et al. / Journal of Clin

(i) for researchers (e.g., building search strategies that accu- rately identify RR) and readers/users of results to iden- tify RRs correctly. This is important as the line may be blurred (both in the conduct and the resulting conclu- sions) between systematic reviews (SRs) that do not meet a high-quality methodological conduct (e.g., low or critical risk using AMSTAR 2) and RRs that use transparent, measured abbreviated methods; (ii) to create and set methodological standards and apply consistent constructs (e.g., Preferred Items in Systematic Reviews and Meta-Analysis [PRISMA] for RRs, AM- STAR for RRs); and (iii) as it results in a heterogeneous set of products under the same name or conversely a homogeneous set of products under different names. The term ‘rapid’ points toward the speed at which the review is performed and not the abbreviation or omission of steps taken to conduct the review. For this reason, researchers have suggested other terms be used, for example, restricted reviews [8,9]. To date, ‘rapid review’ is the term that has been colloquially adopted by the research community and endorsed by various organizations, including Cochrane and the WHO. However, other organizations have chosen other terms, such as rapid evidence assessment by the UK government.

2. Objective

The objective of this systematic scoping review was to identify published RR literature to answer the question: How are RRs defined in the literature? This work will pro- vide a summary of existing definitions identified in the literature on RRs and examine existing definitions to iden- tify common themes across the body of literature. Creating a repository of existing definitions and developing a prelim- inary definition, while allowing for caveats and flexibility depending on the organizational preferences or mandate, is an important step in helping the knowledge synthesis community conduct and identify RRs. In addition, as Co- chrane considers RRs an important piece in their content strategy, this makes this topic very important as these re- sults will inform discussions within Cochrane on the utility of RRs as a product.

3. Methods

This systematic scoping review was guided by estab- lished scoping review methodology [10,11] and has been prepared in accordance with the PRISMA extension for Scoping Reviews (PRISMA-ScR) [12]. A protocol for this work was registered on the Open Science Framework

76 C. Hamel et al. / Journal of Clinical Epidemiology 129 (2021) 74e85

(OSF: https://osf.io/y5f2m/). Methods are briefly described in Table 1, with additional details and deviations from the protocol in Appendix A.

4. Results

The search strategies to identify RRs resulted in 2,657 unique records, of which 422 were evaluated at full text, with 216 RRs included (Figure 1). Several records were excluded at title/abstract as they did not explicitly state the term rapid or a derivative. For feasibility, only those with rapid, expedited, or abbreviated were considered for inclusion, while excluding those that described the review as focused (n 5 347), targeted (n 5 54), or pragmatic (n 5 28). In addition, several other terms which may be considered ‘rapid’ derivatives were identified; however, because of the number of these records and our focus on those who self-declared as ‘rapid’, they were also excluded (n 5 127) (Appendix C).

Among the 216 RRs, 101 were published in 2017, 106 were published in 2018, and nine were published in January of 2019 (Table 2). Most of the RRs (82.5%) were from cor- responding authors from the United Kingdom (n 5 82), Australia (n 5 41), the United States of America (n 5 31), and Canada (n 5 24). Almost two-thirds (63.0%) used the term RR, with others using the terms rapid evidence assessment (10.1%) and rapid systematic re- view (8.8%). Nearly two-thirds (141 of 216; 65.3%) first

Table 1. Methods in brief

Project stage Me

Eligibility criteria - Published rapid reviews using ‘rapid’ or derivative (e.g

- Published between January 2017 and January 2019

- Written in English (for feasibility)

Searching for studies

- Developed by an experienced information specialist wi

- Peer-reviewed using the PRESS checklist [13]

- Search (Dec 2018): MEDLINE� ALL, Embase Classic Science (Appendix B)

- Search strategies not restricted by language

- Supplemented with definitions from rapid review meth

Study selection - Performed in DistillerSR [15]

- Piloted title/abstract (n 5 100) and full-text screening

- Liberal accelerated58 screening for titles and abstracts

- Dual-independent screening based on full text, with co

Data charting - Performed in DistillerSR [15]

- Piloted extractions (n 5 5), conflicts resolved through

- One reviewer extracted studies, a second reviewer verifi

Data synthesis - Rapid review characteristics and studies’ references ex

- Definitions imported into NVivo (version 12) for coding

used the term in the title, with the remaining first using the term in the abstract (Appendix D).

4.1. Definitions from published RRs

In total, 158 (73%) RRs provided a definition. Fifteen provided their own (i.e., 11 providing only their own defi- nition and four referencing their own in addition to other authors), and one provided a definition, but the references in the publication did not line up and therefore no refer- ences were recorded [16]. Some RR authors did not provide an explicit definition, but made reference to another author or method (e.g., ‘‘We conducted a review of the literature using the rapid evidence assessment (REA) method [17,18].’’) [19]. Among the 146 RRs that provided a defini- tion citing another author, 59 unique references were cited a total of 275 times (Appendix E.1). Among all RRs, a me- dian of two references (range 0 to 7) were cited. Further- more, 29 articles were cited once. The top four articles cited were Khangura 2012 (n 5 54) [2], Ganann 2010 (n 5 42) [20], Tricco 2015 (n 5 21) [3], and Grant 2009 (n 5 18) [18] (Table 3).

4.2. Definitions from RR methods articles

In total, 81% (73 of 90) of the RR methods articles pro- vided a definition. These definitions were included in the thematic analysis to supplement the definitions identified in the RRs. Briefly, methods articles were published be- tween 1997 and 2019, with the majority of the articles pub- lished since 2014 (68 of 90 (75.6%)). A total of 200

thod description

., abbreviated) in the title or abstract

th input on search terms by members of the research team

þ Embase, PsycINFO, ERIC, Cochrane Library, CINAHL, Web of

ods scoping review [14]

(n 5 25), conflicts resolved through discussion

nflicts resolved through discussion

discussion

ed all extracted data, conflicts resolved through discussion

ported to MS Excel 2016

into themes

In cl ud ed

Id en tif ic at io n

Sc re en in g

El ig ib ilit y

Records iden fied through database searching N = 3672

Addi onal records iden fied through grey literature searching N = 0

Addi onal records iden fied from methods scoping review N = 90

Records a er duplicates removed N = 2657

Records screened N = 2657

Records excluded N = 2235 -------------------------------------------------------- Record does not discuss rapid review (or its variants) (n = 1679) Pragma c review (n = 28) Targeted review (n = 54) Focused review (n = 347) Other (to exclude) ( = 127)

Full-text ar cles assessed for eligibility N = 422

Full-text ar cles excluded, with reasons N = 206 ------------------------------------------------------------------------ Full-text not available (n=2) Not published in English (n = 11) Other (i.e., animal studies, full conference proceeding, irretrievable due to record error) (n=6) Review not published in 2017, 2018 or January 2019 (n = 4) Authors did not use the term rapid, accelerated, expedited or a variant in the tle, abstract, or methods sec on to describe the review process (n = 183)

Studies included in synthesis N = 216 RRs N = 90 Methods papers

Fig. 1. PRISMA flow diagram.

77C. Hamel et al. / Journal of Clinical Epidemiology 129 (2021) 74e85

definitions were cited, with 21 articles providing their own definition (with or without a reference to other articles) and 50 unique articles. Among the 21 articles that provided their own definition, 10 are those that are often referenced in the RRs [18,20e28]. Methods articles referenced an average of 2.22 references (median: 1, range: 0 to 10) (Appendix E.2). The top four articles referenced were Ga- nann 2010 [20] (n 5 27), Khangura 2012 [2] (n 5 27), Khangura 2014 [29] (n 5 14), and Polisena 2015 [30] (n 5 11). The other top articles in the RRs not in the top four here, Tricco 2015 [3] and Grant 2009 [18], were refer- enced 10 and four times, respectively.

There was overlap between the articles cited in the RRs and methods articles. Across both data sources, there were

79 unique citations, with 30 citations included in both scoping reviews, 29 unique to the RRs, and 20 unique to the methods articles. Among the citations found in only one of the two data sources, most were only referenced one or two times (highlighted in Appendix F).

4.3. Thematic analysis

All definitions from the RRs and the methods articles were thematically analyzed in NVivo. We identified eight major themes (Figure 2). Among the 204 articles that re- ported definitions (75 did not provide a definition and 27 RRs cited other studies with no identifiable themes), the most common themes were theme 4: Compare and contrast

Table 2. Rapid review characteristics

Rapid review characteristics Rapid reviews (N [ 216)

Year publisheda

2017 101 (46.7%)

2018 106 (49.1%)

2019 9 (4.2%)

Countries of the corresponding author

UK 82 (38.0%)

Australia 41 (19.0%)

USA 31 (14.4%)

Canada 24 (11.1%)

Ireland 6 (2.8%)

Italy 5 (2.3%)

Germany 4 (1.9%)

Denmark, South Africa, Switzerland 3 (1.4%) (each)

Finland, India, Spain 2 (0.9%) (each)

Japan, Korea, Nepal, Norway, Poland, Sweden, Taiwan, Thailand & UK 1 (0.5%) (each)

Terminology used (first mentioned in RR)

Rapid review 136 (63.0%)

Rapid evidence assessment 22 (10.1%)

Rapid systematic review 19 (8.8%)

Rapid evidence review, rapid literature review (each) 12 (5.6%)

Systematic rapid evidence assessment, systematic rapid review 2 (0.9%) (each)

Abbreviated review, rapid appraisal, rapid best-fit framework synthesis, rapid evidence-based review, rapid evidence summary, rapid evidence synthesis, rapid meta-review, rapid qualitative review, rapid response review, rapid structured evidence review, rapid synthesis

1 (0.5%) (each)

Terminology first mentioned in

Title 141 (65.3%)

Abstract 75 (34.7%)

References

Total 290

Unique references/citations 59

Median (Range) 2 (0 to 7)

Mean 1.34

Top references

Khangura 2012 54

Gannan 2010 42

Tricco 2015 21

Grant 2009 18

Number of references

0 59 (27.3%)

1 84 (38.9%)

2 35 (16.2%)

3 or more 38 (17.6%)

a Articles may be an Epub before print with the print date after January 2019. Years published are taken as of the search date (December 20, 2018).

78 C. Hamel et al. / Journal of Clinical Epidemiology 129 (2021) 74e85

to SRs (68.1%; 139 of 204), theme 2: Variation in shortcut methods (54.9%; 112 of 204), with theme 1: Accelerated/ rapid process and theme 6: Resource efficiency rationale tied (48.5%; 99 of 204 each) (Figure 3). Definitions often covered more than one of these themes, with a range of 1 to 8 (median: 3; mean: 3).

4.3.1. Theme 1: Accelerated or rapid process/approach The terms accelerated, streamlined, quickly or rapid

were used in terms of the speed or timing for the overall approach to completing the RR. For example, ‘‘rapid re- views have been described as a streamlined alternative to standard systematic reviews [31].’’ [32].

Table 3. e Top four study cited and their definitions

Study Definition Cites

Khangura 2012a ‘‘Given this lack of definition and evolving landscape, we have abstained from applying the label ‘rapid review’ to our KTA syntheses, and have alternatively called them ‘evidence summaries’. Despite this, we consider our evidence summaries to be part of the continuum of rapid reviews, as conceptualized by Ganann and colleagues’’.

Ganann 2010b

Ganann 2010b ‘‘Rapid reviews are literature reviews that use methods to accelerate or streamline traditional systematic review processes.’’

None

Tricco 2015c ‘‘.we used the following working definition, ‘a rapid review is a type of knowledge synthesis in which components of the systematic review process are simplified or omitted to produce information in a short period of time.’’’

Khangura 2012a

Grant 2009d ‘‘They aim to be rigorous and explicit in method and thus systematic but make concessions to the breadth or depth of the process by limiting particular aspects of the systematic review process.’’

Butler 2005e

a Khangura et al. Syst Rev. 2012; 1:10. b Ganann et al. 2010. Implement Sci. 2010; 5:56. c Tricco et al. 2015. BMC Med. 2015; 13:224. d Grant & Booth. 2009. Health Info Libr J. 2009; 26(2):91-108. e Link to Butler 2005 no longer active, update Burton 2007.

79C. Hamel et al. / Journal of Clinical Epidemiology 129 (2021) 74e85

4.3.2. Theme 2: Variation in methods shortcuts There were a variety of words used to describe the short-

cuts used in the methods, including streamlined, restricted, pragmatic, abbreviated, modifications, concessions, expe- dited, simplifying, constraints, truncated, modified or omitted steps, and limiting. The variety of words relate to the lack of a standardized approach in which steps these were applied, with some definitions providing examples on which steps of the review process these shortcuts would be applied. For example, ‘‘Major sources of streamlining can include narrowing the scope of the review questions; limiting literature search databases; the use of single (vs. dual) abstract and full-text screening; reducing the extent of data abstraction; omitting risk of bias/quality appraisal; and restricting the extent of the synthesis [33].’’ [34].

Fig. 2. Eight key theme

4.3.3. Theme 3: Focus/depth/breadth of scope Similar to the theme 2, this was more specific to the

topic, scope, or question being addressed in the RR rather than the methodology. For example, ‘‘Rapid review is an evidence synthesis methodology that applies a systematic approach to evidence identification and syntheses, but with a more limited scope than a systematic review.’’ [35].

4.3.4. Theme 4: Compare and contrast to a full tradi- tional systematic review

Definitions often included a comparison or related RRs to full SRs but provided an explanation in the text as to the difference in general between an SR and the RR. For example, ‘‘A rapid structured review differs from a

s in defining RRs.

99

112

35

139 72

99

50

19

Accelerated/ rapid process

Varia on in methods shortcuts

Focus/ breadth/ depth of scope

Compare and contrast to SR

Stakeholder ra onale

Resource efficiency ra onale

Systema c approach

Bias/limita on

Repor ng of key themes

Fig. 3. Frequency of reporting of key themes.

80 C. Hamel et al. / Journal of Clinical Epidemiology 129 (2021) 74e85

systematic review in relation to the extensiveness of the search and methods used to undertake the analysis [36].’’ [37].

4.3.5. Theme 5: Stakeholder rationale Many definitions referenced performing an RR to inform

policy practice or to meet the needs of stakeholders, including decision makers (e.g., health professionals) and consumers. For example, ‘‘Rapid reviews are an emerging type of knowledge synthesis which aims ‘to inform health-related policy decisions and discussions, especially when information needs are immediate’ [38].’’ [39].

4.3.6. Theme 6: Resource efficiency rationale Definitions often referred to RRs being performed

because of resource constraints, including cost, human re- sources, time, and expertise. The difference between completing a review in a timely way (theme 1) and completing a review in a limited time frame is around the requirement of completing the review, rather than at the speed (e.g., rapidly, timely). For example, ‘‘Rapid reviews use systematic review methods to search and critically appraise existing research within limited resource and time constraints [40].’’ [41].

4.3.7. Theme 7: Systematic approach Although RRs take shortcuts, several definitions stated

that they remain systematic, transparent, rigorous, repli- cable, explicit, robust, using scientific methods. For example, ‘‘‘Rapid reviews’ are knowledge synthesis in which components of the systematic review process are

simplified or omitted, to produce information in a timely manner, while retaining rigor in the selection and appraisal of studies [2,20,22].’’ [42].

4.3.8. Theme 8: Bias/limitations There was some discussion around the bias that may be

introduced due to shortcuts. Although there are few studies that formally evaluate RRs compared with full SRs, there is a potential for bias and limitations when using shortcuts. For example, ‘‘although potential biases related to stream- lining procedures must be acknowledged [2].’’ [43].

4.4. Suggested definition

As there is not one common set of methods shortcuts that can be taken when conducting an RR, there may not be one common definition for an RR. As such, we suggest the following broad definition, which meets a minimum set of requirements identified in the thematic analysis, which will also be used to seek further consensus from the system- atic review community.

‘‘A rapid review is a form of knowledge synthesis that accelerates the process of conducting a traditional system- atic review through streamlining or omitting a variety of methods to produce evidence in a resource-efficient manner.’’

This definition covers the most common themes (i.e., 1, 2, 4, and 6) that were identified in approximately 50% or more of the RRs and methods articles. By using broad words like resources, this definition captures the time element, as well as cost and human elements. Users could

81C. Hamel et al. / Journal of Clinical Epidemiology 129 (2021) 74e85

then tailor this definition accordingly to best meet their in- dividual remit and mandates for producing RRs by adding additional details covered in other themes. For example, if an organization produces RRs only when stakeholders make a request (theme 5), it can be modified to include this requirement.

‘‘A rapid review is a form of knowledge synthesis that accelerates the process of conducting a traditional system- atic review through streamlining or omitting a variety of methods to produce evidence for stakeholders in a resource-efficient manner.’’

Likewise, if the systematic aspect (theme 7) of RRs is important, the definition can be further modified.

‘‘A rapid review is a rigorous and transparent form of knowledge synthesis that accelerates the process of con- ducting a traditional systematic review through streamlin- ing or omitting a variety of methods to produce evidence for stakeholders in a resource-efficient manner.’’

4.5. Collaboration among RR definition references

It was common for RR definitions to use secondary referencing (i.e., quoting or paraphrasing from a source which is mentioned in another text) [44]. As this was not the primary objective of this scoping review, this is further discussed in Appendix G for the interested reader.

5. Discussion

To the best of our knowledge, this is the first systemat- ically developed repository of RR definitions and an anal- ysis of their major common themes. Eight key themes were identified, with the four most common themes being comparing and contrasting RRs to SRs, variation in shortcut methods, with accelerated/rapid process or approach, and resource efficiency rationale tied for third. As a criterion for inclusion was the use of the term rapid or derivative and the goal is to conduct the review rapidly regardless of which stages of conduct are abbreviated/ omitted, it is not surprising that one of the key themes was around the accelerated or rapid approach.

As previously mentioned, some of the problems of lack- ing a common definition are around the difficulty in identi- fying RRs correctly and in having a homogenous set of products under different names. Among the RRs included in this scoping review, 18 different terms were used (Table 1), with an additional 23 terms, that may be consid- ered derivatives, excluded (for feasibility) when screening titles/abstracts (Appendix C). Although the term ‘rapid re- view’ seems to be the generally adopted term, ‘rapid’ points to the speed of the process and not necessarily the methods in which this is achieved. Recently, the term ‘restricted re- view’ has been suggested to better capture the restrictions in the methods [8,9]; however, this does not relate to the speed of production. A common term for labeling these

products may not be feasible, as many organizations have already adopted different terms for the same types of prod- ucts. However, a definition with central tenets may help producers of these reviews to identify their research for easy identification, regardless of the term used to describe the review. The importance of defining (vs. labeling) is further supported in the Cochrane Handbook and MECIR, which state that study design labels may be ambiguous, and a focus on the defining features of the study is more important than the label [45,46].

It is better to rely on the original source of the informa- tion than to rely on the wording of another author who may impose their own interpretation or meaning [44]. Although 48 unique references were cited in the RRs, there is a high level of secondary referencing, as displayed in the collabo- rative map (Appendix G. Figure 1), many pointing to the same smaller set of studies. Therefore, in the context of developing a definition for RRs (and/or a minimum set or criteria/central tenets), the number of definitions used and cited may not be as extensive as what the results from this scoping review demonstrate. Using the suggested definition from this scoping review, and the key citations for addi- tional support, may help lessen the ‘noise’ of what has been used and help guide future research in this area.

When comparing the key themes identified in this scoping review to related research, we see there are some similarities. Kelly et al. (2016) identified seven defining characteristics of RRs through a Delphi process [4]. How- ever, there were some limitations to this process as only 1 reviewer selected the included studies and it is unclear how the initial survey was developed. In addition, the search was run in December 2014, and the progression in the amount of research evaluating RR methods, methodo- logical development and guidance, and an increase in the number of published RRs has grown since this time [14]. This initial work provides a solid foundation to which this methodologically robust scoping review builds on using a more contemporary sample. We were able to map six of these seven defining characteristics to the themes we iden- tified (Table 4). The only key theme not covered is theme 3 related to the focus/breadth or depth of the scope. The only defining characteristic of an RR, identified by Kelly et al., that could not be related to one of the key themes identified in this scoping review was that ‘‘rapid reviews have a pro- tocol describing objectives, scope, PICO, and approach’’, although this is more around the process of developing an RR and less around defining it. Furthermore, Hartling et al. [47] identified 36 rapid products from 20 organiza- tions and concluded that there is extensive variability in products labeled as RRs, but that the range of methods used in developing these products is driven by and supported by close and ongoing communication between the producers of the review and the end user, a concept captured by key themes 2 and 5.

To date, only one definition has emerged at the center [20]: ‘‘Literature reviews that use methods to accelerate

Table 4. Kelly defining characteristics compared with themes identified

Kelly et al. defining characteristics Key theme(s)

Rapid reviews are conducted in less time than a systematic review Theme 1: Accelerated/rapid process or approach

Theme 4: Compare and contrast to SR

Rapid reviews use a spectrum of approaches to complete an evidence synthesis related to a defined research question(s) using the most systematic or rigorous methods as a limited time frame allows

Theme 2: Variation in methods shortcuts

Theme 6: Resource efficiency rationale

Theme 7: Systematic approach

Rapid reviews should have a protocol describing objectives, scope, PICO, and approach None

Rapid reviews should tailor the explicit, reproducible methods conventionally used in a systematic review in some manner to expedite the review process

Theme 1: Accelerated/rapid process or approach

Theme 2: Variation in methods shortcuts

Theme 4: Compare and contrast to SR

Theme 7: Systematic approach

Rapid reviews should transparently report methods and findings with a level of detail needed to adequately answer the research question, meet the requirements of the decision maker commissioning the review, and inform the audience for which the review is intended, while meeting a delivery time line agreed on in advance.

Theme 5: Stakeholder rationale

Theme 6: Resource efficiency rationale

Theme 7: Systematic approach

Rapid reviews should be considered in the context of the decision at hand when emergent or urgent decisions are required.

Theme 6: Resource efficiency rationale

Choices to adapt workflow should be balanced against the yet undetermined impact to conclusions or validity of findings, and this risk should be communicated to the end user.

Theme 8: Bias/limitations

82 C. Hamel et al. / Journal of Clinical Epidemiology 129 (2021) 74e85

or streamline traditional systematic review processes’’. However, this definition does not specifically address vari- ances in types of RRs produced across different contexts, which are likely driven by the mandate or scope of the or- ganization or entity producing them. In addition, when comparing this definition to the eight themes identified in the thematic analysis, it covers three of the eight key themes: accelerated/rapid process or approach (theme 1), variation in methods shortcuts (theme 2), and compare and contrast to traditional systematic reviews (theme 4). As this definition is from 2010, and RRs have been evolving over time, one might expect that it would not cover all key themes.

5.1. Implications for future research

Despite the increased use of RRs in policymaking [48,49], to date, there is no agreed-on definition on what constitutes a ‘rapid review’. Yet, other areas of knowledge synthesis have developed definitions (e.g., what represents a systematic review update, scoping reviews) [50,51] that have been agreed on by the broader knowledge synthesis community. Several other groups and programs have devel- oped their own definitions for RRs. For example, Crawford et al. 2015 describe the REAL� method, which ‘‘utilizes specific tools (e.g., automated online software) and stan- dard procedures (e.g., rulebooks) to rigorously deliver more reliable, transparent and objective SRs in a streamlined

fashion, without compromising quality and at a lower cost than other SR methods’’ [6]. The Department for Interna- tional Development within the UK government has their own program and state on their website that ‘‘Rapid evi- dence assessments provide a more structured and rigorous search and quality assessment of the evidence than a liter- ature review but are not as exhaustive as a systematic re- view’’. They can be used to ‘‘gain an overview of the density and quality of evidence on a particular issue, sup- port programming decisions by providing evidence on key topics, and support the commissioning of further research by identifying evidence gaps’’ [17]. Based on the themes identified in this review, these definitions do not fully define RRs.

As a field of research, RRs need to at least develop a minimum set of criteria. If the concept of ‘rapid review’ is better defined, it will enable future studies of this meth- odology to be a clearly distinguishable approach, measur- able to the extent possible, and understandable in terms of empirical observations. In a wider sense, researchers need to be able to describe what is and what is not a ‘rapid review’. Until such a time that a general working definition is established, it may hinder efforts to promote the utility of RRs to end users who may benefit from more timely evi- dence to inform their decision-making. Lack of an agreed-on definition may also unfairly hamper acceptance of ‘rapid reviews’ by journal editors as a legitimate publi- cation type and limit acknowledgment as a credible

83C. Hamel et al. / Journal of Clinical Epidemiology 129 (2021) 74e85

academic output in terms of promotion and tenure of re- searchers who undertake them. It also results in authors producing a variety of products which are labeled under a wide array of names, contributing to the lack of cohesion and unity around the method. Furthermore, having a widely accepted definition may facilitate the future funding of ‘rapid reviews’ by granting agencies. More generally, a definition would facilitate discussion about RRs and would improve understanding by end users. Collectively, this highlights the need for an evidence-informed definition of RR which can be adopted by researchers.

5.2. Strengths and limitations

This study provides a repository of existing definitions identified in the current literature, identifies general themes, and provides a flexible working definition of RRs to be used by the wider knowledge synthesis community. In addition, through our collaborative mapping, this study has allowed us a first glance at the network of RR researchers who, through their RR and methods work, have provided and cited defining features of RRs.

However, there were some limitations. First, for feasi- bility, only English journalepublished RRs identified in the databases that were searched were captured. The pur- pose of this scoping review was not to identify all RRs writ- ten in the included time period, but rather to get a sense of what definitions are currently being used. We included def- initions from 216 RRs and supplemented these with the definitions from 90 RR methods articles. It is likely that RRs not captured would use definitions that would fall un- der the eight key themes identified. Second, as the main purpose of this review was to extract definitions verbatim from RRs, some information was not extracted (e.g., fund- ing source of RR), as suggested by PRISMA-ScR, or was only extracted by 1 reviewer (e.g., the country of the corre- sponding author). In addition, in some cases, citations may not have specified a definition, but rather alluded to a component of that definition. For example, ‘‘Rapid review is a fairly new approach which has inherent strengths and limitations [2,20,28,52e54].’’ [55]. We did not delve into each reference to see which provided a definition and which were studies that evaluated the inherent strengths and lim- itations of RRs, but rather captured it in its entirety. In other cases, the reference provided was not specific to RRs but pointed to a methodology that was followed: ‘‘We conduct- ed a rapid systematic literature review after a priori devel- oped protocol [56].’’ [57]. It is therefore possible that some of the references may not actually provide a definition for RRs but instead may contain the methods of RRs or ratio- nale as to why one might conduct an RR. Third, several terms were identified during title and abstract screening, some of which may have been RRs but were not identified as such (Appendix C). Because of the number of records with these terms, they were excluded, for feasibility.

Therefore, it is possible that some reviews may have been missed that would qualify as an RR.

6. Conclusion

Eight key themes were identified, which have been considered in developing a preliminary, broad definition of an RR. This suggested definition, with additional caveats and opportunity for flexibility, will help the systematic re- view community define their review with consistency, regardless of the label used to describe it. Failure to use a consistent definition, or at least a minimum set of criteria, will be a barrier to moving the science forward in this field.

CRediT authorship contribution statement

Candyce Hamel: Conceptualization, Funding acquisi- tion, Data curation. Alan Michaud: Investigation, Valida- tion, Writing - review & editing. Micere Thuku: Investigation, Validation, Writing - review & editing. Becky Skidmore: Data curation. Adrienne Stevens: Conceptualization, Funding acquisition, Writing - review & editing. Barbara Nussbaumer-Streit: Conceptualiza- tion, Funding acquisition, Writing - review & editing. Chantelle Garritty: Conceptualization, Data curation.

Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.jclinepi.2020.09.041.

References

[1] Best L, Stevens A, Colin-Jones D. Rapid and responsive health tech-

nology assessment: the development and evaluation process in the

South and West region of England. J Clin Effectiveness 1997;2(2):

51e6.

[2] Khangura S, Konnyu K, Cushman R, Grimshaw J, Moher D. Evi-

dence summaries: the evolution of a rapid review approach. Syst

Rev 2012;1:10.

[3] Tricco AC, Antony J, Zarin W, Strifler L, Ghassemi M, Ivory J, et al.

A scoping review of rapid review methods. BMC Med 2015;13:224.

[4] Kelly SE, Moher D, Clifford TJ. Defining rapid reviews: a modified

DELPHI consensus approach. Int J Technol Assess Health Care

2016;32(4):265e75.

[5] Tricco AC, Langlois EV, Straus SE. (editors). Rapid Reviews to

Strengthen Health Policy and Systems: A Practical Guide. Geneva:

World Health Organization; 2017. Available at https://www.who.

int/alliance-hpsr/resources/publications/rapid-review-guide/en/. Ac-

cessed November 15, 2019.

[6] Crawford C, Boyd C, Jain S, Khorsan R, Jonas W. Rapid Evidence

Assessment of the Literature (REAL(�)): streamlining the systematic

review process and creating utility for evidence-based health care.

BMC Res Notes 2015;8:631.

[7] CADTH Rapid Response Service: about the rapid response service.

Available at https://www.cadth.ca/about-cadth/what-we-do/products-

services/rapid-response-service. Accessed November 12, 2019.

84 C. Hamel et al. / Journal of Clinical Epidemiology 129 (2021) 74e85

[8] Aronson JK, Heneghan C, Mahtani KR, Pluddemann A. A word

about evidence: ‘‘rapid reviews’’ or ‘‘restricted reviews’’? BMJ

Evidence-Based Med 2018;23(6):204e5.

[9] Pluddemann A, Aronson JK, Onakpoya I, Heneghan C, Mahtani KR.

Redefining rapid reviews: a flexible framework for restricted system-

atic reviews. BMJ Evidence-Based Med 2018;23(6):201e3.

[10] Arksey H, O’Malley L. Scoping studies: towards a methodological

framework. Int J Soc Res Methodol 2005;8(1):19e32.

[11] Peters MDJ, Godfrey C, McInerney P, Baldini Soares C, Khalil H,

Parker D. Chapter 11: scoping reviews. In: Aromataris E, Munn Z,

editors. Joanna Briggs Institute Reviewer’s Manual. https://

reviewersmanual.joannabriggs.org/. Accessed October 5, 2018.

[12] Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping

reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med

Published Online September 2018;4.

[13] McGowan J, Sampson M, Salzwdel DM, et al. PRESS Peer Review of

Electronic Search Strategies: 2015 Guideline Statement. J Clin Epide-

miol 2016;75:40e6. https://doi.org/10.1016/j.jclinepi.2016.01.021.

[14] Hamel C, Michaud A, Thuku M, et al. Few evaluative studies exist

examining rapid review methodology across stages of conduct: a sys-

tematic scoping review. J Clin Epidemiol 2020;126:131e40. https:

//doi.org/10.1016/j.jclinepi.2020.06.027.

[15] DistillerSR [Computer Program]. https://v2dis-prod.

evidencepartners.com.

[16] Harrison R, Manias E, Mears S, Heslop D, Hinchcliff R, Hay L. Ad-

dressing unwarranted clinical variation: a rapid review of current ev-

idence. J Eval Clin Pract 2019;25:53e65. [17] Department for International Development. Rapid Evidence Assess-

ments. GOV.UK. 2017. Available at https://www.gov.uk/government/

collections/rapid-evidence-assessments. Accessed November 12, 2019.

[18] GrantMJ, BoothA. A typology of reviews: an analysis of 14 review types

and associated methodologies. Health Info Libr J 2009;26:91e108.

[19] Bagnasco A, Cadorin L, Barisone M, Bressan V, Iemmi M, Prandi M,

et al. Ethical dimensions of paediatric nursing: a rapid evidence

assessment. Nurs Ethics 2018;25(1):111e22.

[20] Ganann R, Ciliska D, Thomas H. Expediting systematic reviews:

methods and implications of rapid reviews. Implement Sci 2010;5:56.

[21] Abou-Setta AM, Jeyaraman M, Attia A, Al-Inany HG, Ferri M,

Ansari MT, et al. Methods for developing evidence reviews in short

periods of time: a scoping review. PLOS ONE 2016;11:e0165903.

[22] Harker J, Kleijnen J. What is a rapid review? A methodological

exploration of rapid reviews in Health Technology Assessments. Int

J Evid Based Healthc 2012;10(4):397e410.

[23] Kelly SE, Moher D, Clifford TJ. Quality of conduct and reporting in

rapid reviews: an exploration of compliance with PRISMA and AM-

STAR guidelines. Syst Rev 2016;5:79.

[24] Schunemann HJ, Moja L. Reviews: Rapid! Rapid! Rapid! ... and sys-

tematic. Sys Rev 2015;4:4.

[25] Thomas J, Newman M, Oliver S. Rapid evidence assessments of

research to inform social policy: taking stock and moving forward.

Evid Policy 2013;9(1):5e27.

[26] Varker T, Forbes D, Dell L, Weston A, Merlin T, Hodson S, et al.

Rapid evidence assessment: increasing the transparency of an

emerging methodology. J Eval Clin Pract 2015;21:1199e204.

[27] Watt A, Cameron A, Sturm L, Lathlean T, Babidge W, Blamey S,

et al. Rapid versus full systematic reviews: validity in clinical prac-

tice? 2008;78(11).

[28] Watt A, Cameron A, Sturm L, Lathlean T, Babidge W, Blamey S,

et al. Rapid reviews versus full systematic reviews: an inventory of

current methods and practice in health technology assessment. Int J

Technol Assess Health Care 2008;(2):133.

[29] Khangura S, Polisena J, Clifford TJ, Farrah K, Kamel C. Rapid re-

view: an emerging approach to evidence synthesis in health technol-

ogy assessment. Int J Technol Assess Health Care 2014;30(1):20e7.

[30] Polisena J, Garritty CM, Kamel C, Stevens A, Abou-Setta AM. Rapid

review programs to support health care and policy decision making: a

descriptive analysis of processes and methods. Syst Rev 2015;4:26.

[31] Peterson K, Floyd N, Ferguson L, Christensen V, Helfand M. User

survey finds rapid evidence reviews increased uptake of evidence

by Veterans Health Administration leadership to inform fast-paced

health-system decision-making. Syst Rev 2016;5.

[32] Coster JE, Turner JK, Bradbury D, Cantrell A. Why do people choose

emergency and urgent care services? A rapid review utilizing a sys-

tematic literature search and narrative synthesis. Acad Emerg Med

2017;24(9):1137e49.

[33] Polisena J, Garritty C, Umscheid CA, et al. Rapid Review Summit:

an overview and initiation of a research agenda. Syst Rev 2015;4:111.

[34] Patnode CD, Eder ML, Walsh ES, Viswanathan M, Lin JS. The use of

rapid review methods for the U.S. Preventive services task force. Am

J Prev Med 2018;54(1S1):S19e25.

[35] Chambers S, Barton KL, Albani V, Anderson AS, Wrieden WL.

Identifying dietary differences between Scotland and England: a

rapid review of the literature. Public Health Nutr 2017;20(14):

2459e77.

[36] Centre forReviews andDissemination. SystematicReviews:CRD’s guid-

ance for undertaking reviews in health care. 2009. Available at http://

file:///C:/Dropbox/Dropbox/Systematic%20review/CRD%20-%20How

%20to%20conduct%20a%20SR.pdf. Accessed September 25, 2018.

[37] Alison Rodriguez, Joanna Smith, Kirstine McDermid. Dignity ther-

apy interventions for young people in palliative care: a rapid struc-

tured evidence review. Int J Palliat Nurs. (7):339.

[38] Lal S, Adair C. E-mental health: a rapid review of the literature. Psy-

chiatr Serv 2014;65(1):24e32.

[39] Huband N, Furtado V, Schel S, et al. Characteristics and needs of

long-stay forensic psychiatric inpatients: a rapid review of the litera-

ture. In: Andreasson AA, Baldwin B, Beer B, Braun B, Butwell C,

et al, editors. The International Journal of Forensic Mental Health,

17 2018:45e60.

[40] Booth A, Sutton A, Papaioannou D. Systematic Approaches to a Suc-

cessful Literature Review. 2nd ed. London, UK: SAGE Publications

Ltd.; 2016.

[41] Clibbens N, Harrop D, Blackett S. Early discharge in acute mental

health: a rapid literature review. Int J Ment Health Nurs 2018;

27(5):1305e25.

[42] Naughton AM, Cowley LE, Tempest V, Maguire SA, Mann MK,

Kemp AM. Ask Me! self-reported features of adolescents experi-

encing neglect or emotional maltreatment: a rapid systematic review.

Child Care Health Dev 2017;43(3):348e60.

[43] M€annist€o II, Pirttimaa RA. A review of interventions to support the

educational attainments of children and adolescents in foster care.

Adoption & Fostering 2018;42(3):266e81.

[44] University of Leicester - Student Learning Development. 1.13 sec-

ondary reference. University of Leicester. Available at https://

www2.le.ac.uk/offices/ld/resources/writing/harvard/content/1.13-

secondary-reference. Accessed November 3, 2019.

[45] McKenzie J, Brennan S, Ryan R, Thomson H, Johnston R, Thomas J.

Chapter 3: defining the criteria for including studies and how they

will be grouped for the synthesis. Cochrane Handbook for Systematic

Reviews of Interventions Version 6.0. 2019. Available at https://

training.cochrane.org/handbook/current/chapter-03. Accessed July

11, 2020.

[46] Higgins J, Lasserson T, Chandler J, et al. Methodological Expecta-

tions of Cochrane Intervention Reviews (MECIR): Standards for

the Conduct and Reporting of New Cochrane Intervention Reviews,

Reporting of Protocols and the Planning. London, UK: Cochrane;

2019. Available at https://community.cochrane.org/mecir-manual.

Accessed November 15, 2019.

[47] Hartling L, Guise J-M, Kato E, Anderson J, Belinson S, Berliner E,

et al. A taxonomy of rapid reviews links report types and methods to

specific decision-making contexts. J Clin Epidemiol 2015;68:

1451e1462.e3. [48] O’Leary DF, Casey M, O’Connor L, et al. Using rapid reviews: an

example from a study conducted to inform policy-making. J Adv

Nurs 2017;73:742e52.

85C. Hamel et al. / Journal of Clinical Epidemiology 129 (2021) 74e85

[49] Lawani MA, Valera B, Fortier-Brochu E, L�egar�e F, Carmichael PH,

Côt�e L, et al. Five shared decision-making tools in 5 months: use

of rapid reviews to develop decision boxes for seniors living with de-

mentia and their caregivers. Syst Rev 2017;6(1):56.

[50] Moher D, Tsertsvadze A. Systematic reviews: when is an update an

update? Lancet 2006;367:881e3.

[51] Colquhoun HL, Levac D, O’Brien KK, Straus S, Tricco AC,

Perrier L, et al. Scoping reviews: time for clarity in definition,

methods, and reporting. J Clin Epidemiol 2014;67:1291e4. [52] Abrami PC, Borokhovski E, Bernard RM, Wade CA, Tamim R,

Persson T, et al. Issues in conducting and disseminating brief reviews

of evidence. Evid Policy 2010;6(3):371e89. [53] Australian Safety and Efficacy Register of New Interventional Proced-

ures -Surgical. Rapid versus full systematic reviews: an inventory of cur-

rent methods and practice in Health Technology Assessment. Report no

60. Stepney: Australian Safety and Efficacy Register of New Interven-

tional Procedures -Surgical (ASERNIP-S). Report number 60 2007.

Available at https://www.crd.york.ac.uk/CRDWeb/ShowRecord.asp?

ID=32007000667&ID=32007000667. Accessed November 18, 2019.

[54] Gough D, Thomas J, Oliver S. Clarifying differences between review

designs and methods. Syst Rev 2012;1:28.

[55] Bhardwaj K, Locke T, Biringer A, Booth A, Darling EK, Dougan S,

et al. Newborn Bilirubin screening for preventing severe Hyperbilir-

ubinemia and Bilirubin encephalopathy: a rapid review. Curr Pediatr

Rev 2017;13(1):67e90. [56] Hartling L, Guise J-M, Kato E, et al. EPC Methods: An Exploration

of Methods and Context for the Production of Rapid Reviews.

Rockville (MD): Agency for Healthcare Research and Quality

(US); 2015.

[57] Aronow WS, Shamliyan TA. Comparative effectiveness and safety of

Rivaroxaban in adults with nonvalvular atrial fibrillation. Am J Ther

2018.