psychology

Clin Psychol Sci Pract. 2018;25:e12271. | 1 of 12 https://doi.org/10.1111/cpsp.12271

wileyonlinelibrary.com/journal/cpsp

1 | INTRODUCTION

Although efficacious treatments have been developed, many individuals who could benefit from these interven- tions do not have access to them (President’s New Freedom Commission on Mental Health, 2003). Efforts to reduce the gap between research findings about effective treatments and the practice of providing effective services have been both long‐standing and conscientious, yet not entirely successful. Additional strategies and efforts are needed to achieve opti- mal implementation.

As noted by the American Psychological Association (APA) Presidential Task Force on Evidence‐Based Practice (EBP; 2006), EBP in psychology is an umbrella term that refers to “the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences” (p. 273). In contrast to EBP, em- pirically supported treatments (ESTs) are typically devel- oped to address a particular disorder and then evaluated in rigorous efficacy studies (i.e., randomized controlled trials [RCTs]). Even though EST is a narrower term, we argue

that protocols that are considered “ESTs” can effectively be implemented using an EBP approach. Thus, we discuss the particular importance of employing flexibility within fidel- ity when implementation efforts are focused on protocols. Consistent with the definition of EBP, this approach includes making appropriate adaptations to protocols based primarily on available research evidence, but also on clinical exper- tise and patient features (APA Presidential Task Force on Evidence‐Based Practice, 2006; Lilienfeld, Ritschel, Lynn, Cautin, & Latzman, 2013).

Research that effectively increases the availability of evidence‐based treatments via large‐scale dissemination and implementation efforts will benefit from development within a theoretical framework. One such framework is the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009), which incorporates five domains: the intervention, the outer setting (i.e., factors ex- ternal to the organization), the inner setting (i.e., character- istics of the organization implementing the intervention), the individual (i.e., characteristics of individuals involved in implementation), and the process of implementation (e.g.,

Received: 31 August 2017 | Revised: 5 February 2018 | Accepted: 8 February 2018

DOI: 10.1111/cpsp.12271

L I T E R A T U R E R E V I E W

Implementing evidence‐based treatment protocols: Flexibility within fidelity

Philip C. Kendall | Hannah E. Frank

© 2018 American Psychological Association. Published by Wiley Periodicals, Inc., on behalf of the American Psychological Association. All rights reserved. For permissions, please email: [email protected]

Temple University, Philadelphia, Pennsylvania

Correspondence Hannah E. Frank, Psychology, Temple University, Philadelphia, PA. Email: [email protected]

Funding information National Institute of Mental Health, Grant/ Award Number: F31MH112211

Abstract Efficacious psychological treatments exist for a variety of mental health conditions, but many who could benefit from these treatments do not receive them. Increasing efforts have been made to disseminate effective protocols, and several approaches for implementing such treatments have been proposed, including the use of proto- cols, principles, practices, and policies. We discuss the relative merits of disseminat- ing protocols and highlight the importance of employing flexibility within fidelity. We describe the benefits of using protocols, including their empirical support, guid- ance for decision‐making, and structure to facilitate training and enhance treatment integrity. We also address several criticisms that have been offered against protocols, citing data that indicate that many of the criticisms are not warranted.

K E Y W O R D S CBT, dissemination, evidence‐based, implementation, protocols

Th is

d oc

um en

t i s c

op yr

ig ht

ed b

y th

e A

m er

ic an

P sy

ch ol

og ic

al A

ss oc

ia tio

n or

o ne

o f i

ts a

lli ed

p ub

lis he

rs .

Th is

a rti

cl e

is in

te nd

ed so

le ly

fo r t

he p

er so

na l u

se o

f t he

in di

vi du

al u

se r a

nd is

n ot

to b

e di

ss em

in at

ed b

ro ad

ly .

2 of 12 | KENDALL AND FRANK

planning, execution). Although each domain requires atten- tion, a necessary first step is selecting the intervention to be disseminated and implemented. Examining and selecting the specific components of evidence‐based care that are best suited for dissemination is a critical element of the CFIR. According to the CFIR (Damschroder et al., 2009), interven- tions include “core components,” which are key elements that cannot be changed and an “adaptable periphery,” which are the parts of the intervention that can be adapted for the imple- mentation context and setting. The present paper considers the strengths of implementing protocols, which have identi- fiable core components and adaptable peripheries that can be leveraged to optimize the successful implementation of EBPs. We also highlight specific advantages of using treat- ment protocols and discuss some of the criticisms and how they have been addressed.

2 | FLEXIBILITY WITHIN FIDELITY

A discussion of the merits of protocols first necessitates consideration of the importance of “flexibility within fidel- ity” (Kendall, Gosch, Furr, & Sood, 2008). Fidelity refers to therapist adherence to a protocol’s treatment components, therapist competence in delivering the intervention, and the degree to which the treatment differs from others (i.e., treat- ment differentiation; Schoenwald et al., 2011). Flexibility within fidelity refers to the implementation of an empirically supported protocol in a manner that contains the core ingredi- ents to attain fidelity, but that adapts its implementation to be in sync with individual client presentations. A core ingredi- ent, for example, could be the use of homework. For fidelity, homework would need to be implemented; for flexibility, the implementation would allow for variation, such as different homework for different‐aged youth (i.e., consideration of cli- ent characteristics). A core ingredient may be similar across protocols for different diagnostic groups, but in that case, flexibility within fidelity would warrant a different approach to implementing that “ingredient” (i.e., treatment differentia- tion). For example, protocols for both depression and anxiety have demonstrated benefits from taking a problem‐solving perspective; this ingredient would be necessary for fidelity, but would be implemented differently for depression and for anxiety. Whereas problem‐solving for depression may focus on addressing barriers to behavioral activation, prob- lem‐solving for anxiety may involve a discussion of the types of exposures that could effectively target a feared stimulus. Similarly, flexibility within fidelity is relevant to the applica- tion of modular approaches to treatment, where core strate- gies are linked to specific problems and applied sequentially (see Chorpita, Daleiden, & Weisz, 2005; Weisz et al., 2012). In essence, at any point along the sequence, the strategies

that are deemed essential for an empirically supported proto- col must be present for the intervention to be delivered with fidelity, but in the spirit of EBP, the application of the core strategy is accomplished in client‐specific fashion (i.e., com- petently). Consistent with the CFIR, “core components” are preserved and presented but within an “adaptable periphery.” Criticisms directed toward evidence‐based protocols are based in part on the misconception that protocols must be im- plemented rigidly. Proper implementation involves a flexible application of manual‐based procedures (protocols) in which the therapist is sensitive and adaptive to client characteris- tics and needs. Thus, when implemented flexibly, protocols lend an ideal opportunity to cultivate an “evidence‐based ori- entation to practice” (APA Task Force on Evidence‐Based Practice with Children & Adolescents, 2008).

Another consideration related to appropriateness for im- plementation is that treatment protocols can be of varying quality. At the least desirable end of the continuum are rigid manuals that lack in the communication of the guiding prin- ciples. In contrast, quality manuals are principle‐driven, ac- knowledge and guide adaptations, and reflect a consistency between principles and procedures. Preferred manuals in- clude an underlying theoretical framework, treatment and session goals, specific activities to meet the stated goals, as well as strategies for addressing and overcoming potential challenges. The use of treatment protocols does require some level of adherence to the protocol, but treatment goals can and should be met using a degree of flexibility and creativity. Several illustrative examples will be provided. Our examples involve cognitive behavioral therapy (CBT), a treatment ap- proach that has been deemed efficacious (Butler, Chapman, Forman, & Beck, 2006; Hollon & Beck, 2013). Specifically, we will use CBT for youth anxiety to illustrate flexible appli- cations for meeting goals.

Across several specific protocols, the manuals for CBT for youth anxiety incorporate cognitive restructuring (i.e., changing self‐talk). Nevertheless, several factors, such as developmental level, insight, and ability to engage in ab- stract thinking, influence the specific ways in which cog- nitive restructuring for anxiety is implemented for children and adolescents. For younger children, we may assist with identifying cognitive distortions without mentioning “cog- nitive distortions,” but by distinguishing between their own thoughts and the things that the “worry monster” tells them. Externalizing anxiety as the “worry monster” often makes it easier for children to generate ideas of what to “say back to the worry monster” (i.e., coping thoughts). In addition, some children prefer to label coping thoughts with their own terms, such as “feel better thoughts.” Older children and adolescents learn about thinking traps, which they can recognize in them- selves and be able to address relative to their own thoughts. Alternatively, adolescents can play a matching game to iden- tify the thinking traps that apply to other peoples’ thoughts.

Th is

d oc

um en

t i s c

op yr

ig ht

ed b

y th

e A

m er

ic an

P sy

ch ol

og ic

al A

ss oc

ia tio

n or

o ne

o f i

ts a

lli ed

p ub

lis he

rs .

Th is

a rti

cl e

is in

te nd

ed so

le ly

fo r t

he p

er so

na l u

se o

f t he

in di

vi du

al u

se r a

nd is

n ot

to b

e di

ss em

in at

ed b

ro ad

ly .

| 3 of 12KENDALL AND FRANK

Discussion of characters from books or television shows can be used to facilitate this type of conversation. A protocol may state that the therapist should address cognitive distortions— but the manner in which this is accomplished varies as the manual “comes to life” (Kendall, Chu, Gifford, Hayes, & Nauta, 1998). Fidelity would not permit a failure to address anxious self‐talk, but flexibility would permit client‐appro- priate adaptions of the topic.

In a review of manual‐based treatments, Wilson (2007) provides additional examples of ways in which therapists in- dividualize protocols, such as formulating a treatment plan that is consistent both with the overall treatment model and the client’s presenting problem, identifying dysfunctional and unhelpful beliefs specific to the client, and maximizing time using strategies found to work for a given client. In short, the benefit of employing a protocol is most evident when it is used with flexibility and within fidelity rather than as a set of strict requirements to which the therapist is expected to rigidly adhere. As emphasized by Stirman et al. (2015), Stirman, Gamarra, Bartlett, Calloway, and Gutner (2017) and Stirman, Miller, Toder, and Calloway (2013), there is a distinction between fidelity‐consistent and fidelity‐incon- sistent modifications to protocols. Modifications are defined as planned or unplanned changes to the content or delivery of a protocol (Stirman et al., 2015). Modifications are often made by therapists to address domains consistent with the CFIR (e.g., inner setting and the individual) that are not spe- cifically incorporated into the treatment protocol. In some instances, these changes are appropriate, fidelity‐consistent, and responsive to contextual factors. In other instances, these modifications are fidelity‐inconsistent and deviate far from the originally designed intervention. As noted, modifications are most acceptable if they are made within the “adaptable periphery” and do not undermine the integrity of the inter- vention (Damschroder et al., 2009; Stirman et al., 2015).

In order to assess changes to intervention delivery, Stirman et al. (2013) developed a useful model for think- ing about modifications to treatment protocols when they are applied in routine care settings. For example, contextual modifications can occur at the format (e.g., group vs. individ- ual), setting (e.g., clinic vs. school), personnel (e.g., mental health professional vs. teacher), and population (e.g., anxiety vs. depression) levels. Protocols may be amenable to such modifications. Content modifications (e.g., tailoring, con- densing, extending, re‐ordering) are also examples of ways that protocols can be implemented flexibly, but only when the decision‐making takes into account therapist knowledge, patient characteristics, and available research evidence. In their review, Stirman et al. (2017) found that the few studies that actually compared adapted and non‐adapted treatments found that outcomes were generally comparable, suggesting that protocols may not be as fragile as sometimes suggested. Relatedly, Goldstein, Kemp, Leff, and Lochman (2012)

provide guidance for a step‐by‐step approach to appropriately adapting protocols for new populations. Additional work is needed to identify the specific types of protocol modifica- tions that are appropriate and sufficiently meet the threshold for fidelity across the various treatment targets.

3 | OVERLAP BETWEEN PROTOCOLS AND PRINCIPLES

The phrase “treatment protocol” can easily elicit a sense of required rule following. “Treatment principles” suggests a less rigid directive that is guided by theory. It is notewor- thy that although empirically supported protocols are dis- tinct from evidence‐based principles, there are meaningful similarities. Empirically supported protocols are generally developed based on a core set of underlying theoretical prin- ciples (Clark, 2004), and although certain features may vary across protocols, the underlying principles are similar. The consistency across protocols is in the adherence to the under- lying principles. For example, versions of CBT for anxiety have been developed for young children (e.g., Being Brave: Hirshfeld‐Becker et al., 2010; Hirshfeld‐Becker et al., 2008), school‐age children (e.g., Coping Cat: Kendall & Hedtke, 2006), adolescents (e.g., C.A.T. project; Kendall, Choudhury, Hudson, & Webb, 2002), and emerging adults (e.g., L.E.A.P; Zakarin & Albano, 2016). Each of these protocols shares similar ingredients (i.e., homework, cognitive restructures, exposure tasks) that have been tailored to be developmen- tally appropriate. Although a therapist may wish to follow the specific protocol for a specific age group, it is also reasonable for a therapist who is proficient with any one of the protocols to make the appropriate developmental adaptations.

A common and potentially accurate argument against using protocols is that therapists would be expected to learn a large number of different protocols in order to be prepared to treat clients with different presenting problems. Carried to an extreme, this argument has some merit. However, the actual number of empirically supported protocols is fewer than one would think, and there are several common princi- ples across them. In essence, one would not need to master a large number of protocols to treat a wide variety of pre- senting problems. Once therapists have an understanding of the underlying theory and principles of the core ingredi- ents across several protocols, as well as an understanding of how to implement them flexibly, they are prepared to treat a wide range of disorders. Learning about the princi- ples that are the underpinnings of the treatment, the factors that maintain disorders, and the flexible implementation of treatments would extend the usefulness of protocols to var- ious ages and presentations. One could assert that, in many instances, there is a blurry distinction between using em- pirically supported protocols and applying evidence‐based

Th is

d oc

um en

t i s c

op yr

ig ht

ed b

y th

e A

m er

ic an

P sy

ch ol

og ic

al A

ss oc

ia tio

n or

o ne

o f i

ts a

lli ed

p ub

lis he

rs .

Th is

a rti

cl e

is in

te nd

ed so

le ly

fo r t

he p

er so

na l u

se o

f t he

in di

vi du

al u

se r a

nd is

n ot

to b

e di

ss em

in at

ed b

ro ad

ly .

4 of 12 | KENDALL AND FRANK

principles. In our perspective, protocols provide structure for treatment and provide a clear starting point for dissem- inating evidence‐based treatments.

4 | STRENGTHS OF EVIDENCE‐ BASED PROTOCOLS

Psychological treatments for mental health conditions have been written about for over a century, but until recently, the descriptions were largely theoretical, difficult to operational- ize, and nonspecific. However, mental health treatments can be operationalized and described in sufficient detail to permit reliable application. The use of treatment protocols represents one approach for operationalizing mental health treatments, but this approach is not without shortcomings. Nevertheless, there are several positive features, including empirical sup- port, decision‐making, treatment integrity, and training.

4.1 | Empirical support Empirically supported protocols, by definition, have been evalu- ated empirically—usually in the context of methodologically rig- orous RCTs. In addition, they are typically developed with a strong theoretical rationale and address the factors that maintain the dis- order. Protocols that qualify as empirically supported (Chambless & Hollon, 1998) have been evaluated at more than one site, by more than one investigator, with real patients and with rigorous and blind evaluation of outcomes. One benefit of using an empiri- cally supported protocol is being able to cite to a client (or a client’s parents) the percentage of people for whom the treatment has been efficacious in the past. Of course, individual variations in response to treatment should be acknowledged, but those varying responses may be addressed by applying the protocol flexibly.

What about treatment protocols that do not have empiri- cal support, or have been found to be less efficacious? Such treatments are not only less likely to have a favorable out- come, but they can also be undermining to the client’s future efforts to change. One would not want a client to end treat- ment thinking, “I went to treatment and it didn’t help.” Less optimal treatments can also be damaging, as in the client who ends treatment and thinks “I must be really troubled; I went to treatment and I am still not any better.” In short, use of an empirically supported (i.e., found to be optimally effective) treatment is preferred not only for the mere probability of success, but also for the reduced likelihood of undermining client motivation and client treatment‐seeking efforts.

4.2 | Decision‐making There are many variations of psychotherapy for mental health disorders. Some have been evaluated favorably, some have been evaluated and found to be less effective, and some

have not yet been properly evaluated. Given the breadth and variety of interventions available, how does one decide what to implement in a “real‐world” setting? As noted by the APA Presidential Task Force on Evidence‐Based Practice (2006), an EBP approach requires clinical expertise (i.e., knowledge and proficiency across several domains) and knowledge of research evidence. The use of empirically supported proto- cols minimizes this burden to some extent because of the use of clear guidelines to dictate research quality, as well as a clear outline for the clinical strategies to be used. Southam‐ Gerow and Prinstein (2014) note that researchers have been attempting since the 1960s to guide provider decision‐mak- ing when it comes to selecting appropriate treatments. These efforts, over the past several decades, have largely focused on assessing the evidence for protocol‐based treatments. The decision‐making process for selecting what treatment to implement is often aided by selecting the protocol with the most and strongest empirical support for the target problem. The Evidence‐Base Update series (e.g., Dorsey, McLaughlin, et al., 2017; Evans, Owens, & Bunford, 2014; Freeman et al., 2014; Kaminski & Claussen, 2017) in the Journal of Clinical Child and Adolescent Psychology pro- vides disorder‐specific reviews of the evidence to guide de- cisions about psychosocial interventions.

In the Evidence‐Base Update articles, treatment ap- proaches are sorted by level of empirical support. The criteria (Southam‐Gerow & Prinstein, 2014) include well‐established, probably efficacious, possibly efficacious, experimental, and treatments of questionable efficacy. These criteria were ad- opted from previous work (e.g., Chambless, Baker, Baucom, Beutler, & Calhoun, 1998; Chambless & Hollon, 1998). As Southam‐Gerow and Prinstein (2014) stated, “the goal of these various evaluation criteria are to inform professionals and consumers about the state of the science” (p. 3). Perhaps one of the greatest strengths in promoting the implementa- tion of empirically supported protocols is the high‐quality scientific evidence to support their use. The Evidence‐Based Update series is a useful resource for understanding the methodological rigor and degree of research support behind each protocol. It highlights the fact that protocols are not all “created equal,” but many have been subject to rigorous test- ing and comparison. In addition, treatments can be selected based on the level of evidence—those with the most evidence may be most appropriate for implementation efforts.

The effects of any given treatment can vary, and there may be pretreatment predictors that suggest that certain cli- ents will benefit more than others. This information can help decision making. In addition, when there are two or more treatments with empirical support, outcomes may dif- fer depending on moderators (e.g., age, gender, ethnicity; Chorpita et al., 2011). When one examines the evidence base for a given protocol, it is useful to consider predictors, potential moderators, and evidence of generalizability.

Th is

d oc

um en

t i s c

op yr

ig ht

ed b

y th

e A

m er

ic an

P sy

ch ol

og ic

al A

ss oc

ia tio

n or

o ne

o f i

ts a

lli ed

p ub

lis he

rs .

Th is

a rti

cl e

is in

te nd

ed so

le ly

fo r t

he p

er so

na l u

se o

f t he

in di

vi du

al u

se r a

nd is

n ot

to b

e di

ss em

in at

ed b

ro ad

ly .

| 5 of 12KENDALL AND FRANK

4.3 | Enhances treatment integrity One challenge facing service providers who seek to imple- ment evidence‐based treatments in real‐world settings is ensuring treatment integrity, or adherence to the treatment protocol. Measuring and maintaining treatment fidelity can be difficult, especially if the criteria for fidelity are not well defined (Schoenwald et al., 2011). Protocols are often well‐ suited for establishing criteria for fidelity, as the inclusion of core strategies is a near‐universal feature of protocols. Importantly, fidelity does not “wipe out” attending to the cli- ent’s needs. Rigid adherence is likely associated with poorer outcomes (Castonguay, Goldfried, Wiser, Raue, & Hayes, 1996), highlighting again the need for measures of compe- tence (Schoenwald et al., 2011), as well as flexibility within fidelity. For example, the Coping Cat manual provides sev- eral analogies relevant to sports to explain self‐rating and self‐reward. Although such examples resonate with many children and adolescents, other youth prefer analogies that relate to their favorite books, television shows, or other extra- curricular activities. Adapted analogies are considered adher- ent (i.e., meet treatment fidelity). In another example, a child who has a fear of vomiting will likely generate a different list of feared situations than a child who is afraid of the dark. The process of generating a list of feared situations would be the essential “core component” that would be evaluated for in- tegrity, but the specific entries would vary by client. In each example, the protocol would be administered with integrity (i.e., adherent to the protocol) while also tailoring it for the individual (i.e., therapist competence).

Another potential advantage of using treatment proto- cols is that they may reduce the number of therapy sessions required. Unlike the gardener who set the hose on spray and seeks to try to water everything, protocols set their hose on stream and soak the target problem. Because protocols include sessions that are intentionally goal‐directed and provide clear strategies for meeting these goals, they are likely to address a target problem after fewer sessions than an unstructured approach.

4.4 | Facilitates training Having training and supervised experience in the delivery of an intervention is central to quality implementation (Schoenwald, Mehta, Frazier, & Shernoff, 2013). Similarly, having the in- tervention in written form (i.e., protocol) facilitates uniform and reliable training and supervision, which should be con- sidered when implementing ESTs in usual‐care settings (i.e., transportability; Schoenwald & Hoagwood, 2001). It is un- likely that multiple trainees with multiple trainers will learn the same program if the program is not written down.

A critical element in effective dissemination and im- plementation of evidence‐based treatments is the means

through which training is provided (Herschell, Kolko, Baumann, & Davis, 2010). Protocols are particularly well‐ suited to training because the intervention, in written form, details the core ingredients needed for fidelity. The simple structure of a protocol, too, can be helpful. Learning may be facilitated by the structure inherent in protocols, as trainees can easily be provided with an agenda and a specific set of goals to remain adherent to the treatment. As noted, clinical judgment remains relevant, but protocols have additional “built‐in” supports to buttress therapists’ use of relevant strategies.

Efforts to train therapists in evidence‐based protocols have been successful. For example, Smith et al. (2017) exam- ined how therapists in practice and research settings delivered the Coping Cat intervention for anxiety disorders. Providers were trained via a workshop led by an expert and subsequent ongoing supervision. Findings indicated that delivery of the intervention was similar in both research and practice set- tings. Although therapists in practice settings delivered an increasing number of non‐CBT interventions throughout the duration of the study, the prevalence of non‐CBT inter- ventions was still relatively low. In another study (Villabø, Narayanan, Compton, Kendall, & Neumer, 2018), thera- pists in clinics in Norway were also trained in the Coping Cat protocol. Although there was some initial hesitancy, the therapists were accepting of the approach and were reliable in their implementation. In addition, the outcomes (reduced youth anxiety) were comparable to the effects found in re- search clinics (Villabø et al., 2018). These studies provide ev- idence that community therapists can successfully be trained in a manual‐based treatment and that outcomes are favorable.

Using a manual‐based treatment may directly address some other concerns about training and implementation. For example, less time may be required to train therapists about evidence‐based protocols than about less structured treatments. Technology can also assist in the effort to be efficient in training. Khanna and Kendall (2008) provided therapists with a computerized training on treatment for youth anxiety (CBT4CBT) and found that participants gained more knowledge about the protocol than those who received manual‐based training. Similar results have been found for a web‐based training program for trauma‐ focused CBT (Cohen & Mannarino, 2008). Such training approaches may increase the feasibility of training in a pro- tocol and improve the speed with which such protocols can be disseminated.

5 | CRITICISMS OF TREATMENT PROTOCOLS

Of course, not all protocols are the same. A detailed pro- tocol does not itself ensure efficacy or effectiveness. Until

Th is

d oc

um en

t i s c

op yr

ig ht

ed b

y th

e A

m er

ic an

P sy

ch ol

og ic

al A

ss oc

ia tio

n or

o ne

o f i

ts a

lli ed

p ub

lis he

rs .

Th is

a rti

cl e

is in

te nd

ed so

le ly

fo r t

he p

er so

na l u

se o

f t he

in di

vi du

al u

se r a

nd is

n ot

to b

e di

ss em

in at

ed b

ro ad

ly .

6 of 12 | KENDALL AND FRANK

evaluated, a protocol is merely a protocol. However, once evaluated and once findings qualify the protocol as empiri- cally supported, the protocol acquires the status of meriting dissemination and implementation. But even after proper evaluation, we would be naïve and uninformed to not ac- knowledge and recognize the several criticisms that have been leveled at protocol‐based treatments. Of note, sev- eral of the misconceptions broadly applicable to EBPs and refuted by Lilienfeld et al. (2013) also apply to protocols specifically. In some instances, the criticisms to protocols are applicable, but less so than critics suggest. In other in- stances, the criticisms may have applied in the past, but they are no longer applicable. The times, they have been a changin’.

5.1 | Protocols are “cookbooks” An early and somewhat common criticism of treatment protocols was that they were “cookbooks:” rigid and in- sensitive to individual clients’ needs. This criticism, if true and taken to extreme, would be legitimate. However, one treatment does not fit all; protocols are not intended to be implemented like a cookbook, but should instead be im- plemented in response to the individual client’s needs and presentation.

As the reader is likely aware, there are several disor- ders—each with their own theory‐driven and empirically supported protocols. And, even within a single protocol, there is room for “flexibility within fidelity.” Core ele- ments of ESTs exist for a variety of disorders, and these elements must be present when assessing adherence to a protocol. Examples include behavioral activation for de- pression (Dimidjian et al., 2006), behavioral parent training for disruptive behavior problems (Kaminski & Claussen, 2017), and exposures for anxiety disorders (Ale, McCarthy, Rothschild, & Whiteside, 2015). As noted previously, core elements must be included, but they are administered flex- ibly. The selected specific activities for behavioral activa- tion, for instance, vary from one individual to another, as might the frequency in which the client is expected to en- gage in these activities.

Many protocols for a target disorder include several nearly universal elements. For anxiety disorders, for example, these elements include homework, problem solving, cognitive re- structuring, and exposure tasks. Protocols may not indicate the exact amount of time that needs to be spent on each el- ement, but the protocol will indicate that the element needs to be incorporated. Clients who benefit from one element (e.g., exposure) but not another (e.g., cognitive restructur- ing) might spend differential amounts of time on these ele- ments. The implementation of a protocol is not the ticking off of items on a list. Rather, adjustments are made based on knowledge of the underlying principles of the treatment and

features of the client, and this knowledge is used to flexibly implement the elements.

On the other hand, too much flexibility is problematic. Too much flexibility can “break” fidelity. If a key “ingre- dient” is not included (e.g., not using exposures for treat- ing anxiety), then there is no treatment fidelity. In a study comparing CBT to usual care in community mental health clinics (Southam‐Gerow et al., 2010), 41% of the community therapists did not use the exposure tasks that were intended elements of the protocol. In those cases, if fidelity is being assessed, not including this key element removes fidelity and may in turn lead to less than optimal outcomes.

Kung Pao Chicken, in restaurants across the United States and Europe, has basic ingredients—but the dish nevertheless varies. Sometimes there are water chestnuts, sometimes there are mushrooms, and sometimes the peanuts are sautéed, yet other recipes have peanuts that are dry. Despite the variability in Kung Pao Chicken, there is always chicken and the dish is never Beef Wellington. Similarly, the specific ways in which the empirically supported protocol is implemented vary, but remain true to the protocol.

5.2 | Protocols detract from the therapeutic alliance For decades, with varying degrees of support, powerful at- tributions for treatment outcome have been made to the therapeutic alliance (i.e., therapist–client relationship). Some argue that the alliance is predictive of positive grains, others show evidence that gains precede the building of an alliance (DeRubeis & Feeley, 1990), and others report a reciprocal relationship between alliance and outcome (Marker, Comer, Abramova, & Kendall, 2013). Meta‐analyses examining the alliance–outcome relationship have shown varying estimates of the degree to which alliance is associated with youth treat- ment outcomes (McLeod, 2011; Murphy & Hutton, 2018; Shirk & Karver, 2003). However, one might consider the fol- lowing: Alliance may be a necessary component, but it is not sufficient.

A criticism of treatment protocols is that the emphasis placed on adherence and delivery of the treatment elements strains or de‐emphasizes the development of the therapeutic relationship. Despite this criticism, data indicate that a strong therapeutic alliance has been associated with better outcomes for manual‐based treatments (Shirk, Karver, & Brown, 2011), and that youth receiving protocol‐driven treatments in research and practice settings have equal or higher alliance relative to those receiving usual care (McLeod et al., 2016). Apparently, there is evidence that high fidelity to protocols and a strong therapeutic alliance can coexist.

To address concerns about the therapeutic relationship, Addis, Wade, and Hatgis (1999) suggested strategies to im- plement during training in evidence‐based protocols, such

Th is

d oc

um en

t i s c

op yr

ig ht

ed b

y th

e A

m er

ic an

P sy

ch ol

og ic

al A

ss oc

ia tio

n or

o ne

o f i

ts a

lli ed

p ub

lis he

rs .

Th is

a rti

cl e

is in

te nd

ed so

le ly

fo r t

he p

er so

na l u

se o

f t he

in di

vi du

al u

se r a

nd is

n ot

to b

e di

ss em

in at

ed b

ro ad

ly .

| 7 of 12KENDALL AND FRANK

as showing videotaped vignettes and completing role‐plays. Again, “flexibility within fidelity” is relevant; therapists can use varying degrees of warmth and directness to deliver the elements of the protocol while maintaining a strong alliance. Therapists may also benefit from the knowledge that the alli- ance is not as fragile as they may believe: Even the introduc- tion of exposures (to fear‐evoking stimuli) in therapy does not rupture the therapeutic alliance (Kendall et al., 2009).

5.3 | Protocols fit only a few (not broadly applicable) Treatment protocols have been described as not applying to diverse clinical presentations or clients. In his review of manual‐based treatments, Wilson (2007) pointed to the “all‐ or‐nothing thinking” inherent in this description. Thinking “flexibility within fidelity” broadens the scope. If therapists were to assume that they were restricted to only using the exact words/examples provided in a manual, protocols would be narrowly applicable. That said, protocols are not intended to treat all presenting problems: They are specific to identi- fied/targeted mental health conditions.

Typically, protocols target the main problem (i.e., the most interfering concern or the presenting problem with the highest severity), but this does not assume the absence of co‐occurring problems. There is often a misconception that RCTs exclude individuals who would be difficult to treat or who have comorbidities. However, Wilson (2007) noted that several studies (e.g., Barlow, Levitt, & Bufka, 1999; Stirman, Derubeis, Crits‐Christoph, & Rothman, 2005) found that RCTs do not have narrow exclusion criteria. In fact, modern RCTs often include individuals with more severe psychopa- thology and higher rates of comorbidity than in routine clini- cal practice (Westbrook & Kirk, 2005). Schindler, Hiller, and Witthoft (2011) substantiated this claim with a study that di- vided a naturalistic sample into two groups: those who would meet typical RCT inclusion criteria and those who would not. Their data analyses indicated no meaningful differences in treatment outcomes between the two groups, though both groups had a smaller effect size than participants in most RCTs. Post, Beer, and Guyatt (2013) also reviewed the lit- erature to assess how best to generalize efficacy results and suggested that results of RCTs should be assumed to apply to wider populations unless there is a compelling reason to believe otherwise.

The value of using diagnostic categories in the context of treatment protocols is typically to develop strategies that will benefit a relatively homogenous group (i.e., people with sim- ilar clusters of symptoms). Variations in presentations due to comorbidities, level of severity, external stressors, or other factors fall within an expected range. People with similar presenting problems should still benefit from similar treat- ments, especially if those treatments are flexibly applied to

the individual. Evidence for this point comes from research that has implemented evidence‐based treatments in uncon- trolled studies and in naturalistic settings and compared the outcomes to tightly controlled RCTs. For example, Peeters et al. (2013) allowed patients to select their preferred treat- ment for depression in a naturalistic setting. Several of the participants had high levels of comorbidity and recurrence of depression, and may not have met inclusion criteria for many efficacy studies. Yet, all groups had a significant de- crease in depression symptoms. Similar results were found when administering a protocol for severe mood disorders in a partial hospitalization program (Bjorgvinsson et al., 2014). Although participants’ symptoms were more severe than may be included in RCTs, the magnitude of symptom improve- ment was comparable to that found in RCTs. These findings suggest that protocols can successfully be applied to clients with severe and complex presenting problems.

It is also worth noting that when protocols are evalu- ated in modern research, the exclusion criteria are for those client features that would dictate an alternate treatment. In RCTs, severe cases and cases with comorbidities are in- cluded, but a case with a presenting problem that warrants a different treatment would be excluded. For example, in the Child/Adolescent Anxiety Multimodal Study (Walkup et al., 2008), a wide variety and severity of anxious youth met diagnostic criteria and received anxiety‐focused treatments (Kendall et al., 2010). Youth who met criteria for major de- pressive disorder, however, would be better treated with a program for depression and, thus, were excluded. Protocols do have broad applicability; the excluded clients are those whose presentation would dictate a different treatment.

One additional criticism of protocols is that they are not applicable to youth with diverse backgrounds. The re- search‐to‐practice gap is particularly large among racial and ethnic minorities in the United States, as indicated by lower utilization of mental health services and lower qual- ity of care (e.g., Alegría et al., 2008; Marrast, Himmelstein, & Woolhandler, 2016). Although RCTs may be limited by a lack of participant diversity (e.g., race, ethnicity, socio- economic status), some research has specifically exam- ined the degree to which evidence‐based protocols can be adapted for cultural and ethnic minorities. For example, Baumann et al. (2015) reviewed the ways in which empir- ically supported parent training protocols have undergone cultural adaptation (see also Stirman et al., 2013). As cited by Baumann et al. (2015), cultural adaptations involve “the systematic modification of an evidence‐based treatment (EBT) to consider language, culture, and context in such a way that is compatible with the client’s cultural patterns, meanings, and values” (Bernal, Bonilla, & Bellido, 1995, p. 362). Although relatively few studies on parenting in- terventions specifically focused on the effectiveness of cultural adaptations, the current evidence suggests that

Th is

d oc

um en

t i s c

op yr

ig ht

ed b

y th

e A

m er

ic an

P sy

ch ol

og ic

al A

ss oc

ia tio

n or

o ne

o f i

ts a

lli ed

p ub

lis he

rs .

Th is

a rti

cl e

is in

te nd

ed so

le ly

fo r t

he p

er so

na l u

se o

f t he

in di

vi du

al u

se r a

nd is

n ot

to b

e di

ss em

in at

ed b

ro ad

ly .

8 of 12 | KENDALL AND FRANK

protocols can be appropriately modified using cultural adaptation frameworks. These results are promising, but addressing the adaptability of protocols for underserved minorities is an area for continued work in all implementa- tion efforts, and does not apply only to protocols.

5.4 | Resource intensive A final issue, which is not necessarily specific to proto- cols, is the amount of training and supervision required for therapists to become proficient in delivering a given inter- vention. In the implementation of an empirically supported or evidence‐based treatment, it does not appear sufficient to simply improve therapists’ knowledge about the proto- col. To change behavior and increase adoption in practice, supervision/consultation appears to be a critical element of training (Beidas, Edmunds, Marcus, & Kendall, 2012; Edmunds et al., 2013). However, the amount of supervision provided in community clinics is often less than that pro- vided in RCTs (e.g., Kolko, Cohen, Mannarino, Baumann, & Knudsen, 2009). Thus, efforts for implementation ap- pear to benefit from an element of ongoing consultation (e.g., Kelly et al., 2000; Miller, Yahne, Moyers, Martinez, & Pirritano, 2004). Although it does not reduce the amount of resources required, Bearman et al. (2013) describe strat- egies that may optimize supervision and increase the likeli- hood that activities discussed in supervision will actually be implemented in practice. Although more work is needed to understand the best strategies for supervision/consulta- tion, Dorsey, Pullmann, et al. (2017) have identified the importance of implementation climate as one predictor of the amount of time dedicated to clinical and EBT‐related discussions in supervision.

As providers experience success in implementing proto- cols, attitudes toward them will likely improve. Research has consistently shown that having more positive attitudes toward EBPs is associated with greater likelihood of EBPs being im- plemented (Farrell, Kemp, Blakey, Meyer, & Deacon, 2016; Harned, Dimeff, Woodcock, & Contreras, 2013; Ruzek et al., 2015; Shafran et al., 2009). To support these arguments, there are several instances in which therapists in commu- nity mental health clinics (Foa et al., 2005) and with limited experience (Ost, Karlstedt, & Widen, 2012) have provided treatments and achieved comparable outcomes as therapists in academic settings and with more experience.

6 | DEMAND FOR AND REPRESENTATION OF ESTS

Efforts to disseminate and implement empirically supported protocols can be said to be motivated by both sound science and a caring social concern. However, there are factors that

can interfere with this good goal. We consider misuse of the claim of ESTs, the social forces that may facilitate eventual adoption of protocols, and the need to hold realistic expecta- tions regarding treatment outcomes.

6.1 | Using the buzzwords A treatment that qualifies as “empirically supported” has had a rigorous level of research evaluation, but the descrip- tive use of the label “empirically supported” to claim a level of support without proper reflection on its accuracy is a real and growing concern. There are times when, for a variety of reasons, members of the mental health services professions are engaged in tasks that do not fit neatly into the typical roles for which they have been explicitly trained (e.g., assessment, research, therapy). One of these alternate tasks is fundraising, be it seeking research support, insur- ance coverage for a clinic, or state funding for an agency. It is often within the arena of financial support that a problem emerges. For example, when seeking continued support from a state, a representative of a clinical service may state that their agency uses ESTs. The state representative who controls the funding wants programs that employ ESTs and reports being impressed by the agency. Although the agency may not intentionally mislead patients or state rep- resentatives, this may inadvertently occur through the use of the “buzzwords” that overstate the agency’s use of em- pirically supported protocols. In this example, the agency representative may or may not have been aware of the im- portant features of research necessary to justify the EST claim, but the fact was that empirically supported protocols were not in place and a claim of providing ESTs was, at best, a stretch.

Why is the continued misuse of the buzzwords (e.g., “EST”) a serious concern? The concern is simple: State of- ficials, program administrators, and the public may be led to draw inaccurate conclusions. By claiming use of an empiri- cally supported protocol, and then not actually implementing such, any less than favorable outcomes will likely be mis- taken as evidence of the ineffectiveness of the protocol. Jump ahead a decade. It will be less than an advancement to hear statements such as, “We tried that protocol or that EST and the outcomes were no better,” when in fact the claim that the protocol‐driven EST was implemented and the conclusion of limited effectiveness are both inaccurate.

6.2 | Forces that may get empirically supported protocols implemented in the real world? In the case of mental health treatment for youth, parents often seek services for their children, whether spurred by recommen- dations from school personnel, mental health professionals, or

Th is

d oc

um en

t i s c

op yr

ig ht

ed b

y th

e A

m er

ic an

P sy

ch ol

og ic

al A

ss oc

ia tio

n or

o ne

o f i

ts a

lli ed

p ub

lis he

rs .

Th is

a rti

cl e

is in

te nd

ed so

le ly

fo r t

he p

er so

na l u

se o

f t he

in di

vi du

al u

se r a

nd is

n ot

to b

e di

ss em

in at

ed b

ro ad

ly .

| 9 of 12KENDALL AND FRANK

their own identification of need. Thus, perhaps parents are the best entry point for the promotion of protocol‐driven ESTs (i.e., direct‐to‐consumer marketing; Becker, 2015). If a par- ent is informed about the effectiveness of a protocol and then seeks a professional trained and competent in providing it, chances are greater that the services provided will have fidel- ity to the protocol. Even better, perhaps, would be if parents were sufficiently informed about the specifics of the protocol‐ driven EST so that they would ask their provider if the core ingredients are being done. Not unlike insurance companies who pushed for more time‐limited and effective treatments, and the related increase in attention to ESTs, parents may be the source for an effective push for professionals to provide protocol‐driven ESTs with youth.

Training programs are an alternate and potentially effec- tive approach. If and when graduate training programs pro- vide ample and sufficient training in ESTs, graduates of these programs (i.e., future providers) will be able to implement the ESTs. One could argue that this process is slow (it would take decades to produce a community of therapists). However, the greatest magnitude of change in treatments, with regard to the development of ESTs, may have been accomplished in the past four decades. Some improvements may lie ahead, but a large portion of the “improvement variance” may have been accomplished. To the degree that this is accurate, the amount of change in ESTs may not be so great that the decades it takes to produce trained service providers would not be worthwhile.

6.3 | Realistic expectations Even when ESTs are not provided, mental health services are not universally bad or without beneficial outcomes. In general, mental health service efforts have beneficial re- sults. For example, ample percentages of youth with anxi- ety benefit from placebo, from non‐protocol treatments, and from treatments not yet well evaluated. What is sought by encouraging the implementation of protocol‐driven treat- ments is an improvement in the percentage of outcomes that are favorable. Going from 20% to 40% and to 60% is a worthwhile effort and goal. But even when a 60% positive treatment response is achieved following implementation of a protocol‐driven EST, not everyone receiving the protocol improves to the degree that is sought, and not all gains are maintained. Implementing a protocol‐driven EST can be said to maximize the likelihood of the higher percentage of improvement. Not perfect, not everyone, but better than the existing alternatives.

7 | CONCLUSIONS

Evidence‐based protocols can be effective in a vari- ety of settings for a variety of clients, and they can be

implemented flexibly and with fidelity. The critiques that protocols are rigid and overly directive are dated but nev- ertheless highlight the importance of flexibility in training and implementation—whether for therapists in graduate school or seasoned therapists attending continuing educa- tion courses. When protocols are implemented properly and successfully, the public perception of psychological services will be improved; nothing speaks louder than success. Improved outcomes will continue to increase the demand for empirically supported protocols among con- sumers, which will spur therapists’ desire to receive train- ing and to implement such interventions.

CONFLICT OF INTEREST

Dr. Philip C. Kendall receives royalties from the sales of ma- terials related to the treatment of anxiety in youth.

ORCID

Philip C. Kendall https://orcid.org/0000-0001-7034-6961 Hannah E. Frank http://orcid.org/0000-0003-2396-4585

REFERENCES

Addis, M. E., Wade, W. A., & Hatgis, C. (1999). Barriers to dissemina- tion of evidence‐based practices: Addressing practitioners' concerns about manual‐based psychotherapies. Clinical Psychology: Science and Practice, 6(4), 430–441. https://doi.org/10.1093/clipsy.6.4.430

Ale, C. M., McCarthy, D. M., Rothschild, L. M., & Whiteside, S. P. H. (2015). Components of cognitive behavioral therapy related to outcome in childhood anxiety disorders. Clinical Child and Family Psychology Review, 18(3), 240–251. https://doi.org/10.1007/ s10567-015-0184-8

Alegría, M., Chatterji, P., Wells, K., Cao, Z., Chen, C.‐N., Takeuchi, D., … Meng, X.‐L. (2008). Disparity in depression treatment among racial and ethnic minority populations in the United States. Psychiatric Services, 59(11), 1264–1272. https://doi.org/10.1176/ appi.ps.59.11.1264

APA Presidential Task Force on Evidence‐Based Practice (2006). Evidence‐based practice in psychology. American Psychologist, 61(4), 271–285. https://doi.org/10.1037/0003-066X.61.4.271

APA Task Force on Evidence‐Based Practice with Children and Adolescents (2008). Disseminating Evidence‐Based Practice For Children & Adolescents: A systems approach to enhancing care. Washington, DC.

Barlow, D. H., Levitt, J. T., & Bufka, L. F. (1999). The dissemination of empirically supported treatments: A view to the future. Behaviour Research and Therapy, 37, S147–S162. https://doi.org/10.1016/ S0005-7967(99)00054-6

Baumann, A. A., Powell, B. J., Kohl, P. L., Tabak, R. G., Penalba, V., Proctor, E. K., … Cabassa, L. J. (2015). Cultural adaptation and implementation of evidence‐based parent‐training: A sys- tematic review and critique of guiding evidence. Children and Youth Services Review, 53, 113–120. https://doi.org/10.1016/j. childyouth.2015.03.025

Th is

d oc

um en

t i s c

op yr

ig ht

ed b

y th

e A

m er

ic an

P sy

ch ol

og ic

al A

ss oc

ia tio

n or

o ne

o f i

ts a

lli ed

p ub

lis he

rs .

Th is

a rti

cl e

is in

te nd

ed so

le ly

fo r t

he p

er so

na l u

se o

f t he

in di

vi du

al u

se r a

nd is

n ot

to b

e di

ss em

in at

ed b

ro ad

ly .

10 of 12 | KENDALL AND FRANK

Bearman, S., Weisz, J., Chorpita, B., Hoagwood, K., Ward, A., Ugueto, A., & Bernstein, A. (2013). More practice, less preach? The role of supervision processes and therapist characteristics in EBP imple- mentation. Administration and Policy in Mental Health and Mental Health Services Research, 40(6), 518–529. https://doi.org/10.1007/ s10488-013-0485-5

Becker, S. J. (2015). Direct‐to‐consumer marketing: A complemen- tary approach to traditional dissemination and implementation ef- forts for mental health and substance abuse interventions. Clinical Psychology: Science and Practice, 22(1), 85–100. https://doi. org/10.1111/cpsp.12086

Beidas, R. S., Edmunds, J. M., Marcus, S. C., & Kendall, P. C. (2012). Training and consultation to promote implementation of an empiri- cally supported treatment: A randomized trial. Psychiatric Services, 63(7), 660–665. https://doi.org/10.1176/appi.ps.201100401

Bernal, G., Bonilla, J., & Bellido, C. (1995). Ecological validity and cul- tural sensitivity for outcome research: Issues for the cultural adap- tation and development of psychosocial treatments with Hispanics. Journal of Abnormal Child Psychology, 23(1), 67–82. https://doi. org/10.1007/BF01447045

Bjorgvinsson, T., Kertz, S. J., Bigda‐Peyton, J. S., Rosmarin, D. H., Aderka, I. M., & Neuhaus, E. C. (2014). Effectiveness of cognitive behavior therapy for severe mood disorders in an acute psychiat- ric naturalistic setting: A benchmarking study. Cognitive Behavior Therapy, 43(3), 209–220. https://doi.org/10.1080/16506073.2014.9 01988

Butler, A. C., Chapman, J. E., Forman, E. M., & Beck, A. T. (2006). The empirical status of cognitive‐behavioral therapy: A review of meta‐analyses. Clinical Psychology Review, 26(1), 17–31. https:// doi.org/10.1016/j.cpr.2005.07.003

Castonguay, L. G., Goldfried, M. R., Wiser, S., Raue, P. J., & Hayes, A. M. (1996). Predicting the effect of cognitive therapy for de- pression: A study of unique and common factors. Journal of Consulting and Clinical Psychology, 64(3), 497–504. https://doi. org/10.1037/0022-006X.64.3.497

Chambless, D. L., Baker, M. J., Baucom, D. H., Beutler, L. E., & Calhoun, K. S. (1998). Update on empirically validated therapies, II. The Clinical Psychologist, 51, 3–16. Retrieved from https://www. div12.org/publications

Chambless, D. L., & Hollon, S. D. (1998). Defining empirically sup- ported therapies. Journal of Consulting and Clinical Psychology, 66(1), 7–18. https://doi.org/10.1037/0022-006X.66.1.7

Chorpita, B. F., Daleiden, E. L., Ebesutani, C., Young, J., Becker, K. D., Nakamura, B. J., … Starace, N. (2011). Evidence‐based treatments for children and adolescents: An updated review of indicators of ef- ficacy and effectiveness. Clinical Psychology: Science and Practice, 18(2), 154–172. https://doi.org/10.1111/j.1468-2850.2011.01247.x

Chorpita, B. F., Daleiden, E. L., & Weisz, J. R. (2005). Modularity in the design and application of therapeutic interventions. Applied and Preventive Psychology, 11(3), 141–156. https://doi.org/10.1016/j. appsy.2005.05.002

Clark, D. M. (2004). Developing new treatments: On the interplay between theories, experimental science and clinical innovation. Behaviour Research and Therapy, 42, 1089–1104. https://doi. org/10.1016/j.brat.2004.05.002

Cohen, J. A., & Mannarino, A. P. (2008). Trauma‐focused cog- nitive behavioural therapy for children and parents. Child and Adolescent Mental Health, 13(4), 158–162. https://doi. org/10.1111/j.1475-3588.2008.00502.x

Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health ser- vices research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4(1), 50. https://doi.org/10.1186/1748-5908-4-50

DeRubeis, R. J., & Feeley, M. (1990). Determinants of change in cogni- tive therapy for depression. Cognitive Therapy and Research, 14(5), 469–482. https://doi.org/10.1007/BF01172968

Dimidjian, S., Hollon, S. D., Dobson, K. S., Schmaling, K. B., Kohlenberg, R. J., Addis, M. E., … Jacobson, N. S. (2006). Randomized trial of behavioral activation, cognitive therapy, and antidepressant medication in the acute treatment of adults with major depression. Journal of Consulting and Clinical Psychology, 74(4), 658–670. https://doi.org/10.1037/0022-006x. 74.4.658

Dorsey, S., McLaughlin, K. A., Kerns, S. E. U., Harrison, J. P., Lambert, H. K., Briggs, E. C., … Amaya‐Jackson, L. (2017). Evidence base update for psychosocial treatments for children and adolescents ex- posed to traumatic events. Journal of Clinical Child and Adolescent Psychology, 46(3), 303–330. https://doi.org/10.1080/15374416.201 6.1220309

Dorsey, S., Pullmann, M. D., Kerns, S. E. U., Jungbluth, N., Meza, R., Thompson, K., & Berliner, L. (2017). The juggling act of supervi- sion in community mental health: Implications for supporting evi- dence‐based treatment. Administration and Policy in Mental Health and Mental Health Services Research, 44(6), 838–852. https://doi. org/10.1007/s10488-017-0796-z

Edmunds, J. M., Kendall, P. C., Ringle, V. A., Read, K. L., Brodman, D. M., Pimentel, S. S., & Beidas, R. S. (2013). An examination of behavioral rehearsal during consultation as a predictor of training outcomes. Administration and Policy in Mental Health and Mental Health Services Research, 40(6), 456–466. https://doi.org/10.1007/ s10488-013-0490-8

Evans, S. W., Owens, J. S., & Bunford, N. (2014). Evidence‐based psychosocial treatments for children and adolescents with atten- tion‐deficit/hyperactivity disorder. Journal of Clinical Child and Adolescent Psychology, 43(4), 527–551. https://doi.org/10.1080/15 374416.2013.850700

Farrell, N. R., Kemp, J. J., Blakey, S. M., Meyer, J. M., & Deacon, B. J. (2016). Targeting clinician concerns about exposure therapy: A pilot study comparing standard vs. enhanced training. Behaviour Research and Therapy, 85, 53–59. https://doi.org/10.1016/j. brat.2016.08.011

Foa, E. B., Hembree, E. A., Cahill, S. P., Rauch, S. A. M., Riggs, D. S., Feeny, N. C., & Yadin, E. (2005). Randomized trial of prolonged ex- posure for posttraumatic stress disorder with and without cognitive restructuring: Outcome at academic and community clinics. Journal of Consulting and Clinical Psychology, 73(5), 953–964. https://doi. org/10.1037/0022-006X.73.5.953

Freeman, J., Garcia, A., Frank, H., Benito, K., Conelea, C., Walther, M., & Edmunds, J. (2014). Evidence base update for psychosocial treatments for pediatric obsessive‐compulsive disorder. Journal of Clinical Child and Adolescent Psychology, 43(1), 7–26. https://doi. org/10.1080/15374416.2013.804386

Goldstein, N. E. S., Kemp, K. A., Leff, S. S., & Lochman, J. E. (2012). Guidelines for adapting manualized interventions for new target populations: A step‐wise approach using anger management as a model. Clinical Psychology: Science and Practice, 19(4), 385–401. https://doi.org/10.1111/cpsp.12011

Th is

d oc

um en

t i s c

op yr

ig ht

ed b

y th

e A

m er

ic an

P sy

ch ol

og ic

al A

ss oc

ia tio

n or

o ne

o f i

ts a

lli ed

p ub

lis he

rs .

Th is

a rti

cl e

is in

te nd

ed so

le ly

fo r t

he p

er so

na l u

se o

f t he

in di

vi du

al u

se r a

nd is

n ot

to b

e di

ss em

in at

ed b

ro ad

ly .

| 11 of 12KENDALL AND FRANK

Harned, M. S., Dimeff, L. A., Woodcock, E. A., & Contreras, I. (2013). Predicting adoption of exposure therapy in a randomized controlled dissemination trial. Journal of Anxiety Disorders, 27(8), 754–762. https://doi.org/10.1016/j.janxdis.2013.02.006

Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010). The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology Review, 30(4), 448–466. https://doi.org/10.1016/j. cpr.2010.02.005

Hirshfeld‐Becker, D. R., Masek, B., Henin, A., Blakely, L. R., Pollock‐ Wurman, R. A., McQuade, J., … Biederman, J. (2010). Cognitive behavioral therapy for 4‐ to 7‐year‐old children with anxiety disor- ders: A randomized clinical trial. Journal of Consulting and Clinical Psychology, 78(4), 498–510. https://doi.org/10.1037/a0019055

Hirshfeld‐Becker, D. R., Masek, B., Henin, A., Blakely, L. R., Rettew, D. C., Dufton, L., … Biederman, J. (2008). Cognitive‐behavioral inter- vention with young anxious children. Harvard Review of Psychiatry, 16(2), 113–125. https://doi.org/10.1080/10673220802073956

Hollon, S. D., & Beck, A. T. (2013). Cognitive and cognitive‐behav- ioral therapies. In M. J. Lambert (Ed.), Bergin and Garfield's hand- book of psychotherapy and behavior change (6th ed., pp. 393–432). Majwah, NJ: Wiley.

Kaminski, J. W., & Claussen, A. H. (2017). Evidence base update for psychosocial treatments for disruptive behaviors in children. Journal of Clinical Child and Adolescent Psychology, 46(4), 477– 499. https://doi.org/10.1080/15374416.2017.1310044

Kelly, J. A., Heckman, T. G., Stevenson, L. Y., Williams, P. N., Ertl, T., Hays, R. B., … Neumann, M. S. (2000). Transfer of research‐based HIV prevention interventions to community service providers: Fidelity and adaptation. AIDS Education and Prevention, 12(Suppl 5), 87–98.

Kendall, P. C., Choudhury, M. S., Hudson, J. L., & Webb, A. (2002). The C.A.T. project therapist manual. Ardmore, PA: Workbook Publishing.

Kendall, P. C., Chu, B. C., Gifford, A., Hayes, C., & Nauta, M. H. (1998). Breathing life into a manual. Cognitive and Behavioral Practice, 5, 177–198. https://doi.org/10.1016/S1077-7229(98)80004-7

Kendall, P. C., Comer, J. S., Marker, C. D., Creed, T. A., Puliafico, A. C., Hughes, A. A., … Hudson, J. (2009). In‐session exposure tasks and therapeutic alliance across the treatment of childhood anxiety disorders. Journal of Consulting and Clinical Psychology, 77(3), 517–525. https://doi.org/10.1037/a0013686

Kendall, P. C., Compton, S. N., Walkup, J. T., Birmaher, B., Albano, A. M., Sherrill, J., … Piacentini, J. (2010). Clinical characteristics of anxiety disordered youth. Journal of Anxiety Disorders, 24(3), 360–365. https://doi.org/10.1016/j.janxdis.2010.01.009

Kendall, P. C., Gosch, E., Furr, J. M., & Sood, E. (2008). Flexibility within fidelity. Journal of the American Academy of Child & Adolescent Psychiatry, 47(9), 987–993. https://doi.org/10.1097/ CHI.0b013e31817eed2f

Kendall, P. C., & Hedtke, K. A. (2006). Cognitive‐behavioral therapy for anxious children: Therapist manual (3rd ed.). Ardmore, PA: W. Publishing Ed.

Khanna, M. S., & Kendall, P. C. (2008). Computer‐assisted CBT for child anxiety: The coping cat CD‐ROM. Cognitive and Behavioral Practice, 15(2), 159–165. https://doi.org/10.1016/j. cbpra.2008.02.002

Kolko, D. J., Cohen, J. A., Mannarino, A. P., Baumann, B. L., & Knudsen, K. (2009). Community treatment of child sexual abuse:

A survey of practitioners in the national child traumatic stress network. Administration and Policy in Mental Health and Mental Health Services Research, 36(1), 37–49. https://doi.org/10.1007/ s10488-008-0180-0

Lilienfeld, S. O., Ritschel, L. A., Lynn, S. J., Cautin, R. L., & Latzman, R. D. (2013). Why many clinical psychologists are resistant to evidence‐based practice: Root causes and constructive reme- dies. Clinical Psychology Review, 33(7), 883–900. https://doi. org/10.1016/j.cpr.2012.09.008

Marker, C. D., Comer, J. S., Abramova, V., & Kendall, P. C. (2013). The reciprocal relationship between alliance and symptom improvement across the treatment of childhood anxiety. Journal of Clinical Child & Adolescent Psychology, 42(1), 22–33. https://doi.org/10.1080/15 374416.2012.723261

Marrast, L., Himmelstein, D. U., & Woolhandler, S. (2016). Racial and ethnic disparities in mental health care for children and young adults: A national study. International Journal of Health Services, 46(4), 810–824. https://doi.org/10.1177/0020731416662736

McLeod, B. D. (2011). Relation of the alliance with outcomes in youth psychotherapy: A meta‐analysis. Clinical Psychology Review, 31(4), 603–616. https://doi.org/10.1016/j.cpr.2011.02.001

McLeod, B. D., Jensen‐Doss, A., Tully, C., Southam‐Gerow, M., Weisz, J. R., & Kendall, P. C. (2016). The role of setting versus treat- ment type in alliance within youth therapy. Journal of Consulting and Clinical Psychology, 84, 453–464. https://doi.org/10.1037/ ccp0000081

Miller, W. R., Yahne, C. E., Moyers, T. B., Martinez, J., & Pirritano, M. (2004). A randomized trial of methods to help clinicians learn motivational interviewing. Journal of Consulting and Clinical Psychology, 72(6), 1050–1062. https://doi.org/10.1037/ 0022-006X.72.6.1050

Murphy, R., & Hutton, P. (2018). Practitioner review: Therapist vari- ability, patient‐reported therapeutic alliance, and clinical outcomes in adolescents undergoing mental health treatment – A system- atic review and meta‐analysis. Journal of Child Psychology and Psychiatry, 59(1), 5–19. https://doi.org/10.1111/jcpp.12767

Ost, L. G., Karlstedt, A., & Widen, S. (2012). The effects of cognitive behavior therapy delivered by students in a psychologist training program: An effectiveness study. Behavior Therapy, 43(1), 160– 173. https://doi.org/10.1016/j.beth.2011.05.001

Peeters, F., Huibers, M., Roelofs, J., van Breukelen, G., Hollon, S. D., Markowitz, J. C., … Arntz, A. (2013). The clinical effectiveness of evidence‐based interventions for depression: A pragmatic trial in routine practice. Journal of Affective Disorders, 145(3), 349–355. https://doi.org/10.1016/j.jad.2012.08.022

Post, P. N., de Beer, H., & Guyatt, G. H. (2013). How to general- ize efficacy results of randomized trials: Recommendations based on a systematic review of possible approaches. Journal of Evaluation in Clinical Practice, 19(4), 638–643. https://doi. org/10.1111/j.1365-2753.2012.01888.x

President's New Freedom Commission on Mental Health (2003). Achieving the promise: Transforming mental healthcare in America. Rockville, MD. Retrieved from https://govinfo.library.unt.edu/men- talhealthcommission/reports/FinalReport/downloads/FinalReport. pdf

Ruzek, J. I., Eftekhari, A., Crowley, J., Kuhn, E., Karlin, B. E., & Rosen, C. S. (2015). Post‐training beliefs, intentions, and use of prolonged exposure therapy by clinicians in the veterans health adminis- tration. Administration and Policy in Mental Health and Mental

Th is

d oc

um en

t i s c

op yr

ig ht

ed b

y th

e A

m er

ic an

P sy

ch ol

og ic

al A

ss oc

ia tio

n or

o ne

o f i

ts a

lli ed

p ub

lis he

rs .

Th is

a rti

cl e

is in

te nd

ed so

le ly

fo r t

he p

er so

na l u

se o

f t he

in di

vi du

al u

se r a

nd is

n ot

to b

e di

ss em

in at

ed b

ro ad

ly .

12 of 12 | KENDALL AND FRANK

Health Services Research, 44(1), 123–132. https://doi.org/10.1007/ s10488-015-0689-y

Schindler, A. C., Hiller, W., & Witthoft, M. (2011). Benchmarking of cognitive‐behavioral therapy for depression in efficacy and effec- tiveness studies–how do exclusion criteria affect treatment out- come? Psychotherapy Research, 21(6), 644–657. https://doi.org/10. 1080/10503307.2011.602750

Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam‐Gerow, M. A. (2011). Toward the effective and efficient measurement of implementation fidel- ity. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 32–43. https://doi.org/10.1007/ s10488-010-0321-0

Schoenwald, S. K., & Hoagwood, K. (2001). Effectiveness, transport- ability, and dissemination of Interventions: What Matters When? Psychiatric Services, 52(9), 1190–1197. https://doi.org/10.1176/ appi.ps.52.9.1190

Schoenwald, S. K., Mehta, T. G., Frazier, S. L., & Shernoff, E. S. (2013). Clinical Supervision in effectiveness and implementation research. Clinical Psychology: Science and Practice, 20(1), 44–59. https:// doi.org/10.1111/cpsp.12022

Shafran, R., Clark, D. M., Fairburn, C. G., Arntz, A., Barlow, D. H., Ehlers, A., … Wilson, G. T. (2009). Mind the gap: Improving the dissemination of CBT. Behaviour Research and Therapy, 47(11), 902–909. https://doi.org/10.1016/j.brat.2009.07.003

Shirk, S. R., & Karver, M. (2003). Prediction of treatment outcome from relationship variables in child and adolescent therapy: A meta‐ana- lytic review. Journal of Consulting and Clinical Psychology, 71(3), 452–464. https://doi.org/10.1037/0022-006x.71.3.452

Shirk, S. R., Karver, M. S., & Brown, R. (2011). The alliance in child and adolescent psychotherapy. Psychotherapy, 48(1), 17–24. https:// doi.org/10.1037/a0022181

Smith, M. M., McLeod, B. D., Southam‐Gerow, M. A., Jensen‐Doss, A., Kendall, P. C., & Weisz, J. R. (2017). Does the delivery of CBT for youth anxiety differ across research and practice set- tings? Behavior Therapy, 48(4), 501–516. https://doi.org/10.1016/j. beth.2016.07.004

Southam‐Gerow, M. A., & Prinstein, M. J. (2014). Evidence base updates: The evolution of the evaluation of psychological treat- ments for children and adolescents. Journal of Clinical Child and Adolescent Psychology, 43(1), 1–6. https://doi.org/10.1080/153744 16.2013.855128

Southam‐Gerow, M. A., Weisz, J. R., Chu, B. C., McLeod, B. D., Gordis, E. B., & Connor‐Smith, J. K. (2010). Does cognitive be- havioral therapy for youth anxiety outperform usual care in commu- nity clinics? An initial effectiveness test. Journal of the American Academy of Child and Adolescent Psychiatry, 49, 1043–1052. https://doi.org/10.1016/j.jaac.2010.06.009

Stirman, S. W., Derubeis, R. J., Crits‐Christoph, P., & Rothman, A. (2005). Can the randomized controlled trial literature generalize to nonran- domized patients? Journal of Consulting and Clinical Psychology, 73(1), 127–135. https://doi.org/10.1037/0022-006x.73.1.127

Stirman, S. W., Gamarra, J. M., Bartlett, B. A., Calloway, A., & Gutner, C. A. (2017). Empirical examinations of modifications and adap- tations to evidence‐based psychotherapies: Methodologies, impact, and future directions. Clinical Psychology: Science and Practice, 24(4), 396–420. https://doi.org/10.1111/cpsp.12218

Stirman, S. W., Gutner, C. A., Crits‐Christoph, P., Edmunds, J., Evans, A. C., & Beidas, R. S. (2015). Relationships between clinician‐level attributes and fidelity‐consistent and fidelity‐inconsistent modifica- tions to an evidence‐based psychotherapy. Implementation Science, 10, 115. https://doi.org/10.1186/s13012-015-0308-z

Stirman, S. W., Miller, C. J., Toder, K., & Calloway, A. (2013). Development of a framework and coding system for modifications and adaptations of evidence‐based interventions. Implementation Science, 8, 65. https://doi.org/10.1186/1748-5908-8-65

Villabø, M. A., Narayanan, M., Compton, S. N., Kendall, P. C., & Neumer, S.‐P. (2018). Cognitive–behavioral therapy for youth anx- iety: An effectiveness evaluation in community practice. Journal of Consulting and Clinical Psychology, 86(9), 751–764. https://doi. org/10.1037/ccp0000326

Walkup, J. T., Albano, A. M., Piacentini, J., Birmaher, B., Compton, S. N., Sherrill, J. T., … Kendall, P. C. (2008). Cognitive behavioral therapy, sertraline, or a combination in childhood anxiety. New England Journal of Medicine, 359(26), 2753–2766. https://doi. org/10.1056/NEJMoa0804633

Weisz, J. R., Chorpita, B. F., Palinkas, L. A., Schoenwald, S. K., Miranda, J., Bearman, S. K., … Gibbons, R. D. (2012). Testing stan- dard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A randomized effectiveness trial. Archives of General Psychiatry, 69(3), 274–282. https://doi. org/10.1001/archgenpsychiatry.2011.147

Westbrook, D., & Kirk, J. (2005). The clinical effectiveness of cognitive behaviour therapy: Outcome for a large sample of adults treated in routine practice. Behavior Research and Therapy, 43(10), 1243– 1261. https://doi.org/10.1016/j.brat.2004.09.006

Wilson, G. T. (2007). Manual‐based treatment: Evolution and evaluation. In T. A. Treat, R. R. Bootzin, & T. B. Baker (Eds.), Psychological clinical science: Papers in honor of Richard M. McFall (pp. 105– 123). Mahwah, NJ: Lawrence Erlbaum.

Zakarin, E. B., & Albano, A. M. (2016). The launching emerging adults program: Overview of the cognitive behavioral and developmen- tal launching emerging adults program model and implementation in a faculty practice. Journal of the American Academy of Child & Adolescent Psychiatry, 55(10), S72. https://doi.org/10.1016/j. jaac.2016.07.721

How to cite this article: Kendall PC, Frank HE. Implementing evidence‐based treatment protocols: Flexibility within fidelity. Clin Psychol Sci Pract. 2018;25:e12271. https://doi.org/10.1111/cpsp.12271

Th is

d oc

um en

t i s c

op yr

ig ht

ed b

y th

e A

m er

ic an

P sy

ch ol

og ic

al A

ss oc

ia tio

n or

o ne

o f i

ts a

lli ed

p ub

lis he

rs .

Th is

a rti

cl e

is in

te nd

ed so

le ly

fo r t

he p

er so

na l u

se o

f t he

in di

vi du

al u

se r a

nd is

n ot

to b

e di

ss em

in at

ed b

ro ad

ly .