Week 2 Assignment
Training Therapists in Evidence-Based Practice: A Critical
Review of Studies From a Systems-Contextual Perspective
Rinad S. Beidas and Philip C. Kendall, Department of Psychology, Temple University
Evidence-based practice (EBP), a preferred psychologi-
cal treatment approach, requires training of community
providers. The systems-contextual (SC) perspective, a
model for dissemination and implementation efforts,
underscores the importance of the therapist, client, and
organizational variables that influence training and con-
sequent therapist uptake and adoption of EBP. This
review critiques the extant research on training in EBP
from an SC perspective. Findings suggest that therapist
knowledge improves and attitudinal change occurs fol-
lowing training. However, change in therapist behaviors
(e.g., adherence, competence, and skill) and client out-
comes only occurs when training interventions address
each level of the SC model and include active learning.
Limitations as well as areas for future research are
discussed.
Key words: dissemination and implementation, evi-
dence-based practice, systems-contextual perspective,
therapist training. [Clin Psychol Sci Prac 17: 1–30, 2010]
The American Psychological Association (APA) and the
American Academy of Child and Adolescent Psychiatry
(AACAP) support the provision of evidence-based prac-
tice (EBP; American Academy of Child and Adolescent
Psychiatry, 2006; American Psychological Association,
2005). However, a report by the United States Surgeon
General (1999) suggests that the majority of clients with
mental illness do not receive EBP. There are obstacles in
the dissemination and implementation (DI) of EBP into
clinical practice (e.g., criticism of treatment manuals,
inadequate training, and unsupportive organizational
climates). Understanding how to best disseminate EBP is
paramount to reducing the gap between research and
practice (Addis & Krasnow, 2000; Hayes, 2002;
Herschell, McNeil, & McNeil, 2004).
Multiple terms have been used interchangeably, and
at times inaccurately, in this area (Kendall & Beidas,
2007). EBP1 as defined by the American Psychological
Association (2005) is ‘‘the integration of the best avail-
able research with clinical expertise.’’ ESTs refer to
psychological interventions that have been evaluated
scientifically (e.g., a randomized controlled trial, RCT)
and satisfy the criteria outlined in Chambless and
Hollon (1998). DI research includes the purposeful
distribution of relevant information and materials to
therapists (i.e., dissemination) and the adoption and
integration of EPB into practice (i.e., implementation;
Lomas, 1993). Our focus is on training as it relates to
DI research: How does training influence therapist
knowledge and behavior (adherence, competence, and
skill), and how does the therapist’s context (organiza-
tional support and client population) influence adop-
tion and implementation of interventions?
To understand training as it relates to DI, a systems-
contextual (SC) approach is warranted. The SC per-
spective recommends that when considering the effects
of training on therapist behavior, contextual factors
such as therapist variables, organizational support, qual-
ity of training program, and client variables (Sanders &
Turner, 2005; Turner & Sanders, 2006) should be
Address correspondence to Rinad S. Beidas, Department of
Psychology, Temple University, Weiss Hall, Philadelphia, PA
19122. E-mail: rbeidas@temple.edu.
This project was supported by the Ruth L. Kirschstein
National Research Service Award (F31 MH083333) awarded
to the first author.
� 2010 American Psychological Association. Published by Wiley Periodicals, Inc., on behalf of the American Psychological Association. All rights reserved. For permissions, please email: permissionsuk@wiley.com 1
examined. Studying the outcomes of training without
addressing contextual variables is insufficient because
training and potential implementations occur within a
system. Effective DI may occur when therapists are
trained appropriately and when the context supports
behavior change (Sanders & Turner, 2005). Such an
approach is holistic and provides an understanding of
how training influences an individual within a system.
Each component of the SC model (i.e., quality of
training, practitioner variables, client variables, and
organizational support) will be operationalized and
expanded upon below.
The quality of training provided to therapists refers to
the availability of training for individuals interested in
implementing an EBP, as well as the content and
method of the training. The current ‘‘gold standard’’ of
training in EBP includes a workshop, a manual, and
clinical supervision (Sholomskas, Syracuse-Siewert,
Rounsaville, Ball, & Nuro, 2005). The quality of train-
ing in EBP is likely to vary based on when therapists
received their graduate training. Therapists trained
before 1995 are unlikely to have had an emphasis on
EBP, while those trained after 1995 have had varied
exposure to EBP during training (Karekla, Lundgren,
& Forsyth, 2004). A 1995 survey of training directors
found that doctoral programs in clinical psychology
covered about 50% of EBP (specifically—ESTs) and
that most internships did not require competence in
EBP (specifically—ESTs; Crits-Christoph, Frank,
Chambless, Brody, & Karp, 1995). Similarly, a survey
of 200 trainees in APA-accredited programs found that
32% had no coursework in EBP (specifically—ESTs)
and over 65% had not read any of the major Task
Force reports regarding EBP (specifically—ESTs).
Those who had read the Task Force reports tended to
be cognitive-behavioral in orientation (Karekla et al.,
2004). These findings suggest that experienced thera-
pists are not likely to have received training in EBP,
and that newer therapists may have received training in
some aspects of EBP.
Importantly in our view, the content and method
(i.e., overall quality) of the training program is crucial
to successful DI. Some suggest that training content is
important and prefer a focus on principles, rather than
teaching the details of manuals (Abramowitz, 2006;
Hayes, 2002; Miller, Yahne, Moyers, Martinez, &
Pirritano, 2004). Miller et al. (2004) noted that the
appropriate training approach includes an emphasis on
principles and the ‘‘underlying spirit’’ of a treatment
rather than a focus on techniques. Further empirical
study is needed to determine the appropriate focus of
training content.
Training method is an important vehicle through
which change in therapist behavior may be achieved.
Current training methods include passively delivered
didactic lectures (e.g., the format of a continuing edu-
cation workshop) despite findings that this type of
instruction has limited effects on behavior change
(El-Tannir, 2002). Active learning, an alternative to
passive learning, is an interactive process that uses
action and reflection. Active learning is useful for skills
that must be employed within a clinical context (Cross,
Matthieu, Cerel, & Knox, 2007) and has been
employed successfully in the Triple P-Positive Parent-
ing Program (Triple P) training program (Sanders &
Turner, 2005). Cross et al. (2007) endorse active learn-
ing methods (modeling, practice opportunities, building
self-efficacy, and interaction among learners), which
may be accomplished through behavioral role-plays.
Including behavioral role-plays as part of training
improved behavior in both clinical and nonclinical
employees in suicide prevention efforts (Cross et al.,
2007; Matthieu, Cross, Batres, Flora, & Knox, 2008).
Change in behavior may be less robust without the
inclusion of behavioral role-plays in training (Wyman
et al., 2008). Further empirical study of the contribu-
tion of active learning to effective DI efforts is needed.
Therapist attributes may be an important aspect of
whether training produces differential learning and sub-
sequent behavior change. Therapist variables include
individual attributes such as clinical experience, theo-
retical orientation, and therapist attitudes towards EBP.
Conflicting evidence exists regarding whether or not
prior clinical experience influences skill acquisition.
One study found that prior general clinical experience
did not influence therapist skill acquisition in cognitive
therapy (CT), but that prior specific clinical experience
in CT facilitated skill acquisition (James, Blackburn,
Milne, & Reichfelt, 2001). Another study comparing
practicing therapists to trainee therapists suggested that
clinical experience did not moderate skill attainment
(DeViva, 2006).
CLINICAL PSYCHOLOGY: SCIENCE AND PRACTICE • V17 N1, MARCH 2010 2
Therapist attitudes towards EBP have been exam-
ined in survey format (e.g., Addis & Krasnow, 2000;
Najavits, Weiss, Shaw, & Dierberger, 2000), with
mixed results. Some suggest that therapists hold favor-
able attitudes towards EBP (Najavits et al., 2000),
whereas others suggest that therapists hold unfavorable
attitudes towards EBP (Addis & Krasnow, 2000). The-
oretical orientation may be an individual difference that
explains these mixed results: In the first study, partici-
pants were mainly identified as cognitive-behavioral,
whereas in the second study, a large variety of theoreti-
cal orientations were surveyed. Another therapist dif-
ference explaining these discrepant findings may be
clinical experience: Those earlier on in their careers
(e.g., predoctoral interns) may hold more favorable
attitudes towards EBP when compared with those who
are more advanced in their careers (e.g., practicing
therapists; Aarons, 2004). Few studies have explicitly
examined therapist attributes and attitudes towards EBP
as predictors of training outcomes, which may inform
who will benefit most from training.
Variables relating to organizational support (e.g., clini-
cal supervision and organizational environment) can
also impact therapist training outcomes. Reading a
manual and attending a workshop may start the transfer
of knowledge (i.e., dissemination), but ongoing super-
vision may be needed for actual therapist behavior
change and skillful implementation (Bazelmans, Prins,
Hoogveld, & Bleijenberg, 2004; Herschell et al., 2004;
Kendall & Southam-Gerow, 1996). Research on evi-
dence-based supervision is sparse (Ellis, Krengel,
Ladany, & Schult, 1996; Holloway & Neufeldt, 1995),
but it is likely that learning occurs and confidence is
built during the supervisory process. Competent super-
visors may be related to therapist adherence and com-
petence (Henggeler, Schoenwald, Liao, Letourneau, &
Edwards, 2002), two therapist behaviors that are crucial
for fidelity to a treatment (Perpepletchikova & Kazdin,
2005). In addition, continued supervision may decrease
therapist psychological barriers to adoption of EBP
(specifically ESTs; Luoma et al., 2007).
The particulars on supervision differ by intervention,
but one feasible model comes from Turner and
Sanders’s (2006) Triple P approach. Supervision
includes updates on Triple P research through newslet-
ters, conferences, a website, and a question and answer
forum. The approach includes self-monitoring and self-
regulation, which allows therapists to direct their own
learning and skill acquisition following training (Sand-
ers & Turner, 2005). Other organizational support
variables important for therapist behavior change
include organizational openness to change and an orga-
nizational structure that supports implementation of
EBP (Zazzali et al., 2008).
Client variables must be accounted for when consid-
ering successful therapist training outcomes. Client
variables include a therapist’s belief that a particular
EBP can be useful for his or her client population
when considering the severity and risk factors of such
clients (Turner & Sanders, 2006). Some believe that
research samples are not representative of community
samples because the selection criteria of RCTs are not
generally inclusive and comorbidity is not highly repre-
sented (Westen, Novotny, & Thompson-Brenner,
2004). However, other researchers (e.g., Stirman,
DeRubeis, Crits-Christoph, & Rothman, 2005) have
demonstrated that these differences may be overstated.
Stirman et al. (2005) mapped charts of individuals seek-
ing treatment under managed care to the criteria of
nearly 100 RCTs and identified that 80% of these indi-
viduals would be eligible for at least one RCT, and the
majority did not have more complex diagnostic profiles
than participants included in RCTs. It is important to
address therapists, concerns that a treatment is viable
for their clients and that a treatment allows for flexible
application (Kendall & Beidas, 2007). Additionally,
given the ultimate goal of DI (i.e., client access to
EBP), client outcomes following therapist training are
key dependent variables that must be examined.
This review examines studies that train therapists in
EBP from an SC perspective as recommended by
Sanders and Turner (2005) and Turner and Sanders
(2006). This perspective is ecological in nature in that
it views DI as occurring through complex bidirectional
processes between the practitioner, practitioner’s envi-
ronment, and quality of the training (Sanders &
Turner, 2005). Additionally, the perspective views the
practitioner as embedded within a broader working
environment that influences practitioner implementa-
tion of innovation (Sanders & Turner, 2005). The
primary aim of this review was to identify training
studies and consider which level of the SC model (i.e.,
TRAINING IN EVIDENCE-BASED PRACTICE: A REVIEW • BEIDAS & KENDALL 3
therapist variables, client variables, organizational
support, and training) was addressed. Secondarily, we
characterize the quality of the training, with an empha-
sis on training method (e.g., active and ⁄ or passive learning strategies) and content (i.e., a focus on
principles or the teaching of a manual session by
session).
METHODS
A systematic literature review for the 18-year period
spanning 1990–2008 was conducted. Published journal
articles, abstracts, and books were identified via Psych-
Info and PubMed. Key words included ‘‘training,’’
‘‘dissemination,’’ ‘‘implementation,’’ ‘‘adherence,’’
‘‘competence,’’ and ‘‘therapists.’’ Additionally, reference
lists of relevant articles guided the identification of
appropriate studies. Studies were included if they trained
service providers (e.g., social workers, psychologists,
physicians, substance abuse counselors, secondary school
staff, master’s-level clinicians, and nurses) in EBP or
ESTs for at-risk or clinical populations.2 Training studies
not affiliated with an evidence base were not included,
nor were trainings in single techniques. All studies
included in this review focused on training in EBP (see
Table 1). Unpublished theses or dissertations were not
included.
We operationalized the four levels of the systems-
contextual approach to DI as follows: (a) therapist vari-
ables: Study included at least one measure of therapist
attitudes and ⁄ or a comprehensive measure of clinical experience and theoretical orientation,3 (b) organiza-
tional support: Study included at least one measure of
organizational characteristics and ⁄ or provided ongoing consultation or supervision, (c) quality of training pro-
cesses:4 Study provided training and described training
sufficiently so that characteristics of training could be
identified, and (d) client variables: Study included at
least one measure of severity, risk factors, and resiliency
Table 1. Evidence-based practices (EBPs) included in this study
EBP References
Classifying body deeming the treatment to be an EBP
Dialectical behavior therapy for borderline personality disorder
Hawkins and Sinha (1998) A
Motivational interviewing for substance abuse Baer et al. (2004); Hunter et al. (2005); Miller and Mount (2001); Miller et al. (2004); Rubel et al. (2000); Schoener et al. (2006); Saitz et al. (2000)
A
Cognitive behavior therapy for eating disorders McVey et al. (2005) A Interpersonal therapy for eating disorders McVey et al. (2005) A Cognitive-behavioral therapy for child anxiety Beidas et al. (2009) A Behavioral therapy for anxiety Gega, Norman, & Marks (2007) A Trauma-focused cognitive-behavioral therapy National Crime Victims Research & Treatment Center
(2007) C
Intervention for youth suicide Chagnon et al. (2007); Cross et al. (2007); Wyman et al. (2008)
B
Cognitive and ⁄ or cognitive-behavioral therapy for substance abuse
Crits-Christoph et al. (1998); Morganstern et al. (2001); Sholomskas et al. (2005); Siqueland et al. (2000); Watkins et al. (2008)
A
Dynamic therapy for substance abuse Crits-Christoph et al. (1998); Siqueland et al. (2000) A Drug counseling for substance abuse Crits-Christoph et al. (1998); Luoma et al. (2007);
Siqueland et al. (2000) B
Triple P parenting program for at-risk youth Sanders, Murphy-Brennan, et al. (2003) B Time-limited dynamic psychotherapy for adults with Axis I
and II disorders Bein et al. (2000); Henry, Schacht, et al. (1993); Henry,
Strupp, et al. (1993) A
Behavioral family therapy for relatives caring for family member with schizophrenia
Brooker and Butterworth (1993); Fadden (1997) A
Cognitive-behavioral therapy for youth and young adults presenting to primary care
Maunder et al. (2008) A
Contingency management for youth substance abuse Henggeler, Chapman, et al. (2008); Henggeler, Sheidow, et al. (2008)
C
Multisystemic therapy for youth substance abuse Henggeler, Sheidow, et al. (2008) C
Note. As cited in A, Chambless & Ollendick (2001); B, Substance Abuse and Mental Health Services Administration (www.nationalregistry.samhsa.gov); C, Special section of Journal of Clinical Child and Adolescent Psychology (Silverman, Pina, & Viswesvaran, 2008).
CLINICAL PSYCHOLOGY: SCIENCE AND PRACTICE • V17 N1, MARCH 2010 4
of therapist’s client population, or included a measure of
client treatment outcome (e.g., diagnostic interview).
We operationalized the training method as employ-
ing passive (e.g., didactic presentation, lectures, and
seminars) and ⁄ or active learning strategies (e.g., behav- ioral role-plays, feedback, coaching, and experiential
exercises). Training content was defined as focusing
on principles of the treatment (i.e., the underlying
foundational principles of the EBP) or teaching the
manual and going through the treatment session by
session.
Outcome measures included both self-reported (e.g.,
therapist attitudes) and independently rated (e.g., thera-
pist adherence) variables. Studies were classified as
using standardized measures if the report indicated ade-
quate psychometric properties (i.e., reliability: intraclass
coefficients or kappa values above 0.60; Landis &
Koch, 1977; adequate validity) and if the measure was
used in more than one research report. Studies were
classified as using nonstandardized investigator-created
measures if psychometric properties were not reported,
were not adequate, or if the measure was only used in
one study. Studies that included the outcome measures
of knowledge,5 adherence, and skill ⁄ competence were emphasized given the importance of these constructs in
the transportation of a treatment (Perpepletchikova &
Kazdin, 2005). Additionally, we note whether gains
made in various outcome measures (i.e., knowledge,
adherence, and skill ⁄ competence) suggest therapist pro- ficiency. An 80% score was used to mark proficiency
in therapist knowledge and behavior. Eighty percent
was used as the cutoff, consistent with training condi-
tions used in other evaluations of EBP as the criterion
to be satisfied prior to being certified to deliver a treat-
ment (e.g., Seng, Prinz, & Sanders, 2006; Sholomaskas
et al., 2005; Walkup et al., 2008).
RESULTS
Many DI studies (32) of training have been reported,
but few (3) assess and measure all domains of the SC
model. Studies in this review are organized as follows:
those that address (a) training, (b) training and organi-
zational support, (c) training and therapist variables, (d)
training and client variables, (e) training, organizational
support, and therapist variables, (f) training, organiza-
tional support, and client variables, and (g) training,
organizational support, therapist variables, and client
variables.6
Training
Description of Studies. Eight studies focused on train-
ing (see Table 2). The following EBPs were studied:
(a) dialectical behavior therapy (DBT) for borderline
personality disorder (Hawkins & Sinha, 1998), (b)
motivational interviewing (MI) for adult substance
abuse (Baer, Rosengren, Dunn, Wells, & Ogle, 2004),
(c) CBT and interpersonal therapy (IPT) for eating dis-
orders (ED) (McVey et al., 2005), (d) CBT for youth
anxiety (Beidas, Barmish, & Kendall, 2009), (e)
trauma-focused CBT (tf-CBT) for traumatized youth
(National Crime Victims Research & Treatment Cen-
ter, 2007), (f) behavior therapy (BT) for anxiety (Gega,
Norman, & Marks, 2007), (g) the Question, Persuade,
Refer (QPR) prevention program for youth suicide
(Cross et al., 2007), and (h) Triple P for externalizing
behaviors in youth (Sanders, Tully, Turner, Maher, &
McAuliffe, 2003). One study used an RCT design
(Gega et al., 2007), whereas the other studies used a
nonrandomized, convenience-sample, pre–post quasi-
experimental design. Study sample sizes ranged from 20
to 4,387 participants and educational level ranged from
bachelor’s to postgraduate degrees (e.g., MD, PhD, and
PsyD). Most participants were community mental
health providers.
Training Method and Content. One study exclusively
included passive learning (e.g., didactic presentation;
Gega et al., 2007), whereas four included both passive
and active learning (e.g., experiential learning, inter-
active computer exercises, role-plays; Baer et al., 2004;
Cross et al., 2007; National Crime Victims Research &
Treatment Center, 2007; Sanders, Tully, et al., 2003).
Two studies did not provide sufficient description of
the training to identify training method (Hawkins &
Sinha, 1998; McVey et al., 2005). When sufficiently
described, training content included a focus on review-
ing each session of the treatment protocol (Beidas
et al., 2009; National Crime Victims Research &
Treatment Center, 2007; Sanders, Tully, et al., 2003)
or a focus on principles and ⁄ or the underlying spirit of the treatment (Baer et al., 2004; Gega et al., 2007).
Training duration varied widely from one hour (Cross
TRAINING IN EVIDENCE-BASED PRACTICE: A REVIEW • BEIDAS & KENDALL 5
T ab
le 2 .
Sy st
em s-
co n te
xt u al
le ve
l: T ra
in in
g
R ef
er en
ce s
n ES
T P at
ie n t
T h er
ap is
t T ra
in in
g A
ct iv
e el
em en
ts O
u tc
o m
e va
ri ab
le s
D es
ig n
R es
u lt
s
B ae
r et
al .
(2 0 0 4 )
2 2
M I
A d u lt
su b st
. u se
A d d ic
ti o n
an d
m en
ta l h ea
lt h
cl in
ic ia
n s
2 -d
ay 1 4 -h
o u r
w o rk
sh o p
w it h
b ri ef
d id
ac ti cs
an d
ex p er
ie n ti al
ac ti vi
ti es
Ex p er
ie n ti al
ac ti vi
ti es
(s m
al l
g ro
u p
p ra
ct ic
e)
1 .
H R
Q 2 .
co d ed
M IS
C (O
Q ,
C R
, R
:Q ,
M I-
C b eh
av io
r, M
I- In
c b eh
av io
r)
C o n ve
n ie
n ce
sa m
p le
; p re
–p o st
as se
ss m
en t;
n o
co n tr
o l
g ro
u p ;
n o
R A
; 2 -m
o n th
f ⁄u
; SM
1 .
In cr
ea se
in H
R Q
+ R
:Q ,
O Q
, M
I- C
at p re
–p o st
2 .
In cr
ea se
in H
R Q
+ R
:Q m
ai n ta
in ed
at f
⁄u B ei
d as
et al
. (2
0 0 9 )
2 0
C B T
C h ild
an xi
et y
P sy
D tr
ai n ee
s 2 -h
o u r
d id
ac ti c
w o rk
sh o p
p lu
s m
an u al
N o n e
1 .
R at
ed A
2 .
R at
ed S
3 .
K
C o n ve
n ie
n ce
sa m
p le
; p re
–p o st
as se
ss m
en t;
n o
co n tr
o l
g ro
u p ;
n o
R A
; n o
f ⁄u
; IC
M
1 .
In cr
ea se
in K
p re
–p o st
as se
ss m
en t
2 .
N o
in cr
ea se
in A
, o r
S C
ro ss
et al
. (2
0 0 7 )
7 6
Q P R
Y o u th
su ic
id e
p re
ve n ti o n
N o n cl
in ic
al em
p lo
ye es
1 -h
o u r
w o rk
sh o p
p lu
s ro
le -p
la y
Ex p er
ie n ti al
b eh
av io
ra l ro
le -
p la
y
1 .
d ec
K 2 .
p er
K 3 .
se lf -e
ffi ca
cy 4 .
Sa 5 .
d if fu
si o n
o f
K 6 .
ra te
d S
C o n ve
n ie
n ce
sa m
p le
; p re
–p o st
as se
ss m
en t;
n o
co n tr
o l
g ro
u p ;
n o
R A
; 6 -w
ee k
f ⁄u
; IC
M
1 .
d ec
K ,
p er
K ,
in cr
ea se
d p re
– p o st
. 2 .
h ig
h Sa
at p o st
as se
ss m
en t.
3 .
H ig
h d if fu
si o n
at p o st
as se
ss m
en t
4 .
S ra
n g ed
b et
w ee
n 1 1 .7
9 ⁄1
5 at
p o st
as se
ss m
en t
an d
1 1 .4
3 ⁄1
5 at
f ⁄u
(n o
si g .
d if f)
. G
eg a,
N o rm
an &
M ar
ks (2
0 0 7 )
9 2
B T
P ri m
. ca
re N
u rs
in g
st u d en
ts 4 -h
o u r
co m
p u te
r tr
ai n in
g o r
4 -h
o u r
w o rk
sh o p
N o n e
1 .
K 2 .
ra te
d S
3 .
Sa
R an
d o m
iz ed
co n tr
o lle
d tr
ia l
w it h
cr o ss
o ve
r; n o
f ⁄u
; IC
M
1 .
B o th
co n d it io
n s
im p ro
ve d
K ,
ra te
d S,
an d
Sa eq
u al
ly w
h en
p ro
vi d ed
al o n e.
N o
ad d ed
b en
efi t
to d o in
g b o th
tr ai
n in
g s.
H aw
ki n s
an d
Si n h a
(1 9 9 8 )
1 0 9
D B T
B P D
C o m
m u n it y
cl in
ic ia
n s
N o t
d es
cr ib
ed N
o t
d es
cr ib
ed 1 .
K C
o n ve
n ie
n ce
sa m
p le
; p re
–p o st
as se
ss m
en t;
n o
co n tr
o l
g ro
u p ;
n o
R A
; n o
f ⁄u
; IC
M
1 .
K in
cr ea
se d
p re
–p o st
2 .
K p re
d ic
to rs
: st
u d y
g ro
u p
at te
n d an
ce ,
ti m
e ap
p ly
in g
tx ,
re ad
in g ,
p ee
r co
n su
lt M
cV ey
et al
. (2
0 0 5 )
3 3 1 5
C B T
⁄I P T
ED H
ea lt h -c
ar e
p ra
ct it io
n er
s w
it h
p ro
fe ss
io n al
tr ai
n in
g
3 -h
o u r
to 4 -d
ay in
te n si
ve w
o rk
sh o p s
an d
m an
u al
N o t
d es
cr ib
ed 1 .
p er
K 2 .
co m
fo rt
in tr
ea ti n g
o r
te ac
h in
g ab
o u t
ED
C o n ve
n ie
n ce
sa m
p le
; p re
–p o st
as se
ss m
en t;
n o
co n tr
o l
g ro
u p ;
n o
R A
; n o
f ⁄u
; IC
M
1 .
In cr
ea se
in p er
K an
d co
m fo
rt p re
–p o st
CLINICAL PSYCHOLOGY: SCIENCE AND PRACTICE • V17 N1, MARCH 2010 6
et al., 2007) to five days (Hawkins & Sinha, 1998).
Three studies used manuals to supplement the didactic
presentation (Beidas et al., 2009; McVey et al., 2005;
Sanders, Tully, et al., 2003).
Outcome Measures. The outcome measures for the
majority of the studies were self-reports, with the
exception of independently rated adherence and skill.
Self-reported outcomes included knowledge,
satisfaction, level of comfort with providing treatment,
self-efficacy, training experiences, and diffusion of
knowledge. Two studies (Baer et al., 2004; Sanders,
Tully, et al., 2003) used psychometrically sound stan-
dardized measures, whereas the other studies used
investigator-created nonstandardized measures.
Summary of Studies. Studies using only self-report
questionnaires (e.g., knowledge and satisfaction) were
considered less rigorous than studies that included inde-
pendently rated adherence and ⁄ or skill given the find- ing that knowledge change is often not generalized as
actual behavior change (Miller, Sorensen, Selzer, &
Brigham, 2006). Surveyed mental health professionals
(n = 3,315) reported a statistically significant increase in
perceived knowledge of ED, an increase in level of
comfort in treating ED, and increased comfort teaching
others about ED treatment after receiving differential
doses of training (McVey et al., 2005). Training varied
across participants, ranging from three hours to four
days, and knowledge was not objectively measured.
Another study objectively measured declarative
knowledge and concluded that community mental
health providers (n = 109) were able to reach profi-
ciency in DBT after receiving unspecified dosages of
training (training varied across participants; Hawkins &
Sinha, 1998). In the group that attended the highest dose
of training (n = 36; a five-day intensive workshop), the
highest percentage of correct answers on a knowledge
test was 67.6. Although the authors conclude that partic-
ipants learned the protocol, they did not achieve profi-
ciency according to the 80% criterion. Another study
objectively measuring knowledge found that a multime-
dia web-based distance learning program for mental
health professionals seeking training in tf-CBT signifi-
cantly increased participants’ knowledge (National
Crime Victims Research & Treatment Center, 2007)Ta b le
2 .
(C o n ti
n u e d )
R ef
er en
ce s
n ES
T P at
ie n t
T h er
ap is
t T ra
in in
g A
ct iv
e el
em en
ts O
u tc
o m
e va
ri ab
le s
D es
ig n
R es
u lt
s
N at
io n al
C ri m
e V
ic ti m
s R
es ea
rc h
& T re
at m
en t
C en
te r
(2 0 0 7 )
4 ,3
8 7
tf -C
B T
Y o u th
tr au
m a
M en
ta l h ea
lt h
p ro
fe ss
io n al
s 1 0 -h
o u r
w eb
-b as
ed se
m in
ar In
te ra
ct iv
e co
m p u te
r p ro
g ra
m (e
.g .,
st re
am in
g vi
d eo
)
1 .
K 2 .
Sa C
o n ve
n ie
n ce
sa m
p le
; p re
–p o st
as se
ss m
en t;
n o
co n tr
o l g ro
u p ;
n o
R A
; n o
f ⁄u
; IC
M
1 .
H ig
h Sa
at p o st
as se
ss m
en t
2 .
Si g .
in cr
ea se
in K
fo r
al l 1 0
m o d u le
s Sa
n d er
s, T u lly
, et
al .
(2 0 0 3 )
3 2
T ri p le
P Y
o u th
b eh
av io
r p ro
b le
m s
G en
er al
p h ys
ic ia
n s
W o rk
sh o p
R o le
-p la
y an
d fe
ed b ac
k 1 .
S 2 .
Sa 3 .
C o n fi d en
ce
C o n ve
n ie
n ce
sa m
p le
; p re
–p o st
as se
ss m
en t;
w ai
tl is
t co
m p ar
is o n ;
SM
1 .
H ig
h er
S in
tr ai
n ed
g ro
u p
2 .
H ig
h er
sa ti sf
ac ti o n
3 .
H ig
h er
co n fi d en
ce
N o te
. M
I, m
o ti va
ti o n al
in te
rv ie
w in
g ;
H R
Q ,
H el
p fu
l R
es p o n se
s Q
u es
ti o n n ai
re ;
M IS
C ,
M o ti va
ti o n al
In te
rv ie
w in
g Sk
ill s
C o d e
(O Q
, o p en
q u es
ti o n s;
C R
, co
m p le
x re
fl ec
ti o n s;
R :Q
, ra
ti o
o f
re fl ec
ti o n s
to q u es
ti o n s;
M I-
C ,
m o ti va
ti o n al
in te
rv ie
w in
g co
n si
st en
t; M
I- In
c, m
o ti va
ti o n al
in te
rv ie
w in
g in
co n si
st en
t) ;
R A
, ra
n d o m
as si
g n m
en t;
f ⁄u
, fo
llo w
-u p
as se
ss m
en t;
SM ,
st an
d ar
d iz
ed m
ea su
re s;
C B T ,
co g n it iv
e- b eh
a vi
o ra
l th
er ap
y; A
, ad
h er
en ce
; S,
sk ill
; K
, kn
o w
le d g e;
IC M
, in
ve st
ig at
o r-
cr ea
te d
m ea
su re
s; Q
P R
, q u es
ti o n ,
p er
su ad
e, re
fe r;
d ec
K ,
d ec
la ra
ti ve
kn o w
le d g e;
p er
K ,
p er
ce iv
ed kn
o w
le d g e;
Sa ,
sa ti sf
ac ti o n ;
B T ,
b eh
av io
r th
er ap
y; D
B T ,
d ia
le ct
ic al
b eh
av io
r th
er ap
y; B P D
, b o rd
er lin
e p er
so n al
it y
d is
o rd
er ;
tx ,
tr ea
tm en
t; IP
T ,
in te
rp er
so n al
th er
ap y;
ED ,
ea ti n g
d is
o rd
er s;
tf -C
B T ,
tr au
m a-
fo cu
se d
C B T .
TRAINING IN EVIDENCE-BASED PRACTICE: A REVIEW • BEIDAS & KENDALL 7
from 61.9% to 82.1%, which suggests that participants
reached proficiency in their knowledge of tf-CBT.
Five studies measured self-reported knowledge and
independently rated therapist behavior. Following a
one-hour training workshop for nonclinical employees
(n = 76) in the QPR suicide prevention program
(Cross et al., 2007), participants reported satisfaction
with training, and significant increases in both per-
ceived knowledge (from 36% to 62%) and declarative
knowledge (from 69% to 85%). A subset of participants
(n = 26) completed a behavioral role-play from which
skill was coded. Fifty-five percent of participants
reached a satisfactory skill level (a score of 12 of 15),
whereas 45% did not.
Similar findings were observed after 20 trainees read
a manual and attended a 2.5-hour CBT workshop for
youth anxiety (Beidas et al., 2009). Participants showed
significant increases in knowledge (all reached the 80%
criterion). Following training, participants completed a
behavioral role-play from which adherence and skill
were coded. After training, none of the participants
reached 80% adherence, whereas 67% reached profi-
cient levels of skill in CBT for child anxiety. A similar
study compared the effectiveness of differing training
modalities (i.e., computer training versus a workshop)
on BT for anxiety. Following training, nursing stu-
dents (n = 92) showed comparable improvement in
knowledge, skills, and satisfaction in both conditions.
Note that although knowledge improved significantly
after training, participants were below the proficiency
level (average 71%). Additionally, skill was rated on
average as 4.3 of 8 points (Gega et al., 2006).
In a different treatment modality (i.e., MI), addiction
and mental health counselors (n = 22) received 24 hours
of training. After training, 53% of clinicians were rated
as proficient in their MI skills, and at two-month fol-
low-up, 42% were rated as proficient (Baer et al., 2004).
A study that trained general practitioners in the Triple P
program for externalizing youth found that a brief train-
ing emphasizing active learning produced significant
improvements in rated skill. Clinicians also reported
increased satisfaction and confidence in treatment deliv-
ery (Sanders, Tully, et al., 2003).
Conclusions. Both perceived and declarative knowledge
increase after receiving training in an EBP. Change in
declarative knowledge is especially important given the
implications it may have on treatment delivery. How-
ever, it remains unclear if knowledge gains indicate
proficiency in treatment delivery (e.g., Hawkins &
Sinha, 1998), and further empirical study is necessary.
Of those studies that included independently rated
behavior, proficiency in therapist skill ranged from 54%
to 67% at post-training and 42% at follow-up (Beidas
et al., 2009; Cross et al., 2007; Gega et al., 2006,
Sanders, Tully, et al., 2003). After training, none of the
participants reached proficiency in adherence (see
Beidas et al., 2009). This finding is alarming when
considering that skill and adherence (Perpepletchikova
& Kazdin, 2005) are crucial to the transportation of a
treatment from research to practice.
Limitations. Study-specific limitations include high
attrition rates (Beidas et al., 2009; National Crime Vic-
tims Research & Treatment Center, 2007), practice
effects, low content validity (e.g., only four questions
per section to assess knowledge; National Crime
Victims Research & Treatment Center, 2007), partici-
pants receiving differential training collapsed into one
larger sample (Hawkins & Sinha, 1998; McVey et al.,
2005), the examination of skill in only a subset of the
sample (Cross et al., 2007), and an inability to procure
all outcome variables (Baer et al., 2004).
Training and Organizational Support
Description of Studies. Six studies focused on both
training and organizational support (see Table 3). The
EBP training included (a) CBT for adult substance use
(Sholomskas et al., 2005), (b) CT, dynamic therapy
(DP), and drug counseling (DC) for adult substance
abuse (Crits-Christoph et al., 1998), (c) group drug
counseling (GDC) for adult substance abuse (Luoma
et al., 2007), (d) Triple P for youth behavior problems
(Sanders, Murphy-Brennan, & McAuliffe, 2003), (e)
time-limited dynamic psychotherapy (TLDP) for adults
(Henry, Strupp, Butler, Schacht, & Binder, 1993), and
(f) MI for substance abuse (Moyers et al., 2008). Three
studies used a nonrandomized, convenience-sample,
pre–post, quasi-experimental design (Crits-Christoph
et al., 1998; Henry, Strupp, et al., 1993; Sanders,
Murphy-Brennan, et al., 2003). Two studies randomly
assigned participants to a training condition and included
CLINICAL PSYCHOLOGY: SCIENCE AND PRACTICE • V17 N1, MARCH 2010 8
T ab
le 3 .
Sy st
em s-
co n te
xt u al
le ve
l: T ra
in in
g an
d o rg
an iz
at io
n al
su p p o rt
R ef
er en
ce s
n ES
T P at
ie n t
T h er
ap is
t T ra
in in
g A
ct iv
e el
em en
ts O
u tc
o m
e va
ri ab
le s
D es
ig n
R es
u lt
s
C ri ts
-C h ri st
o p h
et al
. (1
9 9 8 )
6 5
C T ,
D P ,
D C
A d u lt
su b st
. u se
T h er
ap is
ts an
d co
u n se
lo rs
M an
u al
+ 4
tw o -
d ay
w o rk
sh o p s
(d id
ac ti c,
ro le
- p la
y d is
cu ss
io n )
+ su
p er
vi si
o n
R o le
-p la
ys 1 .
S C
o n ve
n ie
n ce
sa m
p le
; p re
–p o st
as se
ss m
en t;
SM ;
n o
co n tr
o l g ro
u p ;
n o
f ⁄u
; n o
R A
1 .
C T
sh o w
ed p er
tr ai
n in
g ca
se (a
cr o ss
ca se
). 2 .
D C
an d
D P
sh o w
ed le
ar n in
g w
it h in
ca se
s b u t
n o t
ac ro
ss .
H en
ry ,
St ru
p p ,
et al
. (1
9 9 3 )
1 6
T LD
P A
d u lt s
A xi
s I
an d
A xi
s II
P sy
ch ia
tr is
ts an
d p sy
ch o lo
g is
ts
5 0
w ee
kl y
2 -h
o u r
w o rk
sh o p s
(d id
ac ti c)
, m
an u al
, su
p er
vi si
o n
N o n e
1 .
S C
o n ve
n ie
n ce
sa m
p le
; p re
–p o st
as se
ss m
en t;
SM ;
n o
co n tr
o l g ro
u p ;
n o
f ⁄u
; n o
R A
1 .
So m
e S
in cr
ea se
d an
d so
m e
d ec
re as
ed
Lu o m
a et
al .
(2 0 0 7 )
3 0
G D
C A
d u lt
su b st
. u se
C o m
m u n it y
cl in
ic ia
n s
6 -h
o u r
w o rk
sh o p
(d id
ac ti cs
, q
⁄a ,
ro le
- p la
ys )
+ ei
g h t
1 .5
-h o u r
co n su
lt at
io n s
fr o m
an A
C T
an d
re la
p se
p re
ve n ti o n
m o d el
R o le
-p la
ys 1 .
Sa 2 .
Se lf -r
ep o rt
ed ad
o p ti o n
3 .
B u rn
o u t
P re
–p o st
as se
ss m
en t;
2 -
to 4 -m
o n th
f ⁄u
; R
A ;
co n tr
o l
g ro
u p ;
IC M
an d
SM
1 .
Sa h ig
h in
b o th
g ro
u p s
2 .
A d o p ti o n
an d
p er
so n al
ac co
m p lis
h m
en t
h ig
h er
in in
te rv
en ti o n
M o ye
rs et
al .
(2 0 0 8 )
1 2 9
M I
A d u lt
su b st
. u se
B eh
av io
ra l
h ea
lt h
p ro
vi d er
s
Se lf -d
ir ec
te d
tr ai
n in
g vs
. w
o rk
sh o p
vs .
w o rk
sh o p
+ tr
ai n in
g en
ri ch
m en
t
V ar
ie d
p er
tr ai
n in
g co
n d it io
n
1 .
M I
co m
p et
en ce
P re
–p o st
as se
ss m
en t;
4 -m
o n th
f ⁄u
; R
A ;
co m
p ar
is o n
g ro
u p ;
SM
1 .
Im p ro
ve d
co m
p et
en ce
in al
l g ro
u p s
2 .
N o
d if fe
re n ce
b et
w ee
n w
o rk
sh o p
an d
w o rk
sh o p
+ tr
ai n in
g en
ri ch
m en
t 3 .
D ec
ay in
co m
p et
en ce
at f
⁄u Sa
n d er
s, M
u rp
h y-
B re
n n an
, et
al .
(2 0 0 3 )
3 3 1
T ri p le
P A
t- ri sk
yo u th
G en
er al
th er
ap is
ts R
ea d in
g ,
2 -d
ay w
o rk
sh o p ,
1 -d
ay ac
cr ed
it at
io n ,
ac ce
ss to
w eb
su p p o rt
n et
w o rk
A ct
iv e
sk ill
s tr
ai n in
g 1 .
P er
ce iv
ed S
2 .
Sa C
o n ve
n ie
n ce
sa m
p le
; p re
–p o st
as se
ss m
en t;
IC M
; n o
co n tr
o l g ro
u p ;
n o
f ⁄u
; n o
R A
1 .
H ig
h Sa
2 .
In cr
ea se
d p er
ce iv
ed S
Sh o lo
m sk
as et
al .
(2 0 0 5 )
7 8
C B T
A d u lt
su b st
. u se
C o m
m u n it y
cl in
ic ia
n s
M an
u al
o n ly
vs .
m an
u al
+ 2 0 -
h o u r
w eb
tr ai
n in
g vs
. 3 -d
ay d id
ac ti c
w o rk
sh o p
+ th
re e
1 -h
o u r
su p er
vi si
o n
W eb
-b as
ed ro
le -p
la ys
an d
in v iv
o ro
le -
p la
ys
1 .
R at
ed A
2 .
R at
ed S
3 .
K
N o t
fu lly
ra n d o m
iz ed
; p re
–p o st
g ro
u p
as se
ss m
en t;
co n tr
o l
g ro
u p ;
n o
f ⁄u
(p o st
-S u p );
SM
1 .
In cr
ea se
d S
an d
A w
o rk
sh o p
+ su
p er
vi si
o n
w h en
co m
p ar
ed w
it h
m an
u al
+ w
eb at
p o st
+ f
⁄u .
2 .
In cr
ea se
d S,
an d
A in
M an
u al
+ w
eb C
o m
p ar
ed w
it h
M an
u al
o n ly
. 3 .
N o
D if fe
re n ce
in K
n o w
le d g e
N o te
. C
T ,
co g n it iv
e th
er ap
y; D
P ,
d yn
am ic
p sy
ch o th
er ap
y; D
C ,
d ru
g co
u n se
lin g ;
S, sk
ill ;
SM ,
st an
d ar
d iz
ed m
ea su
re ;
R A
, ra
n d o m
as si
g n m
en t;
f ⁄u
, fo
llo w
-u p ;
T LD
P ,
ti m
e- lim
it ed
d yn
am ic
p sy
ch o th
er ap
y; G
D C
, g ro
u p
d ru
g co
u n se
lin g ;
q ⁄a
, q u es
ti o n s
an d
an sw
er s;
A C
T ,
A cc
ep ta
n ce
an d
C o m
m it m
en t
T h er
ap y;
Sa ,
sa ti sf
ac ti o n ;
IC M
, in
ve st
ig at
o r-
cr ea
te d
m ea
su re
; M
I, m
o ti va
ti o n al
in te
rv ie
w in
g ;
C B T ,
co g n it iv
e- b eh
av io
ra l
th er
ap y;
A ,
ad h er
en ce
; K
, kn
o w
le d g e;
p o st
-S u p ,
p o st
su p er
vi si
o n .
TRAINING IN EVIDENCE-BASED PRACTICE: A REVIEW • BEIDAS & KENDALL 9
a comparison condition (Luoma et al., 2007; Moyers
et al., 2008), whereas another study attempted random
assignment but was unable to do so (Sholomskas et al.,
2005). Study samples ranged from 16 to 331 therapist
participants, with educational levels ranging from bache-
lor’s to postgraduate degrees (e.g., MD, PhD, and
PsyD). Most were community mental health providers.
Training Method and Content. All training included
passive learning (e.g., didactic presentation). Five studies
included active learning strategies (e.g., experiential
learning, in vivo and ⁄ or computer role-plays, active learning skills; Crits-Christoph et al., 1998; Luoma
et al., 2007; Moyers et al., 2008; Sanders, Murphy-
Brennan, et al., 2003; Sholomskas et al., 2005). Training
content included a focus on reviewing each session of
the treatment protocol (Crits-Christoph et al., 1998;
Sholomskas et al., 2005) or a focus on principles
(Henry, Strupp, et al., 1993; Moyers et al., 2008).
Training time varied from 6 (Luoma et al., 2007) to
100 hours (Henry, Strupp, et al., 1993). Five studies
required the use of manuals to supplement the didactic
presentation (Crits-Christoph et al., 1998; Henry,
Strupp, et al., 1993; Moyers et al., 2008; Sanders,
Murphy-Brennan, et al., 2003; Sholomskas et al., 2005).
Organizational Support. One study provided peer and
web-based support (Sanders, Murphy-Brennan, et al.,
2003), whereas another used group consultation to
overcome therapist barriers in the use of newly
acquired skills (Luoma et al., 2007). Other work
(Crits-Christoph et al., 1998; Henry, Strupp, et al.,
1993; Sholomskas et al., 2005) included supervision as
part of the training, ranging from 3 one-hour supervi-
sions (Sholomskas et al., 2005) to 50 two-hour weekly
supervisions (Henry, Strupp, et al., 1993). One study
provided supervision that concentrated on feedback on
specific techniques and also provided additional didactic
training (Moyers et al., 2008).
Outcome Measures. Most studies used self-reported
outcome measures, with the exception of independently
rated adherence and skill. Self-reported outcomes
included knowledge, satisfaction, therapist perception
of alliance, adoption, and burnout. Five studies
(Crits-Christoph et al., 1998; Henry, Strupp, et al.,
1993; Moyers et al., 2008; Sanders, Murphy-Brennan,
et al., 2003; Sholomskas et al., 2005) used standardized
psychometrically sound measures, and Luoma et al.
(2007) used a combination of investigator-created non-
standardized measures and standardized psychometrically
sound measures.
Summary of Studies. Physicians (n = 331) were
trained in Triple P (Sanders, Murphy-Brennan, et al.,
2003) and had access to peer support networks and web-
based support. After training, participants reported a
statistically significant increase in how well they believed
they were trained to manage behavior problems,
increased confidence in conducting parent consultation,
and high satisfaction with the training. In another study,
therapists (n = 30) were randomly assigned to a psycho-
logically focused group consultation after attending a
one-day workshop to overcome psychological barriers
in the use of new treatment techniques (Luoma et al.,
2007). In relation to the comparison group, the inter-
vention group reported more adoption of the treatment
at two- and four-month follow-ups, and a greater sense
of personal accomplishment at four-month follow-up.
Two interventions focused on the effects of EBP
training on therapist behavior as part of larger treat-
ment outcome studies (Crits-Christoph et al., 1998;
Henry, Strupp, et al., 1993). Both studies used manu-
als, didactic workshops, and ongoing supervision as part
of training, although one study provided significantly
more supervision (50 two-hour weekly sessions; Henry,
Strupp, et al., 1993). After receiving training and
supervision in TLDP (n = 16), therapist adherence and
general skills (e.g., greater use of open-ended questions)
were more highly rated. Unexpectedly, certain thera-
pist skills decreased (e.g., less optimistic, less supportive,
and more authoritative; Henry, Strupp, et al., 1993).
By contrast, Crits-Christoph et al. (1998) found that
following training in both DP and DC, therapist skill
increased as each case progressed (i.e., within case),
while CT therapists performed more skillfully across
cases (n = 65; Crits-Christoph et al., 1998). In other
words, only CT therapists were able to transfer their
learning from case to case. By the fourth training case,
CT therapists were scoring an average of 46 out of a
possible 48 points, suggesting great improvement and
proficiency in comparison with their first training case.
CLINICAL PSYCHOLOGY: SCIENCE AND PRACTICE • V17 N1, MARCH 2010 10
The most methodologically rigorous investigations
in this category both included random assignment to
varying training methods. One study employed an
investigation of which condition: (a) manual only,
(b) manual + web-based training, and (c) manual +
didactic training + supervision) was most effective in
improving therapist knowledge, adherence, and skill in
CBT for substance abuse (Sholomskas et al., 2005).
The highest dosage of training (manual + didactic
training + supervision) produced the highest levels of
therapist adherence and skill, with the manual + web
training evidencing intermediate scores and the manual
alone having the lowest levels of adherence and skill
after training and at follow-up. It is of note that in the
highest dosage of training, only 54% of clinicians
reached proficiency levels in adherence and skill
(Sholomskas et al., 2005). Knowledge was not signifi-
cantly different between the three groups. Another
study randomly assigned therapists to one of three con-
ditions: (a) workshop only, (b) workshop + training
enrichments (i.e., six supervision calls, specific feedback
on behavior), and (c) self-directed training (Moyers
et al., 2008). All conditions showed gains in compe-
tence from baseline to post-training with effect sizes in
the medium to large range; however, these gains
declined at four-month follow-up. Contrary to expec-
tations, improved competence was not observed in the
group that received training enrichments (i.e., feedback
and consultation calls).
Conclusions. After receiving training and follow-up
organizational support, therapists’ perceptions of their
behavior (e.g., confidence and personal accomplish-
ment) increase (Luoma et al., 2007; Sanders, Murphy-
Brennan, et al., 2003). In some studies, the perception
of behavior change is matched by actual therapist
behavior change; however, these changes in behavior
are not maintained at follow-up. When considering
specific EBP packages, some evidence supports the
notion that MI competence can be gained after attend-
ing a one-day workshop. However, these gains were
not maintained at follow-up, and a number of training
enrichments did not result in expected increased skill.
Contradictory evidence exists regarding the effect of
training in DP on therapist adherence. One study sug-
gests that certain DP-related skills improve, while
others deteriorate after training, particularly skills that
influence the therapeutic relationship (Henry, Strupp,
et al., 1993). A follow-up study suggests that DP skills
influencing the alliance may decrease in the first train-
ing case following training, but that this effect may
diminish as the therapist treats more clients (Crits-
Christoph et al., 1998). Thus, further supervision and
more than one training case may reverse an initial
awkward phase as the therapist grows comfortable
implementing the EBP.
Similarly, contradictory evidence exists regarding the
effect of CT and CBT training on therapist behavior.
One study demonstrates that CT training plus supervi-
sion allows skills to improve, suggesting that CT thera-
pists are able to apply what they have learned in training
and supervision to new cases. However, in another
study of CBT training, approximately half of the thera-
pists were not trained to an acceptable criterion in
adherence and skill, despite receiving the gold standard
in the field of training (Sholomskas et al., 2005).
Limitations. Study-specific issues include limited data
on supervisory practices, self-reported skill (Luoma
et al., 2007; Sanders, Murphy-Brennan, et al., 2003),
biased skill ratings from supervisors (Crits-Christoph
et al., 1998), comparisons across training interventions
that differ in time, difficulty in randomization
(Sholomskas et al., 2005), and difficulty in procuring
samples of therapist sessions and follow-up assessment
(Moyers et al., 2008).
Training and Therapist Variables
Description of Studies. Few studies focused on both
training and therapist variables (see Table 4).3 The
EBP training included (a) MI for adult substance use
(Rubel, Sobell, & Miller, 2000; Saitz, Sullivan, &
Samet, 2000), (b) behavioral family therapy (BFT) for
families caring for a relative with schizophrenia
(Brooker & Butterworth, 1993), and (c) an interven-
tion training program for youth suicide prevention
(Chagnon, Houle, Marcoux, & Renaud, 2007). Three
studies used a nonrandomized, convenience-sample,
pre–post, quasi-experimental design (Brooker &
Butterworth, 1993; Rubel et al., 2000; Saitz et al.,
2000); only one study randomly assigned participants
to a training condition and included a comparison
TRAINING IN EVIDENCE-BASED PRACTICE: A REVIEW • BEIDAS & KENDALL 11
T ab
le 4 .
Sy st
em s-
co n te
xt u al
le ve
l: T ra
in in
g an
d th
er ap
is t
va ri ab
le s
R ef
er en
ce s
n ES
T P at
ie n t
T h er
ap is
t T ra
in in
g A
ct iv
e El
em en
ts O
u tc
o m
e V
ar ia
b le
s D
es ig
n R
es u lt
s
B ro
o ke
r an
d B u tt
er w
o rt
h (1
9 9 3 )
8 B FT
Sc h iz
o .
p at
ie n ts
an d
th ei
r fa
m ili
es C
o m
m u n it y
p sy
ch ia
tr ic
n u rs
es
1 .
2 -d
ay in
tr o .
2 .
1 -w
ee k
sk ill
s tr
ai n in
g u si
n g
ro le
-p la
y an
d vi
d eo
-f ee
d b ac
k
1 .
R o le
-p la
y 2 .
V id
eo ta
p ed
fe ed
b ac
k co
n ce
rn in
g p ro
g re
ss in
sk ill
s.
1 .
R at
ed S
2 .
A C
o n ve
n ie
n ce
sa m
p le
; n o n ra
n d o m
iz ed
; 6 -
an d
1 2 -m
o n th
f ⁄u
; n o
co n tr
o l
g ro
u p ;
SM
1 .
C o m
p et
en t—
ve ry
g o o d
S 2 .
6 -m
o n th
f ⁄u
sk ill
im p ro
ve d
3 .
St re
n g th
en A
h el
d p ri o r
to tr
ai n in
g 4 .
C h an
g ed
A to
B FT
as b es
t tx
5 .
In cr
ea se
in h o u rs
sp en
t w
it h
fa m
ily C
h ag
n o n
et al
. (2
0 0 7 )
7 1
SA M
A t-
ri sk
yo u th
C o m
m u n it y
m em
b er
s 3 -d
ay w
o rk
sh o p
R o le
-p la
y 1 .
A 2 .
K 3 .
R at
ed S
R an
d o m
iz ed
; co
m p ar
is o n
g ro
u p ;
6 -m
o n th
f ⁄u
; IC
M an
d SM
1 .
A t
p o st
-t ra
in in
g ,
im p ro
ve m
en t
in A
, K
, an
d S
2 .
A t
f ⁄u
, A
m ai
n ta
in s
b u t
K an
d S
d ec
re as
e R
u b el
et al
. (2
0 0 0 )
4 4
M I
A d u lt
su b st
. ab
u se
M en
ta l
h ea
lt h
th er
ap is
ts
2 -d
ay w
o rk
sh o p
(d id
ac ti c
& ex
p er
ie n ti al
) &
re co
m m
en d ed
m an
u al
1 .
R o le
-p la
y 2 .
Fi sh
b o w
l ex
er ci
se s
1 .
K 2 .
R at
ed S
3 .
A *
C o n ve
n ie
n ce
sa m
p le
; n o n ra
n d o m
iz ed
; n o
f ⁄u
; n o
co n tr
o l g ro
u p ;
IC M
an d
SM
1 .
K an
d S
im p ro
ve d
fr o m
p re
- to
p o st
- tr
ai n in
g
Sa it z
et al
. (2
0 0 0 )
7 0
M I
A d u lt
su b st
. ab
u se
H ea
lt h -c
ar e
p ro
vi d er
s 2 0 0 -m
in w
o rk
sh o p
(5 0
m in
d id
ac ti c,
1 2 0
m in
ro le
- p la
y, 3 0
m in
d is
cu ss
io n )
R o le
-p la
ys w
it h
fe ed
b ac
k 1 .
Se lf -r
ep o rt
ed ch
an g e
in cl
in ic
al p ra
ct ic
e 2 .
K * *
3 .
A * *
4 .
C * *
5 .
S* *
C o n ve
n ie
n ce
sa m
p le
; n o n ra
n d o m
iz ed
; n o
f ⁄u
; n o
co n tr
o l g ro
u p ;
IC M
1 .
P o st
-t ra
in in
g o ve
ra ll
A w
er e
h ig
h er
2 .
Se lf -r
ep o rt
la rg
e im
p ac
t o n
cl in
ic al
p ra
ct ic
e 3 .
N o
ch an
g e
in K
, S,
o r
C
N o te
. B FT
, b eh
av io
ra l fa
m ily
th er
ap y;
sc h iz
o .
sc h iz
o p h re
n ia
; S,
sk ill
; A
, at
ti tu
d es
; f
⁄u ,
fo llo
w -u
p ;
SM ,
st an
d ar
d iz
ed m
ea su
re s;
tx ,
tr ea
tm en
t; SA
M ,
Su ic
id e
A ct
io n
M o n tr
ea l;
K ,
kn o w
le d g e;
IC M
, in
ve st
ig at
o r-
cr ea
te d
m ea
su re
s; M
I, m
o ti va
ti o n al
in te
rv ie
w in
g ;
A * ,
m ea
su re
d at
p re
, b u t
d o es
n o t
ap p ea
r to
h av
e b ee
n re
ad m
in is
te re
d at
p o st
; C
, co
n fi d en
ce ;
* * o n ly
co lle
ct ed
p re
–p o st
fo r
a sa
m p le
o f
th e
co h o rt
.
CLINICAL PSYCHOLOGY: SCIENCE AND PRACTICE • V17 N1, MARCH 2010 12
condition (Chagnon et al., 2007). Sample sizes ranged
from 8 to 87 therapist participants, with educational
levels ranging from bachelor’s to postgraduate degrees
(e.g., MD, PhD, and PsyD). Participants included psy-
chiatric nurses, mental health clinicians, community
members, and health-care workers.
Training Method and Content. All of the studies
included both passive learning (e.g., didactic presenta-
tion) and active learning strategies (e.g., role-plays with
feedback, experiential learning, and fishbowl exercises).
When sufficiently described, training content included
a focus on principles (Rubel et al., 2000). Training
duration varied from 200 min (Chagnon et al., 2007)
to seven days (Brooker & Butterworth, 1993). One
study recommended a manual to supplement the didac-
tic presentation (Rubel et al., 2000).
Therapist Variables. Studies measured therapist atti-
tudes and theoretical orientation. Two studies surveyed
substance abuse attitudes (Rubel et al., 2000; Saitz
et al., 2000), one study surveyed attitudes about schizo-
phrenia and its treatment (Brooker & Butterworth,
1993), and another measured attitudes towards inter-
vening with suicidal individuals (Chagnon et al., 2007).
Two studies assessed theoretical orientation (Brooker &
Butterworth, 1993; Rubel et al., 2000).3
Outcome Measures. For the majority of the studies,
self-reported outcome measures were used, with the
exception of independently rated skill. Self-reported
outcomes included knowledge, attitudes, and change in
practice. Saitz et al. (2000) used investigator-created
nonstandardized measures, Brooker and Butterworth
(1993) used standardized measures, and two studies uti-
lized both types of measures (Chagnon et al., 2007;
Rubel et al., 2000).
Summary of Studies. A survey of practicing clinicians
(n = 70) trained on MI indicated that training made an
impact on their practice with substance abuse patients,
particularly in regard to asking formal alcohol screening
questions and providing substance abuse counseling
(Saitz et al., 2000). Those participants who completed
pre- and post-training measures showed slightly higher
attitudes towards substance abuse after training,
although knowledge, confidence in ability to take a
substance history, and self-reported skillfulness did not
change.
More methodologically rigorous studies included
independent ratings of skill. Results from a 12-hour
MI training workshop indicated that both knowledge
and rated skill (i.e., percentage of motivational state-
ments to overall statements) improved from pre- to
post-training in mental health therapists (n = 44; Rubel
et al., 2000). However, participants only scored 67%
on the knowledge test—which does not indicate train-
ing to proficiency. Attitudes, measured pretraining,
indicated a psychosocial conceptualization of substance
abuse. A seven-day BFT workshop for community
psychiatric nurses demonstrated that rated skills were at
least satisfactory two months after training, with several
skills improving further at six-month follow-up
(Brooker & Butterworth, 1993). At six-month post-
training, nurses were proficient in both core and adap-
tive skills. Attitudes also changed following training,
such that belief in a psychosocial theory of schizophre-
nia and the usefulness of BFT programs increased.
The most methodologically rigorous study investi-
gated the effects of an intervention training in commu-
nity members who frequently interacted with youth at
risk of suicidal behavior (n = 78). Strengths of this
study included randomization and a comparison group.
In comparison with control subjects, participants in the
intervention group showed significant knowledge, atti-
tude, and skill improvement. At follow-up, knowledge
and skill improvement significantly diminished,
although change in attitudes was maintained. Although
participants improved in knowledge and skill at post-
training, scores were below proficiency level (i.e.,
knowledge 69%, skill 67%).
Conclusions. Survey data suggest that training in an
EBP has a lasting impact on clinical practice and self-
reported skill (i.e., Saitz et al., 2000), but actual behav-
ior change (e.g., rated skill) does not necessarily take
place when studies provide training without taking into
account the different facets of the SC model—provid-
ing training and measuring therapist variables are not
sufficient for DI efforts. Additionally, there appears to
be a lack of therapist proficiency reached after training.
The reported results of three studies indicated that
TRAINING IN EVIDENCE-BASED PRACTICE: A REVIEW • BEIDAS & KENDALL 13
training successfully changed knowledge and indepen-
dently rated skill to a proficient level, but in two of
these studies, exploration of the reported means sug-
gests that this was not the case.
Attitudes shifted following training, and were main-
tained at follow-up. Attitudes improved towards EBP
in all studies that included a pre- to post-training
assessment (Brooker & Butterworth, 1993; Chagnon
et al., 2007; Saitz et al., 2000). Attitude change was
also sustained even when other behaviors diminished at
follow-up (Chagnon et al., 2007). It may be that cur-
rent training efforts engender shifts in participant per-
ceptions and attitudes, but are unable to bring about
lasting skill change.
Limitations. Study-specific limitations include small
sample sizes, highly motivated participants (Brooker &
Butterworth, 1993), low participant response rate
(Rubel et al., 2001), and the use of a nonvalidated vid-
eotape technique to assess for skill acquisition
(Chagnon et al., 2007).
Training and Client Variables
Description of Studies. Two studies focused on train-
ing and client variables (see Table 5). The EBP train-
ing included (a) MI for adult substance use (Miller &
Mount, 2001) and (b) QPR for youth suicide pre-
vention (Wyman et al., 2008). One study used a
nonrandomized, convenience-sample, pre–post, quasi-
experimental design (Miller & Mount, 2001), whereas
the other study randomly assigned participants to a
training condition and included a comparison condition
(Wyman et al., 2008). Study sample sizes ranged from
22 to 249 participants, with education levels at the
bachelor or master’s level. Participants included proba-
tion officers, community corrections officers, and
secondary school staff.
Training Method and Content. Both studies included
passive learning (e.g., didactic presentation), whereas
one included active learning (e.g., small group practice;
Miller & Mount, 2001). Training content was not suf-
ficiently described in either study. Training duration
varied from 2 (Wyman et al., 2008) to 15 hours (Miller
& Mount, 2001). Miller and Mount (2001) used a
manual to supplement the didactic presentation. Ta b le
5 .
Sy st
em s-
co n te
xt u al
le ve
l: T ra
in in
g an
d cl
ie n t
va ri ab
le s
R ef
er en
ce s
n ES
T P at
ie n t
T h er
ap is
t T ra
in in
g A
ct iv
e el
em en
ts O
u tc
o m
e va
ri ab
le s
D es
ig n
R es
u lt
s
M ill
er an
d M
o u n t
(2 0 0 1 )
2 2
M I
A d u lt
su b st
. ab
u se
P O
an d
C C
O 1 .
2 -d
ay w
o rk
sh o p
2 .
M an
u al
3 .
6 o p ti o n al
(9 0
m in
) f
⁄u d is
cu ss
io n *
Sm al
l g ro
u p
p ra
ct ic
e
1 .
Se lf -r
ep o rt
ed S
an d
C 2 .
R at
ed S
3 .
C lie
n t
R
C o n ve
n ie
n ce
sa m
p le
; n o n ra
n d o m
iz ed
; 4 -m
o n th
f ⁄u
; n o
co n tr
o l g ro
u p ;
SM
1 .
In cr
ea se
in se
lf -r
ep o rt
ed S
an d
C p o st
-t ra
in in
g 2 .
M o d es
t in
cr ea
se s
in ra
te d
S at
p o st
-t ra
in in
g ,
w h ic
h w
er e
so m
ew h at
m ai
n ta
in ed
at 4 -m
o n th
f ⁄u
o n
cl ie
n t
3 .
N o
ef fe
ct re
sp o n se
s W
ym an
et al
. (2
0 0 8 )
2 4 9
Q P R
Y o u th
su ic
id e
p re
ve n ti o n
Se co
n d ar
y sc
h o o l
st af
f
1 .
1 .5
-h o u r
w o rk
sh o p
2 .
3 0 -m
in re
fr es
h er
co u rs
e se
ve ra
l m
o n th
s la
te r
N o n e
1 .
K 2 .
Se lf -r
ep o rt
ed A
, B ,
an d
C o
3 .
St u d en
t su
rv ey
R an
d o m
iz ed
; co
m p ar
is o n
g ro
u p ;
1 -y
ea r
f ⁄u
; IC
M
1 .
In cr
ea se
in K
, A
, B
2 .
Sm al
l ef
fe ct
in st
af f
as ki
n g
st u d en
ts re
: su
ic id
e- re
la te
d b eh
av io
rs
N o te
. M
I, m
o ti va
ti o n al
in te
rv ie
w in
g ;
P O
, p ro
b at
io n
o ffi
ce rs
; C
C O
, co
m m
u n it y
co rr
ec ti o n s
o ffi
ce rs
; * N
o t
at te
n d ed
b y
p ar
ti ci
p an
ts ;
f ⁄u
, fo
llo w
-u p ;
S, sk
ill ;
C ,
co n fi d en
ce ;
R ,
re sp
o n se
s; SM
, st
an d ar
d iz
ed m
ea su
re s;
Q P R
, q u es
ti o n ,
p er
su ad
e, re
fe r;
K ,
kn o w
le d g e;
A ,
ap p ra
is al
s; B ,
B eh
av io
r; C
o ,
C o m
m u n ic
at io
n ;
IC M
, in
ve st
ig at
o r-
cr ea
te d
m ea
su re
s.
CLINICAL PSYCHOLOGY: SCIENCE AND PRACTICE • V17 N1, MARCH 2010 14
Client Variables. In one study, client in-session state-
ments were used as a proxy for treatment outcome
(Miller & Mount, 2001), whereas in the other study,
students were surveyed regarding school staff commu-
nication in relation to suicidal ideation (Wyman et al.,
2008). Note that neither study utilized a measure of
severity, risk factors, and ⁄ or resiliency of therapists’ client population as predictor variables for training
efficacy (as recommended by Sanders & Turner,
2005); instead measures of client factors were included
as outcome variables.
Outcome Measures. The outcome measures included
self-report (Wyman et al., 2008) and independently
rated behavior (Miller & Mount, 2001). Self-reported
outcomes included knowledge, skill, appraisals, behav-
ior, and staff-student communication. Independently
rated behavior included rated MI skill. One study
utilized investigator-created nonstandardized measures
(Wyman et al., 2008), whereas the other study utilized
both investigator-created nonstandardized and standard-
ized psychometrically sound measures (Miller &
Mount, 2001).
Summary of Studies. Wyman et al. (2008) measured
the effect of QPR training on therapist knowledge and
behavior. Secondary school staff (n = 249) received
two hours of didactic training on suicide prevention.
In comparison with nontrained staff, knowledge, self-
reported appraisals, and behavior changes were reported
at one-year follow-up. Student responses (i.e., client
variables) suggested a small effect (d = 0.18) in being
asked more frequently about suicide by staff members
(one of the main skills of QPR). Although a statistically
significant effect was present in terms of knowledge
acquisition (d = 0.41), trained staff scored an average of
76% on the knowledge test, while nontrained staff
scored an average of 72% (both below proficiency
levels).
Miller and Mount (2001) investigated the effect of
training in MI on community corrections and proba-
tion officers working with adult substance abusers
(n = 22). Participants received 15 hours of didactic
training, which included small group practice and a
manual. Self-reports by therapists indicated large
increases in MI skills, while observational measures
reflected more modest skill changes after training that
were somewhat retained at four-month follow-up. Cli-
ents did not show the response changes found to be
predictive of better outcome with MI.
Conclusions. The findings suggest that self-reported
perceptions of change are frequently not matched by
actual behavior change. Neither study was able to
engender significant change at the client level, suggest-
ing that the main point of initiating these training
studies (i.e., to improve client response to treatment)
may not be occurring. Lack of therapist behavior
change may help explain this finding. This is concern-
ing given that therapist confidence was very high in
regard to their own skill, suggesting that they may not
feel that they need further training or consultation in
the implementation of the EBP (Miller & Mount,
2001).
An important consideration for both of these studies
is that client response was included as an outcome vari-
able. However, client variables (e.g., severity and resil-
iency) were not considered as moderators of training
effectiveness. Such data need to be co-varied in statisti-
cal analyses to depict the interactions between training
and client variables. For example, a severe population
(such as suicidal youth) may be more difficult to access;
so, while training may have succeeded in therapist
behavior change, client variables may make the treat-
ment more difficult to implement.
Limitations. Study-specific limitations included low
staff enrollment (Wyman et al., 2008) and a small sam-
ple size with participants choosing to not attend
optional follow-up discussion sessions that may have
contributed valuable organizational support (Miller &
Mount, 2001). High therapist attrition and ⁄ or low par- ticipation in follow-up training in DI research must be
studied systemically due to the frequency of this occur-
rence in research studies (e.g., Baer et al., 2004; Miller
& Mount, 2001).
Training, Organizational Support, and Therapist Variables
Description of Studies. Several studies (n = 5) focused
on training, organizational support, and therapist vari-
ables (see Table 6).3 The EBP training studied
included (a) BFT for families (Fadden, 1997), (b) CBT
TRAINING IN EVIDENCE-BASED PRACTICE: A REVIEW • BEIDAS & KENDALL 15
T ab
le 6 .
Sy st
em s-
co n te
xt u a l le
ve l:
T ra
in in
g ,
o rg
an iz
at io
n al
su p p o rt
, an
d th
er ap
is t
va ri ab
le s
R ef
er en
ce s
n ES
T P at
ie n t
T h er
ap is
t T ra
in in
g A
ct iv
e el
em en
ts O
u tc
o m
e va
ri ab
le s
D es
ig n
R es
u lt
s
Fa d d en
(1 9 9 7 )
8 6
B FT
Fa m
ili es
w it h
sc h iz
o .
m em
b er
N u rs
in g -l
ev el
cl in
ic ia
n s
1 .
O ri en
ta ti o n
2 .
M an
u al
3 .
3 -d
ay sk
ill s
w o rk
sh o p
4 .
T en
1 .5
-h o u r
su p er
vi si
o n s
5 .
O n g o in
g m
o n th
ly su
p er
vi si
o n
R o le
-p la
ys ,
vi d eo
ta p ed
d em
o n st
ra ti o n s
o f
se ss
io n s
1 .
N u m
b er
o f
fa m
ili es
se en
2 .
B ar
ri er
s to
u se
o f
tx 3 .
Im p ac
t o f
B FT
o n
w o rk
C o n ve
n ie
n ce
sa m
p le
; n o n ra
n d o m
iz ed
; n o
f ⁄u
; n o
p re
-; n o
co n tr
o l
g ro
u p ;
IC M
1 .
7 0 %
re p o rt
ed u si
n g
B FT
2 .
M ea
n fa
m ili
es se
en =
1 .7
3 .
6 6 %
re p o rt
ed d if fi cu
lt to
u se
4 .
T h er
ap is
ts in
th e
co m
m .
o r
w h er
e m
o re
st af
f m
em b er
s w
er e
tr ai
n ed
in B FT
sa w
m o re
fa m
ili es
5 .
Li tt
le at
ti tu
d e
ch an
g e
H en
g g el
er ,
C h ap
m an
, et
al .
(2 0 0 8 )
4 3 2
C M
Su b st
. ab
u se
in ad
o l.
P u b lic
se ct
o r
cl in
ic ia
n s
1 .
M an
u al
2 .
1 -d
ay d id
ac ti c
an d
ex p er
ie n ti al
w o rk
sh o p
Ex p er
ie n ti al
ro le
-p la
ys 1 .
A d o p
2 .
A 3 .
O rg
*
C o n ve
n ie
n ce
sa m
p le
; n o n ra
n d o m
iz ed
; 6 -m
o n th
f ⁄u
; n o
co n tr
o l
g ro
u p ;
IC M
+ SM
1 .
5 8 %
re p o rt
ed u se
2 .
P ra
ct .
va ri ab
le s
p re
d ic
te d
ad o p
3 .
P ra
ct .
an d
o rg
. va
ri ab
le s
p re
d ic
te d
A M
au n d er
et al
. (2
0 0 8 )
2 5
C B T
A d u lt
an d
yo u th
in P C
M D
s 1 .
3 -
to 4 -h
o u r
w o rk
sh o p
2 .
M an
u al
3 .
Fo u r
1 .5
-h o u r
ca se
d is
cu ss
io n s
C as
e d is
cu ss
io n
g ro
u p s
1 .
K 2 .
Se lf -r
ep o rt
ed u se
3 .
A tt
an d
Sa
C o n ve
n ie
n ce
sa m
p le
; n o n ra
n d o m
iz ed
; n o
f ⁄u
; n o
co n tr
o l
g ro
u p ;
IC M
+ SM
1 .
H ig
h Sa
2 .
In cr
ea se
d K
3 .
In cr
ea se
d u se
M o rg
an st
er n
et al
. (2
0 0 1 )
2 9
C B T
A d u lt
su b st
. ab
u se
Su b st
. ab
u se
co u n se
lo rs
1 .
1 0 0
h o u rs
tr ai
n in
g (3
5 -h
o u r
d id
ac ti c
tr ai
n in
g an
d fe
ed b ac
k, su
p er
vi si
o n ,
an d
m an
u al
)
R o le
-p la
ys an
d fe
ed b ac
k 1 .
R at
ed A
2 .
R at
ed S
3 .
A tt
an d
Sa
R an
d o m
iz ed
; co
m p ar
is o n
g ro
u p ;
n o
f ⁄u
; IC
M an
d SM
1 .
H ig
h Sa
2 .
H ig
h A
tt to
w ar
d C
B T
3 .
P ro
fi ci
en t
A &
S
Si q u el
an d
et al
. (2
0 0 0 )
6 2
C T , D P ,
D C
A d u lt
su b st
. ab
u se
Su b st
. ab
u se
co u n se
lo rs
1 .
M an
u al
2 .
Fo u r
2 -d
ay w
o rk
sh o p s
3 .
U n sp
ec ifi
ed su
p er
vi si
o n
R o le
-p la
ys 1 .
R at
ed A
2 .
R at
ed C
C o n ve
n ie
n ce
sa m
p le
; n o n ra
n d o m
iz ed
; n o
f ⁄u
; n o
co n tr
o l
g ro
u p ;
IC M
an d
SM
1 .
In C
T g en
er al
ex p er
ie n ce
an d
sp ec
ifi c
ex p er
ie n ce
in C
T p re
d ic
te d
C
N o te
. B FT
, b eh
av io
ra l
fa m
ily th
er ap
y; sc
h iz
o .,
sc h iz
o p h re
n ia
; tx
, tr
ea tm
en t;
f ⁄u
, fo
llo w
-u p ;
IC M
, in
ve st
ig at
o r-
cr ea
te d
m ea
su re
s; C
M ,
co n ti n g en
cy m
an ag
em en
t; su
b st
., su
b st
an ce
; ad
o l.,
ad o le
sc en
t; A
d o p ,
ad o p ti o n ;
A ,
ad h er
en ce
; O
rg ,
o rg
an iz
at io
n al
va ri ab
le s;
SM ,
st an
d ar
d iz
ed m
ea su
re s;
C B T ,
co g n it iv
e- b eh
av io
ra l th
er ap
y; P C
, p ri m
ar y
ca re
; M
D s,
p h ys
ic ia
n s;
K ,
kn o w
le d g e;
A tt
, at
ti tu
d es
; Sa
, sa
ti sf
ac ti o n ;
S, sk
ill ;
C ,
co m
p et
en ce
; C
T ,
co g n it iv
e th
er ap
y; D
P ,
d yn
am ic
th er
ap y;
D C
, d ru
g co
u n se
lin g .
* n o t
in cl
u d ed
as an
o u tc
o m
e va
ri ab
le ,
b u t
m ea
su re
d o rg
an iz
at io
n al
ch ar
ac te
ri st
ic s
as p re
d ic
to rs
.
CLINICAL PSYCHOLOGY: SCIENCE AND PRACTICE • V17 N1, MARCH 2010 16
in primary care (Maunder, Milne, & Cameron, 2008),
(c) CBT for adult substance abuse (Morganstern, Mor-
gan, McCrady, Keller, & Carroll, 2001), (d) contin-
gency management (CM) for youth substance abuse
(Henggeler, Chapman, et al., 2008), and (e) CT, DP,
and DC for adult substance abuse (Siqueland et al.,
2000). Three studies used a nonrandomized, conve-
nience-sample, pre–post, quasi-experimental design
with no comparison group (Henggeler, Chapman,
et al., 2008; Maunder et al., 2008; Siqueland et al.,
2000), whereas one study surveyed participants after
training (Fadden, 1997). The most rigorous study
included randomization and a comparison group (Mor-
ganstern et al., 2001). Sample sizes ranged from 25 to
432 participants. Educational level ranged from bache-
lor to postgraduate level (i.e., PhD or MD), and parti-
cipants included community mental health therapists,
nurses, and physicians.
Training Method and Content. All studies included
both passive learning (e.g., didactic presentation) and
active learning strategies (e.g., experiential role-plays:
Fadden, 1997; Henggeler, Chapman, et al., 2008; Mor-
ganstern et al., 2001; Siqueland et al., 2000; case dis-
cussion groups: Maunder et al., 2008). Training
content included going through session by session
(Henggeler, Chapman, et al., 2008; Siqueland et al.,
2000) and a focus on principles of the treatment
(Maunder et al., 2008), with one study including both
a focus on session-by-session administration and princi-
ples of the treatment (Morganstern et al., 2001). Train-
ing duration varied from one day (Henggeler,
Chapman, et al., 2008) to 100 hours (Morganstern
et al., 2001). All studies used a manual to supplement
the didactic presentation.
Organizational Support. Most studies (Fadden, 1997;
Maunder et al., 2008; Morganstern et al., 2001;
Siqueland et al., 2000) included ongoing supervision as
part of the training intervention. Supervision time
ranged from four 1.5-hour case discussion groups
(Maunder et al., 2008) to 65 hours of supervision
(Morganstern et al., 2001). One study did not include
supervision but included measures of organizational
characteristics (i.e., organizational readiness for change;
Henggeler, Chapman, et al., 2008).
Therapist Variables. Two studies surveyed participant
attitudes towards the patient population before and
after training (Fadden et al., 1997; Morganstern et al.,
2001), and two studies assessed participant attitudes
towards the training received and treatment modality
(i.e., CBT; Maunder et al., 2008; Morganstern et al.,
2001). One study measured pretraining experience
(e.g., number of cases and supervision hours received)
as a potential predictor of post-training competence
(Siqueland et al., 2000), whereas another study mea-
sured pretraining demographics and experience (e.g.,
years of experience and caseload characteristics) and
therapist attitudes towards EBP as a predictor of
adherence, adoption, and implementation of an EBP
(Henggeler, Chapman, et al., 2008).3
Outcome Measures. Outcome measures included self-
report (Fadden, 1997; Henggeler, Chapman, et al.,
2008; Maunder et al., 2008) or self-report and inde-
pendently rated behavior (Morganstern et al., 2001;
Siqueland et al., 2000). Self-reported outcomes
included declarative knowledge, implementation,
acceptability of treatment, use of treatment, barriers to
use, attitudes towards patient population ⁄ treatment modality, and adoption. Independently rated behavior
included adherence, skill, and competence. Four studies
used both investigator-created nonstandardized and
standardized psychometrically sound measures (which
were often modified for the purposes of each study;
Henggeler, Chapman, et al., 2008; Maunder et al.,
2008; Morganstern et al., 2001; Siqueland et al., 2000),
and one study used an investigator-created nonstan-
dardized survey (Fadden, 1997).
Summary of Studies. Three studies utilized only self-
report questionnaires to observe the impact of training
on therapist knowledge, attitudes, and behavior. Thera-
pist participants trained in BFT (n = 86) were surveyed
after completing a 39-hour four-phase training
9 months to 3.5 years after completing the training
(Fadden, 1997). No pretraining assessment was com-
pleted. Seventy percent of therapists reported they had
used BFT in their work since training; however, the
average number of families seen was 1.7. Forty percent
of families were seen by 8% of the trained therapists,
indicating that a small proportion of trainees was seeing
TRAINING IN EVIDENCE-BASED PRACTICE: A REVIEW • BEIDAS & KENDALL 17
a large percentage of the families. Therapist variables
influencing the number of families receiving BFT
included therapist location (community vs. inpatient
unit) and the number of therapists trained in each
service area. Little attitudinal change was observed, and
did not predict the number of families seen. In a more
recent study, primary care physicians (n = 25) trained
in CBT (i.e., three- to four-hour workshop and four
follow-up case discussions; Maunder et al., 2008)
reported high satisfaction with the training, as well as
an increase in the use of CBT techniques with patients.
Declarative knowledge scores increased after training
(Maunder et al., 2008) to proficiency levels.
Mental health therapists (n = 432) received a one-
day workshop on CM with adolescent substance abus-
ers (Henggeler, Chapman, et al., 2008). Although
supervision was not provided, important organizational
and therapist variables were identified. Fifty-eight per-
cent of workshop attendees attempted to use the treat-
ment post-training and therapist variables predicted
adoption of the treatment: More educated and more
experienced therapists who held favorable attitudes
towards manualized therapy and lacked expertise in the
treatment were more likely to implement the treat-
ment. Self-reported adherence was also investigated.
Therapist variables predicted self-reported adherence:
Adherence was higher for younger therapists certified
in addictions, with larger caseloads, higher numbers of
youth on their caseloads, with and who held more
positive views regarding treatment manuals. Organiza-
tional characteristics, specifically, organizational motiva-
tional readiness to change, and greater organizational
training exposure and utilization were associated with
increased adherence to the treatment modality.
More methodologically rigorous studies included
independent ratings of skill ⁄ competence. Two studies investigated differing treatment modalities for substance
abuse. One study conducted training (manual and four-
day workshops) in three different treatment modali-
ties—CT, DP, and DC (n = 62; Siqueland et al.,
2000)—and included supervision focusing on feedback
from taped sessions. General experience as a therapist
and specific experience in treating patients in CT had a
positive impact on change in competence post-training,
whereas those having had more previous supervision
showed less change in competence after training.
The most rigorous study included random assign-
ment and a comparison condition, and investigated
CBT training (�100 hours of didactics and supervision) on independently rated adherence and skill in substance
abuse counselors (n = 29; Morganstern et al., 2001).
Participants reported high levels of training satisfaction
and also endorsed high ratings for the utility of CBT as
a treatment. Attitudes towards the conceptualization of
substance abuse changed in the intervention group.
Most participants reached independently rated adher-
ence levels of somewhat to extensively adherent and
independently rated skill levels of poor to good.
Conclusions. One important conclusion is the lack of
consensus on the role of therapist variables on compe-
tence and adherence in cognitive and ⁄ or behavioral approaches (i.e., BFT, CM, and CT). With regard to
therapist competence post-training, it is unclear
whether or not pretraining competence predicts post-
training competence, and may differ by treatment
modality (e.g., CT). Interestingly, multiple studies
report that therapists with more previous supervision
show less change in competence, perhaps due to more
allegiance to their own conceptualizations. When con-
sidering therapist self-reported adherence to CM, sev-
eral therapist and organizational predictors emerged,
but only one study investigated this question, thus
making it difficult to make any conclusions other than
the importance of future replication and study.
Another conclusion emerges: When three levels of
the SC model are addressed (i.e., training, organiza-
tional, and therapist levels), training in CBT may be
more successful as evidenced by two studies (Maunder
et al., 2008; Morganstern et al., 2001). After training in
CBT, participant outcomes included high satisfaction,
attitudinal change, utilization, increased knowledge,
and rated proficiency in adherence and skill (Maunder
et al., 2008; Morganstern et al., 2001).
Limitations. Study-specific limitations include lack of
pretraining measurement of therapist behavior and
inconsistency in survey administration (Fadden, 1997),
low response rates (Maunder et al., 2008), and small
sample sizes (Maunder et al., 2008; Morganstern et al.,
2001; Siqueland et al., 2000). The only study that
utilized a control condition did not compare the
CLINICAL PSYCHOLOGY: SCIENCE AND PRACTICE • V17 N1, MARCH 2010 18
control group with the intervention group on rated
skillfulness or adherence in delivery of CBT, thus
reducing the internal validity of the study (Morganstern
et al., 2001).
Training, Organizational Support, and Client Variables
Description of Studies. Four studies focused on train-
ing, organizational support, and client variables (see
Table 7). The EBP training included (a) group CBT for
adult substance abuse (Watkins, Osilla, Hepner, Sandres,
& Thompson, 2008), (b) MI for substance abuse (Scho-
ener, Madeja, Henderson, Ondersma, & Janisse, 2006),
(c) TLDP for adults (Bein et al., 2000), and (d) MST for
adolescent substance abuse (Henggeler, Sheidow,
Cunningham, Donohue, & Ford, 2008). Three studies
used a nonrandomized, convenience-sample, pre–post,
quasi-experimental design (Bein et al., 2000; Schoener
et al., 2006), one study did not include pretraining mea-
sures (included a comparison group for client outcomes;
Watkins et al., 2008), and one study used a randomized
design with a comparison group (Henggeler, Sheidow,
et al., 2008). Sample sizes ranged from 5 to 30 partici-
pants. Educational level ranged from bachelor to post-
graduate level (i.e., PhD or MD), and participants
included community mental health therapists, psycholo-
gists, physicians, and substance abuse counselors.
Training Method and Content. All studies included
passive learning (e.g., didactic presentation), whereas
two included active learning strategies (e.g., practice
with feedback; Henggeler, Sheidow, et al., 2008;
Schoener et al., 2006). One study did not include any
active training strategies (Bein et al., 2000), whereas
another did not provide enough of a description to
characterize method (Watkins et al., 2008). Training
content included a focus on principles of the treatment
(Bein et al., 2000), a focus on principles and skills acqui-
sition (Schoener et al., 2006), or a focus on session-by-
session instruction (Henggeler, Sheidow, et al., 2008).
One study did not provide an adequate description of
training content (Watkins et al., 2008). Training dura-
tion varied from a two-day workshop and eight supervi-
sions (Schoener et al., 2006) to 100 hours of seminar
and supervision (Bein et al., 2000). Two studies used a
manual to supplement the didactic presentation (Bein
et al., 2000; Henggeler, Sheidow, et al., 2008).
Organizational Support. All studies included ongoing
supervision as part of the training intervention, ranging
from eight supervision sessions (Schoener et al., 2006)
to 100 hours of supervision (Bein et al., 2000). In one
study, the level of organizational support was manipu-
lated, where following a workshop, therapists received
standard consultation or intensive quality assurance
consisting of weekly consultation and quarterly booster
training (Henggeler, Sheidow, et al., 2008).
Client Variables. Studies measured client in-session
statements or treatment outcome to indicate whether
or not therapist training effected client behavior
change. In one study, client change talk was used as a
proxy for treatment outcome (Schoener et al., 2006).
One study included client ratings of therapist adherence
as the primary outcome measure (Henggeler, Sheidow,
et al., 2008).
Outcome Measures. The outcome measures for two
studies included self-report and independently rated
behavior (Bein et al., 2000; Watkins et al., 2008),
whereas two studies included only independently rated
behavior as the outcome measure (Henggeler,
Sheidow, et al., 2008; Schoener et al., 2006). Self-
reported outcomes included client psychopathology
(Bein et al., 2000; Watkins et al., 2008). Independently
rated behavior included adherence (Henggeler, Shei-
dow, et al., 2008), competence (Watkins et al., 2008),
and skill (Bein et al., 2000; Schoener et al., 2006). All
four studies used a standardized psychometrically sound
measure (albeit adapted for each study; Bein et al.,
2000; Henggeler, Sheidow, et al., 2008; Schoener
et al., 2006; Watkins et al., 2008).
Summary of Studies. Clinicians (n = 30) were trained
in MST through a manual and workshop and then
were randomized to a workshop-only condition (access
to materials and phone access to an MST expert as
needed) or an intensive supervision condition (weekly
supervision and consultation, booster training quarterly,
improvement of skills incorporated into clinician devel-
opment plans; Henggeler, Sheidow, et al., 2008).
Supervisors were also evaluated to augment therapists’
use of the intervention. Significant differences between
youth-reported clinician adherence to MST techniques
TRAINING IN EVIDENCE-BASED PRACTICE: A REVIEW • BEIDAS & KENDALL 19
T ab
le 7 .
Sy st
em s-
co n te
xt u al
le ve
l: T ra
in in
g ,
o rg
an iz
at io
n al
su p p o rt
, an
d cl
ie n t
va ri ab
le s
R ef
er en
ce s
n ES
T P at
ie n t
T h er
ap is
t T ra
in in
g A
ct iv
e el
em en
ts O
u tc
o m
e va
ri ab
le s
D es
ig n
R es
u lt
s
B ei
n et
al .
(2 0 0 0 )
1 6
T LD
P A
d u lt
p sy
ch .
M D
⁄P h D
1 .
5 0
w ee
kl y
2 -
h o u r
se m
in ar
s an
d su
p er
vi si
o n
(f o cu
s o n
p ri n ci
p le
s) 2 .
M an
u al
3 .
O n e
tr ai
n in
g ca
se
N o n e
1 .
C lie
n t
p sy
ch .
2 .
T h er
ap is
t p ro
fi ci
en cy
C o n ve
n ie
n ce
sa m
p le
; n o n ra
n d o m
iz ed
; 1 -y
ea r
f ⁄u
; p re
tr ai
n in
g co
m p ar
is o n ;
SM
1 .
N o
d if fe
re n ce
s b et
w ee
n p re
- an
d p o st
-t ra
in in
g cl
ie n t
p sy
ch .
at p o st
- an
d 1 -y
ea r
f ⁄u
2 .
O n ly
2 8 %
re ac
h ed
m in
im al
le ve
l o f
p ro
fi ci
en cy
H en
g g el
er ,
Sh ei
d o w
, et
al .
(2 0 0 8 )
3 0
M ST
A d o l.
su b st
. ab
u se
M as
te r’
s- le
ve l
cl in
ic ia
n 1 .
M an
u al
2 .
W o rk
sh o p
3 .
R an
d o m
iz ed
to w
o rk
sh o p
o n ly
o r
in te
n si
ve q u al
it y
as su
ra n ce
(s u p er
vi si
o n
an d
q u ar
te rl y
b o o st
er tr
ai n in
g )
R o le
-p la
y w
it h
fe ed
b ac
k an
d p o si
ti ve
re in
fo rc
em en
t
1 .
Y o u th
- an
d ca
re g iv
er -
re p o rt
ed th
er ap
is t
A
B as
el in
e, p o st
w o rk
sh o p
th en
ra n d o m
iz ed
; co
m p ar
is o n
g ro
u p ;
4 -m
o n th
f ⁄u
; SM
1 .
Si g n ifi
ca n t
ad h er
en ce
re p o rt
ed b y
yo u th
at p o st
an d
d if fe
re n ce
b et
w ee
n ad
h er
en ce
to M
ST an
d C
B T
at f
⁄u 2 .
Si g n ifi
ca n t
d if fe
re n ce
b et
w ee
n M
ST an
d C
B T
te ch
n iq
u es
re p o rt
ed b y
ca re
g iv
er at
p o st
b u t
n o t
f ⁄u
3 .
N o
ch an
g e
in u se
o f
M ST
yo u th
o r
m o n it o ri n g
b y
ca re
g iv
er re
p o rt
Sc h o en
er et
al .
(2 0 0 6 )
1 0
M I
A d u lt
su b st
. ab
u se
an d
C O
D Fr
o n tl in
e cl
in ic
ia n s
1 .
2 -d
ay w
o rk
sh o p
2 .
Ei g h t
su p er
vi si
o n s
Su p er
vi se
d p ra
ct ic
e an
d fe
ed b ac
k d u ri n g
w o rk
sh o p
1 .
T h er
ap is
t S
2 .
C lie
n t
ch an
g e
ta lk
C o n ve
n ie
n ce
sa m
p le
; n o n ra
n d o m
iz ed
; p re
- to
p o st
d es
ig n ;
n o
f ⁄u
; n o
co m
p ar
is o n
g ro
u p ;
SM
1 .
T h er
ap is
t im
p ro
ve d
M I-
co n si
st en
t sk
ill s*
2 .
T h er
ap is
t d ec
re as
ed M
I- in
co n si
st en
t sk
ill s
3 .
O b se
rv ed
cl ie
n t
ch an
g e
ta lk
W at
ki n s
et al
. (2
0 0 8 )
5 C
B T
A d u lt
su b st
. ab
u se
Su b st
. ab
u se
ad d ic
ti o n s
co u n se
lo rs
1 .
2 -d
ay w
o rk
sh o p
2 .
W ee
kl y
su p er
vi si
o n
3 .
1 -d
ay b o o st
er tr
ai n in
g
N o t
d es
cr ib
ed 1 .
A 2 .
C 3 .
C lie
n t
o u tc
o m
e
C o n ve
n ie
n ce
sa m
p le
; n o n ra
n d o m
iz ed
; n o
p re
-; n o
f ⁄u
; co
m p ar
is o n
g ro
u p
fo r
tx ;
SM
1 .
9 1 %
A 2 .
4 .3
⁄6 C
3 .
C lie
n t
d ep
re ss
io n
im p ro
ve d
N o te
. T LD
P ,
ti m
e- lim
it ed
d yn
am ic
p sy
ch o th
er ap
y; p sy
ch .,
p sy
ch o p at
h o lo
g y;
f ⁄u
, fo
llo w
-u p ;
SM ,
st an
d ar
d iz
ed m
ea su
re s;
M ST
, m
u lt is
ys te
m ic
th er
ap y;
M I,
m o ti va
ti o n al
in te
rv ie
w in
g ;
su b st
., su
b st
an ce
; C
O D
, co
-o cc
u rr
in g
d is
o rd
er s;
S, sk
ill ;
C B T ,
co g n it iv
e- b eh
av io
ra l
th er
ap y;
A ,
ad h er
en ce
; C
, co
m p et
en ce
; tx
, tr
ea tm
en t.
* St
ill b el
o w
p ro
fi ci
en cy
le ve
ls .
CLINICAL PSYCHOLOGY: SCIENCE AND PRACTICE • V17 N1, MARCH 2010 20
emerged when comparing the two conditions after
training. Both youth-reported adherence (75%) and
caregiver-reported adherence (65%) were higher in
the intensive supervision condition. However, the
caregiver-reported adherence effect was not sustained
at follow-up.
Schoener et al. (2006) investigated the effect of
therapist behavior on client change talk following
training (i.e., independently coded from actual therapy
sessions). Community clinicians (n = 10) were trained
in MI through a two-day workshop and eight supervi-
sion sessions for patients with co-occurring substance
abuse and Axis I disorders. After training, indepen-
dently rated behavior suggested that therapists showed
improved MI skill (empathy, MI-spirit, and reflective
listening) and decreased MI-inconsistent behavior
(closed-ended questions and advising without permis-
sion). Additionally, change in therapist MI skill was
accompanied by a change in client self-talk, a variable
associated with subsequent client behavior change.
However, therapists exhibited lower average MI profi-
ciency in comparison with past MI training trials and
did not reach the recommended proficiency level (5 ⁄ 7 Likert; Miller, 2000). Organizational and ⁄ or therapist variables influenced training response: Therapists who
had been employed in the same agencies for long peri-
ods of time appeared less capable of benefitting from
training for certain MI skills (i.e., open-ended ques-
tioning). Poststudy interviews provided contextual
environmental information that may help explain these
results: Many therapists reported high levels of anxiety
and a number of systemic and structural challenges to
the application of newly acquired MI skills.7
Two studies investigated therapist training effects on
client treatment outcome in differing therapy modali-
ties. Substance abuse counselors (n = 5) received two
days of didactic training, ongoing weekly supervision,
and a one-day booster training for group CBT for
depression (Watkins et al., 2008). Counselors treated
61 clients and 35% of the sessions were coded for
adherence and competence. Therapists demonstrated
91% adherence (two or three on a 0–3 scale) across
coded sessions. Additionally, therapists delivered com-
petent CBT with a mean competence rating of 4.3 (of
6). Client depression scores improved with treatment
when compared with a comparison group. In another
treatment modality, clinicians (n = 16) received 50
weekly two-hour seminars and supervision in TLDP
for adult psychopathology. Training in TLDP did not
improve patient outcome at post-training or at one-
year follow-up. Follow-up analyses examined therapist
proficiency at TLDP by independently rating two cases
for each therapist. Only 9 of the 32 training cases
(28%) were judged to have been conducted with a
minimal level of skill (Bein et al., 2000).
Conclusions. If therapists reach proficient levels in
adherence ⁄ competence, and have adequate consultation and organizational support, it is possible to elicit behav-
ior change in client outcomes. In three of the four
studies (Bein et al., 2000; Henggeler, Sheidow, et al.,
2008; Schoener et al., 2006), therapists did not reach
proficient levels in adherence and ⁄ or competence. This is concerning, particularly given that appropriate super-
vision and training seems to have been provided, par-
ticularly in the MST and TLDP studies. Given the
striking amount of intervention, specifically targeting
problem-solving barriers to MST implementation, these
results are surprising. Similarly, in the study of TLDP,
an adequate dosage of supervision was provided (one
year of supervision—approximately 100 hours). How-
ever, this study only included one training case, which
may not have been sufficient for gaining mastery over
the skills needed to administer TLDP.
The importance of examining contextual variables
at the level of therapist and organizational support
becomes clear. What was it about these therapists that
made it difficult for them to reach proficiency levels
in MI, MST, or TLDP? One possibility may be their
lack of experience. In previous studies of MI, thera-
pist participants had prior experience in the treatment
modality, suggesting that training further consolidated
this experience (e.g., Miller & Mount, 2001; Miller
et al., 2004). Similarly, in the study that reported
positive client outcomes, therapists endorsed past
experience with the treatment and high motivation to
learn (Watkins et al., 2008). Perhaps therapists who
are naı̈ve to the fundamental principles of a treatment
may require further training and follow-up supervi-
sion. Other therapist variables may also be important,
such as allegiance to current therapeutic approaches
and commitment to learning a new treatment
TRAINING IN EVIDENCE-BASED PRACTICE: A REVIEW • BEIDAS & KENDALL 21
modality. In one study, follow-up interviews sug-
gested that systemic and contextual issues made it dif-
ficult to implement an EBP, highlighting the
importance of organizational variables.
Limitations. Study-specific limitations include the
small sample size of all the studies and the manner in
which client outcome was operationalized—as a change
in either self-talk (Schoener et al., 2006) or self-report
rather than structured interviews (Bein et al., 2000;
Watkins et al., 2008). Additionally, independently rated
adherence from a youth and caregiver perspective only,
without the inclusion of an expert perspective, was a
weakness (Henggeler, Sheidow, et al., 2008).
Training, Organizational Support, Therapist, and Client
Variables
Description of Studies. Few studies focused on all lev-
els of the SC model (see Table 8). The EBP training
included (a) TLDP for adults (Henry, Schacht, Strupp,
Butler, & Binder, 1993), (b) MI for addictions (Miller
et al., 2004), and (c) a comparison of supportive mental
health counseling, MI, and group CBT for substance
abuse and comorbid depression (Hunter, Watkins,
Wenzel, Gilmore, Sheehe, & Griffin, 2005). One study
used a nonrandomized, convenience-sample, pre–post,
quasi-experimental design (Henry, Schacht, et al.,
1993), whereas another study included a comparison
group (Hunter et al., 2005). The most rigorous study
included randomization to one of five groups and a
comparison group (Miller et al., 2004). Study sample
sizes ranged from 13 to 140 participants. Educational
level ranged from bachelor to postgraduate level (i.e.,
PhD or MD), and participants included community
mental health therapists, licensed substance abuse coun-
selors, and medical professionals.
Training Method and Content. All studies included
both passive learning (e.g., didactic presentation) and
two included active learning strategies (e.g., interactive
educational meetings, Hunter et al., 2005; and feedback
and coaching, Miller et al., 2004). Training content
included a focus on principles of the treatment for all
three studies. Training duration varied from two days
(Miller et al., 2004) to 100 hours (Henry, Schacht,
et al., 1993). Two studies used a manual to supplement Ta b le
8 .
Sy st
em s-
co n te
xt u al
le ve
l: T ra
in in
g ,
o rg
an iz
at io
n al
, cl
ie n t,
an d
th er
ap is
t va
ri ab
le s
R ef
er en
ce s
n ES
T P at
ie n t
T h er
ap is
t T ra
in in
g A
ct iv
e el
em en
ts O
u tc
o m
e va
ri ab
le s
D es
ig n
R es
u lt
s
H en
ry ,
Sc h ac
h t,
et al
. (1
9 9 3 )
1 6
T LD
P A
d u lt
p sy
ch .
M D
an d
P h D
1 .
M an
u al
2 .
5 0
w ee
kl y
se m
in ar
s an
d su
p er
vi si
o n
3 .
T ra
in in
g ca
se
N o n e
1 .
T h er
ap is
t A
2 .
T h er
ap is
t S
P re
- to
p o st
d es
ig n ;
co n ve
n ie
n ce
sa m
p le
; n o
f ⁄u
; n o
co m
p ar
is o n
g ro
u p
1 .
T h er
ap is
t va
ri ab
le s
in fl u en
ce d
A 2 .
C lie
n t
va ri ab
le s
in fl u en
ce d
S
H u n te
r et
al .
(2 0 0 5 )
1 3
M I,
g ro
u p
C B T
A d u lt
su b st
. +
C O
D M
en ta
l h ea
th co
u n se
lo rs
1 .
2 .5
-h o u r
1 2 -w
ee k
tr ai
n in
g 2 .
B iw
ee kl
y 3 .
Su b se
t re
ce iv
ed d ep
re ss
io n
tr ai
n in
g
In te
ra ct
iv e
ed u ca
ti o n al
m ee
ti n g s
(n o t
sp ec
ifi ed
)
1 .
K 2 .
A 3 .
Jo b
Sa 4 .
Jo b
M
P re
- to
p o st
d es
ig n ;
co n ve
n ie
n ce
sa m
p le
; 1 -y
ea r
f ⁄u
; co
m p ar
is o n
g ro
u p
1 .
K in
cr ea
se d *
2 .
A in
cr ea
se d *
3 .
Jo b
Sa +
M
M ill
er et
al .
(2 0 0 4 )
1 4 0
M I
A d u lt
su b st
. ab
u se
Li ce
n se
d su
b st
. ab
u se
p ro
fe ss
1 .
W S
2 .
W SF
3 .
W SC
4 .
W SC
+ F
5 .
SG T
V ar
ie d
ac ro
ss tr
ai n in
g (W
SF ,
W SC
, an
d W
SC +
F)
1 .
C 2 .
C lie
n t
ch an
g e
ta lk
R an
d o m
iz ed
; co
m p ar
is o n
g ro
u p ;
4 -,
8 -,
1 2 -m
o n th
f ⁄u
; SM
1 .
A ll
fo u r
w o rk
sh o p s
sh o w
ed in
cr ea
se in
C p re
–p o st
2 .
W S
g ro
u p
re ve
rs ed
g ai
n s
at 4 -m
o n th
f ⁄u
3 .
R ed
u ct
io n
in M
I- in
co n st
. b eh
av io
r
N o te
. T LD
P ,
ti m
e- lim
it ed
d yn
am ic
p sy
ch o th
er ap
y; p sy
ch .,
p sy
ch o p at
h o lo
g y;
A ,
ad h er
en ce
; S,
Sk ill
; f
⁄u ,
fo llo
w -u
p ;
SM ,
st an
d ar
d iz
ed m
ea su
re s;
M I,
m o ti va
ti o n al
in te
rv ie
w in
g ;
C B T ,
co g n it iv
e- b eh
av io
ra l
th er
ap y;
su b st
., su
b st
an ce
; C
O D
, co
-o cc
u rr
in g
d is
o rd
er s;
K ,
kn o w
le d g e;
Sa ,
sa ti sf
ac ti o n ;
M ,
m o ra
le ;
W S,
w o rk
sh o p
o n ly
; W
SF ,
w o rk
sh o p
p lu
s fe
ed b ac
k; W
SC ,
w o rk
sh o p
p lu
s co
ac h in
g ;
W SC
+ F,
w o rk
sh o p
p lu
s co
ac h in
g an
d fe
ed b ac
k; SG
T ,
se lf -g
u id
ed tr
ai n in
g ;
C ,
co m
p et
en ce
. * N
o st
at is
ti ca
l an
al ys
es co
m p le
te d
b et
w ee
n co
m p ar
is o n
an d
in te
rv en
ti o n
g ro
u p
b ec
au se
o f
5 0 %
st af
f at
tr it io
n an
d sm
al l sa
m p le
si ze
.
CLINICAL PSYCHOLOGY: SCIENCE AND PRACTICE • V17 N1, MARCH 2010 22
the didactic presentation (Henry, Schacht, et al., 1993;
Miller et al., 2004).
Organizational Support. Two studies (Henry, Schacht,
et al., 1993; Hunter et al., 2005) included ongoing
supervision as part of the training intervention, ranging
from biweekly supervision (Hunter et al., 2005) to
100 hours of supervision and a training case (Henry,
Schacht, et al., 1993). One study included two training
enhancement procedures (ongoing feedback and coach-
ing), which were considered to be most in line with
organizational support given their similarity to supervi-
sion (Miller et al., 2004).
Therapist Variables. Hunter et al. (2005) surveyed
participant attitudes towards mental illness, job satisfac-
tion, and job morale pre- and post-training (Hunter
et al., 2005); Miller et al. (2004) measured pretraining
characteristics, history of substance abuse, self-esteem,
and interpersonal style as predictors of post-training
competence and client change outcome; and Henry,
Schacht, et al. (1993) measured pretraining demograph-
ics and experience (e.g., years of experience and prior
competence) as mediators of therapists’ response to
training.3
Client Variables. Client characteristics were examined
as mediators of therapist training response in two stud-
ies (Henry, Schacht, et al., 1993; Hunter et al., 2005),
whereas in Miller et al. (2004), client change talk was
used as a proxy for treatment outcome.
Outcome Measures. The outcome measures included
self-report only (Hunter et al., 2005) or self-report and
independently rated behavior (Henry, Schacht, et al.,
1993; Miller et al., 2004). Self-reported outcomes
included knowledge, attitudes about mental health, job
satisfaction, and job morale. Independently rated
behavior included interviewing style, adherence, inter-
personal processes (Henry, Schacht, et al., 1993), and
MI competence (e.g., MI-consistent behaviors, reflec-
tion:question ratio; Miller et al., 2004). One study used
an investigator-created nonstandardized survey measure
(Hunter et al., 2005), whereas two studies utilized stan-
dardized psychometrically sound measures (Henry,
Schacht, et al., 1993; Miller et al., 2004).
Summary of Studies. One study used only self-report
questionnaires to assess the effect of training on thera-
pist knowledge, attitudes, and behavior. Mental health
counselors (n = 13) trained in supportive mental health
counseling and MI for addictions received 12 weeks of
training (2.5 hours each) and biweekly supervision
(Hunter et al., 2005). Note that a subset of counselors
received 20 hours of training in group CBT for
depression. Client characteristics were collected.
Although the study included a comparison group, only
descriptive analyses were completed due to high staff
attrition (50%) in both the intervention and compari-
son site and the resulting very small sample. Declarative
knowledge about substance abuse and co-occurring dis-
orders increased in the intervention site after training
(M = 83%) and was sustained over time (M = 81%) in
comparison with the control site (post-training
M = 68%; follow-up M = 74%). Attitudes became
more positive at the intervention site at post-training
and follow-up. Client characteristics were reported but
not analyzed.
More methodologically rigorous studies included
the effect of therapist training on independent ratings
of adherence and competence. TLDP training con-
sisted of 50 weekly two-hour in-person seminar and
supervision sessions including didactic presentations of
principles, a manual, and treatment of a training case
with supervision (n = 16; Henry, Schacht, et al.,
1993). Individual differences were quantified amongst
the two trainers—the better trainer was directive and
specific regarding the learning task, focused on thera-
pist rather than patient dynamics, and provided posi-
tive reinforcement regarding specific therapist action.
Therapist variables influenced training response—
therapists with more prior supervision were less
adherent, while therapists with certain interpersonal
styles showed greatest technical adherence. Client
variables also influenced training response—therapists
showed greater improvements in skill when working
with patients traditionally deemed less suitable for
short-term techniques.
The gold standard for studies conducted in this liter-
ature includes random assignment and a comparison
condition (Miller et al., 2004). Therapists were ran-
domized to one of five training conditions (n = 140):
(a) two-day workshop only, (b) two-day workshop
TRAINING IN EVIDENCE-BASED PRACTICE: A REVIEW • BEIDAS & KENDALL 23
plus feedback on tapes, (c) two-day workshop plus six
individual coaching sessions, (d) two-day workshop
plus coaching and feedback, and (e) a control condi-
tion—self-guided learning (therapist manual and train-
ing videotapes). Therapist variables did not predict
therapist training response. All four intervention condi-
tions (i.e., a–d) produced increased competence at
post-training. At four-month follow-up, the work-
shop-only group lost their gains and returned near the
levels of the comparison group. Only the groups
receiving feedback and/or coaching reached profi-
ciency levels in global MI-spirit (five of a seven-point
Likert scale) and MI-consistent responses (95%) at post-
training and follow-up. Client response only improved
in the most intensive training (workshop, coaching,
and feedback). The biggest effect in training was not
an increase in therapist MI-consistent responses but a
reduction in MI-inconsistent responses.
Conclusions. Intervening at all four levels of the SC
model may be the key to producing effective out-
comes. Therapist variables are influenced by training
when measured as outcome variables. For example,
attitudes and job satisfaction can be improved following
training (Hunter et al., 2005). However, when examin-
ing therapist variables as predictor variables or mediat-
ing variables, contradictory evidence exists. One study
found that therapist variables did influence response to
training (Henry, Schacht, et al., 1993), while the most
rigorous study found no effect of therapist variables
(Miller et al., 2004).
Support for client variables as predictors of training
and outcome variables emerges. When examined as a
predictor of training response, it appears that therapist
training improved when the client population was
more difficult and more educated. There may be a
complex relationship with supervision present here—
perhaps more difficult clients call for more supervision
time. In terms of outcomes, clients improved when
therapists received the most intensive training interven-
tion (Miller et al., 2004).
An interesting and important finding implicates the
importance of training and organizational factors.
Training style produced differential outcomes in thera-
pist behavior in one study, suggesting that certain
methods of training are preferential. Further, training
method emerged as an important variable when manip-
ulated—the only training that produced proficiency in
MI at long-term follow-up included active learning
strategies (i.e., coaching and ⁄ or feedback). The only training to elicit actual client change was the one that
included the highest dosage of active learning strate-
gies—feedback and coaching. This suggests the impor-
tance of studying training method (e.g., active versus
passive) in future studies.
An issue that emerges from these studies is the iden-
tification of the barriers and challenges faced in com-
munity research. Hunter et al. (2005) published a
follow-up article documenting barriers to DI research,
including recruitment falling behind schedule, staff
attrition, a comparison site instituting new practices
that duplicated the intervention site, and the increase
in patients already receiving mental health treatment
when presenting to the intervention site (Wenzel,
Ebener, Hunter, Watkins, & Gilmore, 2005). Follow-
up client outcome research was not published due to
difficulties in recruitment, although the intervention
site continues to use the treatment in the community
(S. Hunter, personal communication). These barriers
make it difficult to disseminate and implement EBP in
the community.
Limitations. Study-specific limitations include very
high staff attrition that precluded the use of statistical
analyses and difficulty in recruitment for client treat-
ment outcome investigations (Hunter et al., 2005).
None of these studies measured client outcome using
the preferred technique of semi-structured interviews.
GENERAL DISCUSSION
The most definitive conclusion (see Table 9) is that
training influences therapist knowledge, attitudes, and
perceived behavior. Therapist behavior change (e.g.,
adherence and skill) can occur following training, but a
number of conditions must be met. First, it is necessary
for training to address most, if not all, levels of the SC
model (the therapist is nested within a context and
organizational, therapist, and client variables all interact
transactionally with training to influence skillful imple-
mentation). Additionally, training must utilize active
learning strategies to influence therapist behavior
change.
CLINICAL PSYCHOLOGY: SCIENCE AND PRACTICE • V17 N1, MARCH 2010 24
Knowledge acquisition follows training. Both per-
ceived and declarative knowledge increase after receiv-
ing training in EBP, and this finding appears to hold
across treatment modalities and therapists. Therapist
attitudes also change and this change is maintained over
time following training. Importantly, training engen-
ders self-reported behavior change that does not always
match actual behavior change—pointing to the impor-
tance of including independently rated behavior in all
future assessments of EBP training. The mismatch in
perception and actual behavior is highly problematic
and has important implications because clinicians may
believe they are implementing a treatment with fidelity
and skill (Miller & Mount, 2001). Additionally, this has
deleterious consequences for client treatment outcome
given the finding that better outcomes are observed in
clients when a treatment is applied with fidelity (Elliot
& Mihalic, 2004).
A most surprising finding is the lack of proficiency in
treatment adherence, competence, and skill reached by
therapists trained in the current gold standard (i.e.,
workshop, manual, and clinical supervision; Sholomskas
et al., 2005). Multiple studies across differing treatments
found that training (workshop, manual, and supervision)
was not enough to produce proficient change in thera-
pist adherence, competence, and skill. This is
concerning, considering the importance of these
constructs in administering a treatment with fidelity
(Perpepletchikova & Kazdin, 2005). In turn, therapists
were not necessarily able to engender client change. If
therapists lack proficiency in a treatment, perhaps linked
to inadequate training, then it is unlikely that clients will
experience the benefits associated with a particular EBP.
Although proficiency in therapist behavior was not
often achieved, when studies intervened at the levels of
training, organizational, and therapist variables, thera-
pists did reach proficiency levels in adherence, compe-
tence, and skill, particularly in CBT. Similarly, when
all levels of the SC model (i.e., Miller et al., 2004)
were addressed and active learning was used, both ther-
apist and client change occurred. As posited by the SC
model, therapist, client, and organizational variables
interplay and influence the effects of training. To
achieve client change, it is necessary for therapists to
reach proficiency levels in knowledge, adherence,
competence, and skill. This perspective speaks to the
importance of training therapists to criteria prior to
implementing treatments.
What occurs within training that may produce dif-
ferential outcomes? An intriguing finding highlights the
quality (i.e., content and method) of training. Active
learning appears to be most effective—the only study
to evidence client change included two active learning
processes (i.e., coaching and feedback).
Most aspects of the SC model were understudied.
An SC perspective dictates consideration of the organi-
zational forces and factors that create the climate for
successful DI efforts. Research that measures ways to
understand organizations (see Organizational Social
Context; Glisson et al., 2008) will inform needed orga-
nizational change for DI efforts. Zazzali et al. (2008)
interviewed administrators of state organizations where
an EBP had been implemented and found that organi-
zational characteristics (e.g., interest in EBP and avail-
able resources) and organizational structures influenced
the adoption and implementation of EBP. Similarly,
low workplace support predicted less utilization of the
Triple P program following training in service
providers (Sanders, Prinz, & Shapiro, 2009). Further
examination of the organizational context in the imple-
mentation of EBP is needed.
Table 9. General conclusions
Across treatment modalities and therapists, perceived and declarative knowledge increase following training in EBP.
Across treatment modalities and therapists, attitudes improve after training in EBP and this is maintained at follow-up.
Across treatment modalities and therapists, perceived (self-reported) therapist behavior change does not match actual behavior change.
Generally speaking, therapists trained in the current format (i.e., workshop, manual, and brief supervision) do not reach proficiency in treatment adherence, competence, and skill.
There is insufficient information about how therapist variables, client characteristics, and organizational variables influence therapist behavior following training (adherence, competence, and skill).
Generally speaking, therapist training in EBP does not currently engender improved client outcomes.
The quality of training is important to engender client change: Active learning during training is integral to influence both therapist and client change.
Provisional evidence suggests that when addressing all levels of the systems-contextual model, therapists reach proficiency levels in adherence, competence, and skill, particularly in CBT, and in turn influence client change.
Note. EBP, evidence-based practice.
TRAINING IN EVIDENCE-BASED PRACTICE: A REVIEW • BEIDAS & KENDALL 25
The incomplete literature leaves an absence of
conclusions regarding therapist variables. Consistent
measurement is a needed first step. One measure of ther-
apist attitudes, the Evidence Based Practice Attitude
Scale (EBPAS; see Aarons, 2004), provides a beginning
and may help to identify those therapist variables associ-
ated with adherence to and competence with EBP.
Client characteristics were rarely investigated,
despite the consensus that they are probably important.
The influence of client variables cannot be answered
adequately at this time. Future research needs to
include measures of client risk, resiliency, and symptom
severity. Importantly, future research should occur in
tandem with RCTs as such a step would permit exam-
ining whether variations in therapist training engender
client change.
Future Directions
Future DI research will benefit from addressing organi-
zational, therapist, and client variables to demonstrate
how these important contextual factors interact with
training. RCTs with adequate samples and standardized
measures, regardless of the EBP being studied, are
encouraged. Investigations are needed to assess whether
training on one EBP generalizes to training on other
EBPs, and studies are needed to evaluate whether uni-
form guidelines can be created regarding best practices
for training across treatment modalities.
The following questions regarding best practices for
training require research:
1. What elements (e.g., manual, workshop, and super-
vision) should be included in training?
2. What is the optimal duration of training?
3. What is the optimal focus of training (i.e., general
principles of a treatment or session by session)?
4. What type of active learning methods should be
included?
5. What role does ‘‘training to criteria’’ play? Should
trainees be certified in a treatment when they reach
prespecified levels of competency (Sanders,
Murphy-Brennan, et al., 2003; Sanders, Tully, et
al., 2003)?
6. How much supervision is needed to achieve thera-
pist behavior change and competent administration
of an EBP?
Based on this review, we make the following rec-
ommendations:
1. Future training efforts focus on active learning and
behavioral rehearsal with less emphasis on didactics.
2. Follow the SC model. Influencing one variable (e.g.,
therapist training) within a system is unlikely to result
in effective implementation without addressing con-
textual factors. It is unlikely that training and DI will
succeed without the understanding that therapists
function within a context and that multiple variables
(i.e., organizational support, client factors, and thera-
pist factors) affect this context.
3. To assess that actual behavior change is occurring,
future investigations of training must include direct
measures of therapist behavior (e.g., adherence and
skill) in addition to assessments of knowledge and
perceived behavior change.
4. Standardized measurement is necessary to make cross-
study comparisons. For example, a psychometrically
reliable (Moyers, Martin, Manuel, Hendrickson, &
Miller, 2005) system that is treatment specific includes
the Motivational Interviewing Treatment Integrity
scale (MITI; Moyers, Martin, Catley, Harris, & Ah-
luwalia, 2003). Studies of training in MI could use this
measure rather than investigator-created measures.
Additionally, developing measures that can be used
across treatment programs would be useful.
5. The competencies for each treatment program
should be identified so that it is clear what types of
competencies are necessary for each EBP (see Roth
& Pilling, 2008). For instance, for CBT for child
anxiety, active treatment components could be
identified (e.g., exposure and cognitive restructur-
ing), and therapists trained in this treatment should
achieve competencies in these domains.
6. Identifying the barriers to training and utilization of
treatment is important and may be addressed early
in training to problem-solve perceived barriers
(Seng et al., 2006).
7. Supervision is crucial for skillful treatment delivery,
and understanding the role of supervision and its
key features are valuable areas for future research.
Despite the importance of EBP, we know less than
preferred regarding how to best train therapists in EBP.
CLINICAL PSYCHOLOGY: SCIENCE AND PRACTICE • V17 N1, MARCH 2010 26
Training efforts in EBP are underway, yet we are not
informed by how best to proceed. If those who under-
take EBP are not adequately trained in EBP implementa-
tion, will a subsequent evaluation of the benefits of EBP
be unwittingly and unfairly compromised? Evaluations
of training methods and variables, within contextual
variables, will be informative. Without such an undertak-
ing, one’s best EBP intentions may go unfulfilled.
NOTES
1. Going forward, EBP will be used as the terminology of
choice, rather than empirically supported treatments
(ESTs), given that it is a less restrictive term and allows for
a consideration of both rigorous research applied with
clinical judgment. However, when studies explicitly refer
to ESTs rather than EBP, it will be noted in the text.
2. Studies were included if they met criteria as an evidence-
based practice by an external review (e.g., National Regis-
try of Evidence Based Programs and Practice; Chambless
& Hollon, 1998; Chambless & Ollendick, 2001). For a
listing of all treatments reviewed, see Table 1.
3. Studies that just included one question on a demographics
questionnaire were not operationalized as measuring thera-
pist variables.
4. To be included in this review, all studies had to include
training as a variable of interest.
5. Note that certain studies distinguished between perceived
knowledge (e.g., perception of knowledge gained) and
declarative knowledge (e.g., direct measure of knowledge
gained), and we reported on this when this distinction was
made.
6. Direct access to the conclusions can be reached by refer-
ring to the General Discussion section.
7. Note that these were not statistically examined.
REFERENCES
Aarons, G. (2004). Mental health provider attitudes toward
adoption of evidence-based practice: The Evidence-Based
Practice Attitude Scale (EBPAS). Mental Health Services
Research, 6, 61–74.
Abramowitz, J. (2006). Toward a functional analytic approach
to psychologically complex patients: A comment on
Ruscio and Holohan. Clinical Psychology: Science and Prac-
tice, 13, 163–166.
Addis, M., & Krasnow, A. (2000). A national survey of prac-
ticing psychologists’ attitudes toward psychotherapy treat-
ment manuals. Journal of Consulting and Clinical Psychology,
68, 331–339.
American Academy of Child and Adolescent Psychiatry. (2006).
American Academy of Child and Adolescent Psychiatry
policy statement on evidence-based practice in psychiatry.
Retrieved July 5, 2007, from http://www.aacap.
org/cs/root/policy_statements/evidence_based_practice
American Psychological Association. (2005, August). Ameri-
can Psychological Association policy statement on evi-
dence-based practice in psychology. Retrieved March 6,
2006, from http://www.apa.org/practice/ebpreport.pdf
*Baer, J. S., Rosengren, D. B., Dunn, C. W., Wells, E. A.,
& Ogle, R. L. (2004). An evaluation of workshop train-
ing in motivational interviewing for addiction and mental
health clinicians. Drug and Alcohol Dependence, 73, 99–
106.
Bazelmans, E., Prins, J., Hoogveld, S., & Bleijenberg, G.
(2004). Manual-based cognitive behavior therapy for
chronic fatigue syndrome: Therapists’ adherence and per-
ceptions. Cognitive Behavior Therapy, 33, 143–150.
*Beidas, R. S., Barmish, A. J., & Kendall, P. C. (2009).
Training as usual: Can therapist behavior change after
reading a manual and attending a brief workshop on cog-
nitive behavioral therapy for youth anxiety? The Behavior
Therapist, 32, 97–101.
*Bein, E., Anderson, T., Strupp, H. H., Henry, W. P.,
Schacht, T. E., Binder, J. L., et al. (2000). The effects of
training in time-limited dynamic psychotherapy: Changes
in therapeutic outcome. Psychotherapy Research, 10, 119–
132.
*Brooker, C., & Butterworth, T. (1993). Training in psycho-
social intervention: The impact on the role of community
psychiatric nurses. Journal of Advanced Nursing, 18, 583–
590.
*Chagnon, F., Houle, J., Marcoux, I., & Renaud, J. (2007).
Control-group study of an intervention training program
for youth suicide prevention. Suicide and Life-Threatening
Behavior, 37, 135–144.
Chambless, D., & Hollon, S. (1998). Defining empirically
supported therapies. Journal of Consulting and Clinical Psy-
chology, 66, 7–18.
Chambless, D., & Ollendick, T. (2001). Empirically sup-
ported psychological interventions: Controversies and evi-
dence. Annual Review of Psychology, 52, 685–716.
Crits-Christoph, P., Frank, E., Chambless, D., Brody, C., &
Karp, J. (1995). Training in empirically-validated treat-
ments: What are clinical psychology students learning?
Professional Psychology, 26, 514–522.
Crits-Christoph, P., Siqueland, L., Chittams, J., Barber, J. P.,
Beck, A. T., Frank, A., et al. (1998). Training in cogni-
tive, supportive-expressive, and drug counseling therapies
TRAINING IN EVIDENCE-BASED PRACTICE: A REVIEW • BEIDAS & KENDALL 27
for cocaine dependence. Journal of Consulting and Clinical
Psychology, 66, 484–492.
*Cross, W., Matthieu, M., Cerel, J., & Knox, K. (2007).
Proximate outcomes of gatekeeper training for suicide
prevention in the workplace. Suicide and Life-Threatening
Behavior, 37, 659–670.
DeViva, J. (2006). The effects of full-day and half-day work-
shops for health care providers in techniques for increasing
resistant clients’ motivation. Professional Psychology: Research
and Practice, 37, 83–90.
Elliot, D., & Mihalic, S. (2004). Issues in disseminating and
replicating effective prevention programs. Prevention Sci-
ence, 5, 47–53.
Ellis, M., Krengel, M., Ladany, N., & Schult, D. (1996).
Clinical supervision research from 1981–1993: A method-
ological critique. Journal of Counseling Psychology, 43, 35–
50.
El-Tannir, A. (2002). The corporate university model for
continuous learning, training, and development. Education
& Training, 44, 76–81.
*Fadden, G. (1997). Implementation of family interventions
in routine clinical practice following staff training pro-
grams: A major cause for concern. Journal of Mental Health,
6, 599–612.
Gega, L., Norman, I., & Marks, I. (2007). Computer-aided
vs. tutor-delivered teaching of exposure therapy for phobia ⁄ panic: randomized controlled trial with pre-registration
nursing students. International Journal of Nursing Studies, 44,
397–405.
Glisson, C., Landsverk, J., Schoenwald, S., Kelleher, K.,
Hoagwood, K., Mayberg, S., et al. (2008). Assessing the
organizational social context (OSC) of mental health ser-
vices: Implications for research and practice. Administration
and Policy in Mental Health and Mental Health Services
Research, 35, 98–113.
*Hawkins, K. A., & Sinha, R. (1998). Can front-line clini-
cians master the conceptual complexities of dialectical
behavior therapy? An evaluation of a State Department on
Mental Health training program. Journal of Psychiatric
Research, 32, 379–384.
Hayes, S. (2002). Getting to dissemination. Clinical Psychology:
Science and Practice, 9, 410–415.
*Henggeler, S., Chapman, J., Rowland, M., Halliday-
Boykins, C., Randall, J., Shackelford, J., et al. (2008).
Statewide adoption and initial implementation of contin-
gency management for substance-abusing adolescents. Jour-
nal of Consulting and Clinical Psychology, 76, 556–567.
Henggeler, S., Schoenwald, S., Liao, J., Letourneau, E., &
Edwards, D. (2002). Transporting efficacious treatments to
field settings: The link between supervisory practices and
therapist fidelity in MST programs. Journal of Clinical Child
and Adolescent Psychology, 31, 155–167.
*Henggeler, S., Sheidow, A., Cunningham, P., Donohue,
B., & Ford, J. (2008). Promoting the implementation of
an evidence-based intervention for adolescent marijuana
abuse in community settings: Testing the use of intensive
quality assurance. Journal of Clinical Child and Adolescent
Psychology, 37, 682–689.
*Henry, W. P., Schacht, T. E., Strupp, H. H., Butler, S. F.,
& Binder, J. L. (1993). Effects of training in time-limited
dynamic psychotherapy: Mediators of therapists’ responses
to training. Journal of Consulting and Clinical Psychology, 61,
441–447.
*Henry, W. P., Strupp, H. H., Butler, S. F., Schacht, T. E.,
& Binder, J. L. (1993). Effects of training in time-limited
dynamic psychotherapy: Changes in therapist behavior.
Journal of Consulting and Clinical Psychology, 61, 434–440.
Herschell, A., McNeil, C., & McNeil, D. (2004). Clinical
child psychology’s progress in disseminating empirically
supported treatments. Clinical Psychology: Science and Prac-
tice, 11, 267–288.
Holloway, E. L., & Neufeldt, S. A. (1995). Supervision: Its
contributions to treatment efficacy. Journal of Consulting
and Clinical Psychology, 63, 207–213.
*Hunter, S., Watkins, K., Wenzel, S., Gilmore, J., Sheehe,
J., & Griffin, B. (2005). Training substance abuse treat-
ment staff to care for co-occurring disorders. Journal of
Substance Abuse Treatment, 28, 239–245.
James, I., Blackburn, I., Milne, D., & Reichfelt, F. (2001).
Moderators of trainee therapists’ competence in cogni-
tive therapy. British Journal of Clinical Psychology, 40,
131–141.
Karekla, M., Lundgren, J., & Forsyth, J. (2004). A survey of
graduate training in empirically supported and manualized
treatments: A preliminary report. Cognitive and Behavioral
Practice, 11, 230–242.
Kendall, P. C., & Beidas, R. S. (2007). Smoothing the trail
for dissemination of evidence-based practices for youth:
Flexibility within fidelity. Professional Psychology: Research
and Practice, 38, 13–20.
Kendall, P. C., & Southam-Gerow, M. (1996). Long-term
follow-up of treatment for anxiety disordered youth. Jour-
nal of Consulting and Clinical Psychology, 64, 724–730.
Landis, J., & Koch, G. (1977). The measurement of observer
agreement for categorical data. Biometrics, 45, 233–268.
Lomas, J. (1993). Diffusion, dissemination and implementa-
tion: Who should do what? Annals of the New York Acad-
emy of Science, 703, 226–235.
CLINICAL PSYCHOLOGY: SCIENCE AND PRACTICE • V17 N1, MARCH 2010 28
*Luoma, J., Hayes, S., Twohig, M., Roget, N., Fisher, G.,
Padilla, M., et al. (2007). Augmenting continuing educa-
tion with psychologically focused group consultation:
Effects on adoption of group drug counseling. Psychother-
apy: Theory, Research, Practice, Training, 44, 463–469.
*Matthieu, M., Cross, W., Batres, A., Flora, C., & Knox, K.
(2008). Evaluation of gatekeeper training for suicide pre-
vention in veterans. Archives of Suicide Research, 12, 148–
154.
*Maunder, L., Milne, D., & Cameron, L. (2008). Pilot
evaluation of brief training in CBT for primary care
therapists. Behavioural and Cognitive Psychotherapy, 36,
341–351.
*McVey, G., Davis, R., Kaplan, A., Katzman, D., Pinhas, L.,
Geist, R., et al. (2005). A community-based training pro-
gram for eating disorders and its contribution to a provin-
cial network of specialized services. International Journal of
Eating Disorders, 37, 35–40.
Miller, W. (2000). Rediscovering fire: Small interventions,
large effects. Psychological Addictions & Behavior, 14, 6–
18.
*Miller, W. R., & Mount, K. A. (2001). A small study of
training in motivational interviewing: Does one workshop
change clinician and client behavior? Behavioural and Cog-
nitive Psychotherapy, 29, 457–471.
Miller, W., Sorensen, J., Selzer, J., & Brigham, G. (2006).
Disseminating evidence-based practices in substance abuse
treatment: A review with suggestions. Journal of Substance
Abuse Treatment, 31, 25–39.
*Miller, W., Yahne, C., Moyers, T., Martinez, J., & Pirritano,
M. (2004). A randomized trial of methods to help clinicians
learn motivational interviewing. Journal of Consulting and
Clinical Psychology, 72, 1050–1062.
*Morganstern, J., Morgan, T., McCrady, B., Keller, D., &
Carroll, K. (2001). Manual-guided cognitive-behavioral
therapy training: A promising method for disseminating
empirically supported substance abuse treatments to the
practice community. Psychology of Addictive Behaviors, 15,
83–88.
*Moyers, T., Manual, J., Wilson, P., Hendrickson, S.,
Talcott, W., & Durand, P. (2008). A randomized trial
investigating training in motivational interviewing for
behavioral health providers. Behavioural and Cognitive Psy-
chotherapy, 36, 149–162.
Moyers, T., Martin, T., Catley, D., Harris, K., & Ahluwalia,
J. (2003). Assessing the integrity of motivational inter-
viewing interventions: Reliability of the Motivational
Interviewing Skills Code. Behavioural and Cognitive Psycho-
therapy, 31, 177–184.
Moyers, T., Martin, T., Manuel, J., Hendrickson, S., & Miller,
W. (2005). Assessing competence in motivational inter-
viewing. Journal of Substance Abuse Treatment, 28, 19–26.
Najavits, L., Weiss, R., Shaw, S., & Dierberger, A. (2000).
Psychotherapists’ views of manuals. Professional Psychology:
Research and Practice, 31, 404–408.
*National Crime Victims Research & Treatment Center.
(2007). TF-CBT web: First year report. Charleston, SC:
Medical University of South Carolina.
Perpepletchikova, F., & Kazdin, A. (2005). Treatment integrity
and therapeutic change: Issues and research recommenda-
tions. Clinical Psychology: Science and Practice, 12, 365–383.
Roth, A., & Pilling, S. (2008). Using an evidence-based
methodology to identify the competencies required to
deliver effective cognitive and behavioral therapy for
depression and anxiety disorders. Behavioural and Cognitive
Psychotherapy, 36, 129–147.
*Rubel, E. C., Sobell, L. C., & Miller, W. R. (2000). Do
continuing education workshops improve participants’
skills? Effects of a motivational interviewing workshop on
substance-abuse counselors’ skills and knowledge. Behavior
Therapist, 23, 73–77.
*Saitz, R., Sullivan, L. M., & Samet, J. H. (2000). Training
community-based clinicians in screening and brief inter-
vention for substance abuse problems: Translating evi-
dence into practice. Substance Abuse, 21, 21–31.
*Sanders, M., Murphy-Brennan, M., & McAuliffe, C.
(2003). The development, evaluation, and dissemination
of a training programme for general therapists in evi-
dence-based parent consultation skills. International Journal
of Mental Health Promotion, 5, 13–20.
*Sanders, M., Prinz, R., & Shapiro, C. (2009). Predicting
utilization of evidence-based parenting interventions with
organizational, service-provider and client variables.
Administrative Policy and Mental Health, 36, 133–143.
*Sanders, M., Tully, L., Turner, K., Maher, C., &
McAuliffe, C. (2003). Training GPs in parent consultation
skills. Australian Family Physician, 32, 763–768.
Sanders, M., & Turner, K. (2005). Reflections on the chal-
lenges of effective dissemination of behavioural family
intervention: Our experience with the Triple P—Positive
parenting program. Child and Adolescent Mental Health, 10,
158–169.
*Schoener, E. P., Madeja, C. L., Henderson, M. J.,
Ondersma, S. J., & Janisse, J. (2006). Effects of motiva-
tional interviewing training on mental health therapist
behavior. Drug and Alcohol Dependence, 82, 269–275.
*Seng, A., Prinz, R., & Sanders, M. (2006). The role of
training variables in effective dissemination of evidence-
TRAINING IN EVIDENCE-BASED PRACTICE: A REVIEW • BEIDAS & KENDALL 29
based parenting interventions. International Journal of Mental
Health Promotion, 8, 19–27.
*Sholomskas, D. E., Syracuse-Siewert, G., Rounsaville, B. J.,
Ball, S. A., & Nuro, K. F. (2005). We don’t train in vain:
A dissemination trial of three strategies of training clini-
cians in cognitive behavioral therapy. Journal of Consulting
and Clinical Psychology, 73, 106–115.
Silverman, W. K., Pina, A. A., & Viswesvaran, C. (2008).
Evidence based psychological treatment for phobic and
anxiety disorders in children and adolescents. Journal of
Clinical Child and Adolescent Psychology, 37, 105–130.
*Siqueland, L., Crits-Christoph, P., Barber, J. P., Butler,
S. F., Thase, M., Najavits, L., et al. (2000). The role of
therapist characteristics in training effects in cognitive,
supportive expressive, and drug counseling therapies for
cocaine dependence. Journal of Psychotherapy Practice &
Research, 9, 123–130.
Stirman, S. W., DeRubeis, R. J., Crits-Christoph, P., &
Rothman, A. (2005). Can the randomized controlled trial
literature generalize to non-randomized patients? Journal of
Consulting and Clinical Psychology, 73, 127–135.
Turner, K., & Sanders, M. (2006). Dissemination of evidence-
based parenting and family support strategies: Learning from
the Triple P—Positive Parenting Program system approach.
Aggression and Violent Behavior, 11, 176–193.
United States Surgeon General. (1999). Mental health: A report
of the surgeon general. Washington, DC: U.S. Government
Printing Office.
Walkup, J., Albano, A. M., Piacentini, J., Birmaher, B.,
Compton, S., Sherrill, J., et al. (2008). Cognitive behavioral
therapy, sertraline, or a combination in childhood anxiety.
New England Journal of Medicine, 359, 2753–2766.
*Watkins, K., Osilla, K., Hepner, K., Sandres, P., &
Thompson, J. (2008, February). Project BRIGHT: Build-
ing recovery by improving goals, habits and thoughts.
Symposium conducted at the Co-occurring Disorders
Conference.
Wenzel, S., Ebener, P., Hunter, S., Watkins, K., & Gilmore,
J. (2005). Research-practice partners assess their first joint
project. Science & Practice Perspectives, 3, 38–45.
Westen, D., Novotny, C., & Thompson-Brenner, H. (2004).
The empirical status of empirically supported psychothera-
pies: Assumptions, findings, and reporting in controlled
clinical trials. Psychological Bulletin, 130, 631–663.
*Wyman, P., Brown, C., Inman, J., Cross, W., Schmeelk-
Cone, K., Guo, J., et al. (2008). Randomized trial of a
gatekeeper program for suicide prevention: 1-year impact
on secondary school staff. Journal of Consulting and Clinical
Psychology, 76, 104–115.
Zazzali, J., Sherbourne, C., Hoagwood, K., Greene, D.,
Bigley, M., & Sexton, T. (2008). The adoption and
implementation of an evidence based practice in child and
family mental health service organizations: A pilot study of
functional family therapy in New York State. Administra-
tion and Policy in Mental Health and Mental Health Services
Research, 35, 38–49.
*References marked with an asterisk indicate studies included
as reviewed articles.
Received February 14, 2009; revised June 8, 2009; accepted
June 10, 2009.
CLINICAL PSYCHOLOGY: SCIENCE AND PRACTICE • V17 N1, MARCH 2010 30
Copyright of Clinical Psychology: Science & Practice is the property of Blackwell Publishing Limited and its
content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's
express written permission. However, users may print, download, or email articles for individual use.