4503
l_8986
journal_a_article_3.pdf
An Opportunity to Bridge the Gap Between Clinical Research and Clinical Practice: Implications for Clinical Training
Rachel Hershenberg Stony Brook University
Deborah A. G. Drabick Temple University
Dina Vivian Stony Brook University
Clinical researchers and clinical practitioners share a goal of increasing the integration of research and clinical practice, which is reflected in an evidence-based practice (EBP) approach to psychology. The EBP framework involves the integration of research findings with clinical expertise and client charac- teristics, values, and preferences, and consequently provides an important foundation for conducting clinically relevant research, as well as empirically based and clinically sensitive practice. Given the critical role that early training can play in the integration of science and practice and in promoting the future of the field, the present article addresses predoctoral training programs as a context for adopting an EBP approach to clinical work. We address training in the three components of EBP and provide suggestions for curriculum development and practicum training that we hope will contribute to bridging the gap between research and practice.
Keywords: evidence-based practice, clinical training, scientist-practitioner, clinical scientist, supervision
The setting that is perhaps best poised for achieving the lofty but essential goal of bridging the gap between research and practice is the training setting. Although our discussion may touch on is- sues that are relevant to other levels of clinical training (e.g., predoctoral internships, postdoctoral fellowships, and continuing education programs), we focus on predoctoral training as it repre- sents the first step in the clinical training progression, and it provides an ideal milieu for the integration of science and practice. Indeed, particularly in PhD graduate programs dedicated to the training of scientist–practitioners and clinical scientists, the goals are to teach, model, and provide hands-on experience in two key activities: clinical research and clinical practice. The training set- ting is thus replete with opportunities to seamlessly integrate practice and research. For example, psychology training clinics are often housed in the same setting in which basic coursework is provided, and faculty are often participating in their own research activities, as well as providing supervision. A curriculum that encourages the trainee to consume and produce basic and applied research, and a clinical practicum that integrates and translates this information, is likely to promote evidence-based practice (EBP) in external clinical settings, provide suggestions for clinically rele-
vant research, and, consequently, shape the nature of the trainee’s professional development after graduate school. In fact, we hy- pothesize that trainees whose early clinical and research experi- ences embody the integration of science and practice are likely to adopt and maintain this approach as they progress through subse- quent stages of professional development.
In the present article, we use the EBP framework to identify and recommend training opportunities that can bridge clinical practice and clinical research. EBP involves the integration of three pri- mary components: the best research evidence; clinical expertise; and client characteristics, values, and preferences (Collins, Leff- ingwell, & Belar, 2007; Hunsley, 2007; Spring, 2007). Thus, EBP is a rather broad construct and is not synonymous with empirically supported treatments (ESTs), although ESTs are one type of re- search evidence considered within the EBP framework. Address- ing each of these EBP components in training is no small task, and we recognize that there is some uncertainty about what training in EBP should entail (e.g., Bauer, 2007; Beidas & Kendall, 2010). Nevertheless, we believe that this framework (a) allows more in-depth and careful consideration of critical elements relevant to clinical practice that can improve the quality of services we pro- vide and (b) highlights potential pitfalls and opportunities for scaffolding and improving the research-practice bridge in psychol- ogy training settings.
The present article is organized into three sections reflecting each of the EBP components: research evidence, clinical expertise, and client characteristics. Throughout, we consider issues, high- light recent advances and gaps in the literature, and provide prac- tical recommendations related to curriculum development and supervision, as well as issues related to practicum performance, including the acquisition of clinical competencies and client re- sponse to treatment (Falender et al., 2004). Though PhD programs
Rachel Hershenberg, Department of Psychology, Stony Brook Univer- sity; Deborah A. G. Drabick, Department of Psychology, Temple Univer- sity; and Dina Vivian, Department of Psychology, Stony Brook University.
We would like to thank Marv Goldfried, Bethany Teachman, and Barry Wolfe for their helpful comments on earlier versions of this manuscript. Preparation of this manuscript was supported in part by NIMH 1K01 MH07317-01A2 awarded to Dr. Drabick.
Correspondence concerning this article should be addressed to Rachel Hershenberg, Department of Psychology, Stony Brook University, Stony Brook, NY 11794. E-mail: [email protected]
Psychotherapy © 2012 American Psychological Association 2012, Vol. 49, No. 2, 123–134 0033-3204/12/$12.00 DOI: 10.1037/a0027648
123
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
prepare students to function as both clinicians and researchers, we focus on suggestions most pertinent to clinical training, recogniz- ing, of course, that preparedness for one can and does reinforce preparedness for the other. We believe that training programs are an ideal place for instilling skills in clinical research and clinical practice in trainees, the future of our field, by creating opportuni- ties for their synergy. We agree with others (Bauer, 2007; Collins et al., 2007; DiLillo & McChargue, 2007; Hunsley, 2007; Spring, 2007) that an evidence-based approach to clinical training can provide the foundation for bridging the gap between clinical prac- tice and research. In the following sections, we provide recom- mendations within the EBP framework to meet this important but challenging goal. A summary of our training suggestions is pre- sented in Table 1.
Considering “Best” Research Evidence
Skills to Approach the Ever-Changing Evidence
Among the three components of EBP, we believe that many predoctoral training programs are perhaps most successful at pro- viding coursework that includes “best” research evidence that informs clinical practice (e.g., Spring, 2007). However, there are likely gaps in the curriculum that should be considered to permit students to best use the EBP framework to inform their clinical practice and clinical research efforts. For example, several authors (Bauer, 2007; Collins et al., 2007; DiLillo & McChargue, 2007; Drabick & Goldfried, 2000; Spring, 2007) have recommended training in clinical and research methods, as well as epidemiology, clinical trials and qualitative methodology, and research skills that include secondary sources, systematic reviews, and informatics (e.g., resources, guidelines, electronic records) to enable students to find the best available evidence for the problems encountered in their clinical (and research) work. Indeed, we believe that a critical skill for EBP training is learning how to obtain and integrate the best available evidence.
This deceptively simply stated goal is in fact quite complex. For example, it can be difficult to identify, synthesize, and critically evaluate the evidence relevant to client’s presenting problems because of rapid changes in available evidence; differences in terms used to describe clinical phenomena (e.g., impulsivity may be construed as poor executive control, difficult temperament, sensation-seeking, risk taking, or attention-deficit/hyperactivity disorder); the absence of translational efforts that facilitate relevant work across disciplines; and the ever-expanding information avail- able on the World Wide Web. Thus, we believe that training in clinical research must include strategies for identifying and criti- cally evaluating a variety of sources (Collins et al., 2007; Falender et al., 2004; Falzon, Davidson, & Bruns, 2010; Spring, 2007). Students are often taught to value primary sources of information (e.g., empirical studies, theoretical reviews, practice guidelines) above other sources (e.g., secondary sources that are summaries of primary sources, including both qualitative and quantitative [meta- analytic] reviews). Though primary sources of information are valuable, we recommend that students learn to approach and integrate multiple lines of evidence, including, but not limited to, primary sources.
Given expected changes in available evidence over time, train- ing should include strategies for reviewing and critically evaluat-
ing the constantly evolving literature base relevant to clients’ presenting issues. Alongside this goal, students should be exposed to the literature regarding clinical decision-making and potential heuristics and biases that influence such decision-making (Dawes, Faust, & Meehl, 1989; Hunsley, 2007; Swets, Dawes, & Monahan, 2000). Although we know that this is a typical component of assessment training, we believe that opportunities to practice and examine these heuristics through supervision, case consultations, and case conference presentations would be useful for evaluating the effects of potential biases on clinical case management and the development of skills to search for and weigh the best evidence. Overall, this interactive approach is consistent with the EBP prin- ciple of lifelong learning and provides an important framework for systematically approaching the ever-changing evidence base (Spring, 2007). Further, we believe that this proposed approach to training facilitates the use of clinical research to appropriately inform clinical practice (Bauer, 2007; Goldfried & Wolfe, 1996; Stricker, 1992).
What Constitutes “Evidence”
A related issue concerns what evidence students are taught to value and approach. Training in ESTs is often viewed as one major source of evidence and, consequently, a major aspect of clinical training (Calhoun, Moras, Pilkonis, & Rehm, 1998). However, there are other sources of evidence from which students can draw relevant clinical information. In addition, other approaches to psychology training integrate many of these EBP domains and principles and can illustrate a possible approach to implement- ing the EBP framework in clinical training settings. Develop- mental psychopathology, for example, provides an alternative, complementary framework for training that considers how to not only conceptualize the “best research evidence,” but also develop clinical expertise and take client characteristics into account. Here we briefly address the contribution of develop- mental psychopathology as a heuristic framework to obtain the best evidence.
The developmental psychopathology perspective is a framework that many psychologists use for understanding clinical issues across the life span. The developmental psychopathology perspec- tive considers issues such as risk and resilience (i.e., thriving despite adversity), developmental pathways or course, contextual influences, determination of typical and atypical behaviors based on developmental demands, and translational efforts that can in- form prevention and intervention efforts (Drabick & Kendall, 2010; Drabick & Steinberg, 2011; Hart & Marmorstein, 2009; Jensen & Hoagwood, 1997; Rutter & Sroufe, 2000; Steinberg & Avenevoli, 2000). Both the EBP and developmental psychopathol- ogy frameworks recommend considering multiple lines of evi- dence drawn from different research approaches and across differ- ent biological, psychological, and social domains, as well as gathering convergent and divergent evidence for hypotheses and case conceptualizations. Thus, the idiographic, client-centered ap- proach used by both the EBP and developmental psychopathology frameworks provides implications for curriculum development in training programs. Specifically, this approach requires coursework that addresses (and teaches students to approach) biological, cog-
124 HERSHENBERG, DRABICK, AND VIVIAN
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
Table 1 Summary of Recommendations for an Evidence-Based Practice (EBP) Training Program
Considering “Best” Research Evidence
Skills to approach the ever-changing evidence • Coursework includes “best” research, clinical and research methods, epidemiology, clinical trials, and qualitative methodologies • Research skills to identify, critically evaluate, and integrate multiple lines of evidence drawn from biological, psychological, and social domains
that are relevant to a client’s presenting problems • Skills to consult primary and secondary sources, systematic reviews, and informatics • Clinical decision-making skills, potential heuristics, and biases
What constitutes evidence and obtaining clinical evidence • EBP research including but not limited to empirically supported treatments (ESTs) • Curriculum that emphasizes client-centered research including biological, emotional, developmental, contextual, and cultural domains • Evidence-based clinical assessment skills for initial and ongoing assessments • Integrate client variables and relevant literature to develop case conceptualization and “principle-informed” treatment plan • Link ongoing standardized outcome assessments and alliance monitoring to treatment implementation, clinical-decision making, and supervision • Systematic outcome assessment is shared with the client and collaborative client feedback is used as a vehicle for change • Practicum includes client-centered research with clients in psychology training clinics (e.g., controlled single case studies, time-series analyses) • Data collected in psychology training clinics are used in group-based outcome studies that include benchmarking methodologies to assess
treatment effectiveness • Training programs establish an outcome-based research infrastructure that is integrated in clinical practicum
Developing Clinical Expertise Treatment outset
• Socialize clients toward their role in therapy and foster realistic expectations for change • Promote a hopeful stance toward change and share potential explanations for the presenting problems with the client • To decrease premature termination, build a strong alliance early in treatment and engage in a collaborative effort to promote realistic expectation
for rate of change Case conceptualization and selection of interventions
• Identify common change strategies across different theoretical models and promote a transdiagnostic approach to treatment • Principle-oriented conceptualization skills promote flexible application of therapeutic strategies • Provide trainees with systematic methods of case conceptualization and treatment planning (e.g., via evidence-based assessment and treatment,
reliance on basic and outcome literature, and evaluation of reliability and validity of case formulations and treatment implementation) The therapeutic relationship, therapist emotions, and therapist variables
• Promote a dialectical approach to the therapeutic relationship and the clinical techniques, and focus on both and their interaction to promote change
• Increase trainees’ ability to identify and use their own in-session emotional responses to promote client change and strengthen the alliance • Attend to and foster trainee’s qualities that are related to a positive alliance (e.g., flexibility, honesty, alertness, warmth) • Teach trainees to monitor therapist and relationship variables via standardized measures, observational methods (e.g., videotaping and critically
self-reviewing sessions), and feedback from supervisors Supervision: From clinical experience to clinical expertise
• Supervision can benefit supervisees with regard to treatment knowledge, skills acquisition, self-efficacy, and therapeutic alliance • Harmful effects of supervision (e.g., invalidation, poor communication) need attention • Supervisors should model and foster a stance of empirical curiosity in their trainees (e.g., encourage literature review, hypothesis testing, use of
data-driven case-conceptualization and treatment implementation) rather than assuming the role of “experts” • Empirical evaluation of consensus-derived competency-based models of supervision (e.g., reliability and validity) in an EBP framework is sorely
needed • Multimethod determination and observation of trainees’ clinical competencies are desirable • Client outcome data should be used as one index of therapist, as well as supervisor, competence and effectiveness • Absolute and relative ratings of trainees competencies should be provided • Inter-rater reliability of competency ratings should be evaluated • Supervision sessions should be videotaped and a subset rated by external raters for content validity
Relevant Client Characteristics • Trainees should consider the multifaceted evidence and cultural domains that may influence clients’ functioning, course, and response to
intervention beyond DSM-IV-based nosologies • Knowledge of the characteristics of targeted populations (e.g., co-morbidity of psychiatric problems) can inform trainees’ assessment skills and
treatment planning • Clients’ dynamic patterns of interpersonal functioning need to be incorporated into case conceptualizations • Clients’ characteristics could be used to determine which treatment or components of treatment may be appropriate for a particular client and in
which circumstances • Collaboration between trainee and client in developing and implementing the treatment plan ensures sensitivity and responsiveness to the client’s
values, preferences, and readiness to change • Sharing outcome ratings with clients on an ongoing basis (e.g., with standardized instruments) increases a collaborative approach to change and
may increase clients’ engagement in treatment • Systematically tracking clients’ response to therapy allows comparisons with established clinically significant recovery curves in outpatient
settings • Information about pre–post therapy changes using standardized instruments allows for estimating effect sizes benchmarked against controlled trials
125GRADUATE TRAINING PROGRAMS
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
nitive, emotional, developmental, contextual, and cultural domains that can influence individuals’ functioning, and thus the trainees’ assessments, case conceptualizations, and interventions.
Obtaining Evidence With Ongoing Assessment Data
Training in clinical assessment is a well-integrated, core com- ponent of most psychology training programs. However, we be- lieve that two related areas may require additional attention to realize a more effective link between clinical practice and clinical research. First, it is unclear to what extent initial assessment findings influence case conceptualizations and related decisions about prognosis, prevention, and intervention (Bauer, 2007; Col- lins et al., 2007; Hunsley, 2007). Second, ongoing assessment to monitor client progress and the therapeutic alliance is important but receives less attention in training than initial clinical assess- ment (Borkovec, 2004; Collins et al., 2007; Hunsley, 2007). We believe that the use of systematic and ongoing assessment during treatment is critical to consider within an EBP framework; indeed, perhaps no aspect of the curriculum is better suited to model the synergy of science and practice than the use of ongoing assessment as a source of evidence. Broadly, training should highlight and illustrate strategies for linking ongoing assessment information to clinical intervention, particularly to determine whether changes in the intervention approach are needed (Lambert, in press; Youn, Kraus, & Castonguay, in press), and because inclusion of ongoing assessment information in treatment (e.g., providing feedback to clients) can be used to promote change (Harmon, Hawkins, Lam- bert, Slade, & Whipple, 2005).
Ongoing assessment information can be used in numerous ad- ditional ways. First, we encourage programs to formally integrate ongoing assessment data into client-centered research (e.g., con- trolled single case studies, time-series analyses) as part of the clinical practicum training. Students may conduct their own re- search that tracks symptom status of one or a few psychotherapy clients across baseline and intervention phases (Borckardt et al., 2008). Second, students can use intake and ongoing assessment data (e.g., depressive and anxious symptoms, hypothesized medi- ators such as dysfunctional attitudes) to examine group-level changes from pre- to posttherapy (i.e., effectiveness data), which can be qualitatively compared with efficacy data from other re- sources, such as randomized clinical trials, or quantitatively through benchmarking methods (e.g., Minami et al., 2009). Data also can be collected to examine client and therapist characteris- tics, as well as process issues, that may be associated with positive outcomes (Borkovec, 2002; Gard, Tremblay, DiLillo, & Pantesco, 2002). Recent efforts to promote this research infrastructure in training programs have included establishing faculty-student re- search teams to generate ideas for research, ensure that human subjects’ approval is obtained, and monitor ongoing research proj- ects conducted in psychology training clinics, including mainte- nance of relevant outcome data (Sauer & Huber, 2007). We believe that actively participating in this research (a) teaches trainees (and clients) to operationalize the variables that clients want to change, (b) enhances trainees’ ability to develop a meth- odology to track change over time in conjunction with their clinical observational and interviewing skills, and (c) promotes case con- ceptualization as a set of testable hypotheses. Finally, we encour- age the incorporation of client-centered research into practicum
because it provides the trainee with the opportunity to sit in two chairs (i.e., those of the researcher and clinician) simultaneously (Wolfe, in press).
In sum, there is a plethora of research that informs clinical practice. Given the speed with which this research base transforms, we believe that one of the most critical aspects of this EBP component is providing trainees with methodologies to approach and incorporate this information into their treatment, including the use of ongoing literature searches involving a variety of sources and collection of data with clients.
Developing Clinical Expertise
Although we recognize that clinical experience is an important foundation of clinical expertise, clinical expertise is a much more complicated construct than experience, and requires supervision, knowledge of current research, skill in developing and maintaining the therapeutic alliance, as well as assessing and treating clients, among other abilities (e.g., Collins et al., 2007). Several of these aspects of clinical expertise are addressed through training in ESTs (Calhoun et al., 1998). However, a consequence of the EST move- ment is the proliferation of manuals, particularly individual man- uals for individual disorders. A discussion of the rationale, advan- tages, and disadvantages of manuals is beyond the scope of this article (see, e.g., Addis, Cardemil, Duncan, & Miller, 2006). Nev- ertheless, it is important to recognize potential concerns that have been raised regarding training in ESTs. For example, manuals often prescribe overlapping strategies for change and there is little empirical basis to guide efforts to combine or select among par- ticular ESTs (Chorpita, Daleiden, & Weisz, 2005). Related issues involve concerns about cognitively overloading trainees and the importance of thinking flexibly about the methods with which to attain the goals of an intervention, given the heterogeneity of client variables. Thus, in addition to training in the “treatment method” that is supported by ESTs (Chambless et al., 2006), the following section includes recommendations to train students in factors im- portant in therapy implementation from treatment outset through the course of treatment (irrespective of specific treatment method), to learn to approach case conceptualizations with flexibility that are guided by principles, to attend to the therapeutic relationship, to promote certain therapist variables, and to manage and make use of the trainee’s own emotions.
Treatment Outset
Core processes that cut across theoretical orientation and treat- ment modality, common factors, are an important aspect of clinical training (Spring, 2007), and the presence of these factors at the outset of therapy are positively associated with outcome. For example, as reviewed in DeFife and Hilsenroth (2011), trainees should learn to foster realistic and positive expectations in the client, to socialize the client to his or her role in treatment (role preparation), and to engage in a collaborative formulation regard- ing presenting problems and treatment goals (see also Hilsenroth & Cromer, 2007, for key behaviors associated with alliance for- mation during the assessment phase of therapy). These elements dovetail with suggestions put forth by Wampold (2007) that the most critical elements of treatment involve offering the client a more functional explanation for his or her problems and related set
126 HERSHENBERG, DRABICK, AND VIVIAN
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
of actions to ameliorate those problems, provided by a therapist who delivers that treatment with the expectation (e.g., based on the literature, clinical expertise, etc.) that it will be effective. Address- ing the alliance at the outset of treatment may be particularly important in a training setting, given that attrition rates in psychol- ogy training clinics can be as high as 77.5% (Callahan, Aubuchon- Endsley, Borja, & Swift, 2009; Callahan & Hynan, 2005), com- pared with rates of about 40% to 60% in community outpatient clinics (Clarkin & Levy, 2003; Garfield, 1994). Notably, a signif- icant portion of the variance in premature termination in psychol- ogy training clinics may be accounted for by clients’ unrealisti- cally high pretreatment expectations about the rate and speed of recovery (Callahan, Aubuchon-Endsley, et al., 2009; Swift & Callahan, 2008a), and evidence suggests that helping clients to acquire more realistic expectations about the degree of improve- ment that they can expect across sessions substantially decreases premature termination rates (Swift & Callahan, 2008b). Taken together, regardless of the predominant theoretical orientation, it is important that trainees learn to orient their clients to therapy by addressing, and providing brief education regarding, these essen- tial elements that set the stage for the therapeutic process.
Case Conceptualization and Selection of Intervention
Because there is a limit to the amount of information that anyone can retain, and because the inundation of individual ESTs has led to many “treatments that work” (Minami et al., 2009; Nathan & Gorman, 1998; Stiles, Barkham, Mellor-Clark, & Con- nell, 2007), we do not recommend encouraging trainees to read and apply ESTs in isolation. Instead, there are a variety of ad- vances in the field that augment training in ESTs, including (a) methodologies to identify shared techniques of empirically sup- ported interventions (e.g., interpersonal psychotherapy and cogni- tive behavioral therapy both include problem solving and activity scheduling; Chorpita et al., 2005); (b) transdiagnostic approaches to treatment (e.g., Allen, McHugh, & Barlow, 2008); and (c) delineation of common change processes (e.g., principles of change; Castonguay & Beutler, 2006). First, identifying the most common techniques across validated treatments potentially can provide an empirically informed starting point, though this re- search is still in its infancy and the utility of this approach remains a question deserving of further empirical scrutiny. Second, training with transdiagnostic approaches can increase broad conceptualiza- tion skills and generalizability of particular change strategies, enabling trainees to identify and treat core deficits that are com- mon across diagnoses (e.g., enhancing emotion regulation and present-focus awareness skills). Finally, learning to conceptualize therapy goals from a lens of common principles of change that cut across treatment type may help to instill trainee confidence (i.e., reducing the focus on the delivery of one specific technique and broadening the focus to the goal of the intervention), which may be particularly useful for individuals who do not respond to the first change strategy used.
Indeed, we believe it is paramount to provide training in com- mon change processes so that trainees learn to think with flexibil- ity about the methods with which to attain the goals of an inter- vention. For example, one strategy is to teach trainees to conceptualize their clinical cases from a broader principles ap- proach (e.g., Bauer, 2007; Helge Rønnestad & Ladany, 2006), an
approach that is pertinent at treatment outset (assessment), as well as throughout the course of therapy (intervention). Principles of change refer to a level of abstraction that is more specific than theory but more general than the strategies or techniques (Caston- guay & Beutler, 2006), and have been delineated broadly (Gold- fried, 1980; Weinberger, 1995), as well as in reference to specific problem domains (e.g., depression, anxiety, personality disorders) that involve change processes pertinent to the relationship, client, and treatment model (Castonguay & Beutler, 2006). This argument is similar to suggestions to train in the “underlying spirit” of a treatment rather than overemphasize techniques (Miller, Yahne, Moyers, Martinez, & Pirritano, 2004).
Training to conceptualize from a principle approach encourages greater flexibility in choosing a technique to facilitate a particular change process (Boswell, Nelson, Nordberg, Mcaleavey, & Cas- tonguay, 2010) (e.g., “devil’s advocate” or “two-chair technique” to increase awareness of critical voices that prompt rumination). Though the principle (e.g., increasing clients’ awareness about factors contributing to their life problems; Goldfried, 1980) may provide a suggestion for a technique consistent with the theoretical orientation of the trainee/supervisor initially, the willingness to engage in functional equivalence (that is, turning to an alternative technique to attain the same treatment goal) may reduce the possibility of strains to the alliance. Indeed, thinking flexibly about an intervention may reduce a clinician’s frustration related to his or her client’s ambivalence or resistance, enabling the clinician to think creatively about the problem to be solved rather than con- tinually attempting to inflexibly apply the same technique. This is an important goal, given that inflexible application of therapeutic strategies reduces the therapeutic alliance (e.g., responding to strains in the alliance by further attempts to persuade the client of the validity of the treatment rationale; Ackerman & Hilsenroth, 2001; Castonguay, Goldfried, Wiser, Raue, & Hayes, 1996; Henry, Strupp, Butler, Schacht, & Binder, 1993).
To aid in the development of flexible, empirically informed, and principle-oriented conceptualizations, we recommend training in methodologies to develop case conceptualization skills (Eells, 2007; Kuyken, Padesky, & Dudley, 2009). Indeed, case concep- tualization skills, considered a “core competency” in psychother- apy (Page & Stritzke, 2006), are at the heart of training practitio- ners in EBP, as conceptualization sets the stage for how therapy will proceed and should be the roadmap to which the clinician refers throughout treatment. To bridge the gap between research and practice, we encourage graduate training programs and stu- dents’ supervisors to provide trainees with a methodology for approaching a conceptualization and treatment plan in a systematic way (Page & Stritzke, 2006; Page, Stritzke, & McLean, 2008). Not only does this foundation involve learning to provide evidence- based assessments (Hunsley & Mash, 2007; Mash & Hunsley, 2005), conducting systematic literature searches (Falzon et al., 2010), extrapolating from findings (DiLillo & McChargue, 2007), and incorporating client variables into a principle-oriented concep- tualization and treatment plan, but importantly, evaluating whether case formulations are reliable and valid. Page et al. (2008) provide an interesting illustration of how supervision can facilitate inte- grating science-informed clinical case formulation into graduate training. Specifically, the authors developed case vignettes with assessment data; identified requisite conceptualization domains to cover (e.g., problem list, predisposing factors, maintaining factors,
127GRADUATE TRAINING PROGRAMS
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
problems potentially hindering treatment, strengths, and assets); and created a scoring system for rating students’ conceptualiza- tions based on advanced undergraduate students, fellow trainees, and experienced clinicians to enable comparisons with various developmental levels. Their goal is to create a standard set of videotaped interviews with clients with different psychological problems that vary in severity so that trainees can practice devel- oping reliable case conceptualizations, compare their skills to benchmark ratings to identify areas of strength and weakness, and test the validity of these ratings by examining associations between benchmarking scores and treatment outcomes. Evidence that case conceptualization skills improve with training is encouraging (Kendjelic & Eells, 2007); moreover, such research on case con- ceptualization illustrates another application of an evidence-based approach to clinical practice and research in training.
The Therapeutic Relationship
Like case conceptualization, the therapeutic relationship is a ubiquitous aspect of therapy and accounts for roughly 10% of the variance in treatment outcome (Chambless et al., 2006) and thus needs to be addressed in training. Within a principle-oriented approach, the establishment of a relationship is a necessary pre- requisite for conducting therapy (Goldfried, 1980; Weinberger, 1995), and at times, attending to the “here and now” of the relationship itself may be part of the intervention. Rather than engage in “either/or” dichotomous thinking— do we teach the relationship or the technique—we believe that the answer neces- sitates a dialectical stance; as cogently argued by Goldfried and Davila (2005), it is not the technique or the relationship but rather, both and their interaction. When to place a greater emphasis on the relationship or the technique depends on the conceptualization and the moment-to-moment interaction. Taking a dialectical stance thus involves monitoring the alliance and repairing alliance rup- tures. Toward these ends, we encourage trainees to learn to mon- itor and quantitatively assess the alliance (consistent with our recommendation for ongoing assessment; for example see Duncan et al [2003], 4-item Working Alliance Inventory), and obtain supervision in repairing alliance ruptures (Safran & Muran, 2000; Safran, Muran, Samstag, & Stevens, 2001). Indeed, Miller, Dun- can, and colleagues (2006) found that including and addressing clients’ responses to two four-item measures pertaining to the alliance and progress in treatment doubles the effect size of “treat- ment as usual.” Thus, these data suggest that monitoring clients’ progress and the therapeutic relationship is related to more positive outcomes than not monitoring these issues.
Therapist Emotions
Another major aspect of maintaining and repairing the alliance is the ability to detect and make use of one’s own (potentially negative) emotions in session (Binder & Strupp, 1997; Mc- Cullough, 2000; Wolf, Goldfried, & Muran, 2012). Though space limitations prohibit a full discussion, we suggest that helping the trainee to understand the function of these emotions is critical. On the one hand, negative emotions may be idiosyncratic to the trainee and therefore empathy development is needed (e.g., from frustra- tion to compassion). On the other hand, one’s affective reaction may be directly tied into the client’s presenting problems, in which
case it is important to receive supervision that enables the trainee to appropriately make use of this response as part of the interven- tion itself (McCullough, 2000), which may be a vehicle to promote change and/or to strengthen the alliance. Finally, in addition to skillfully and planfully making use of their negative emotional response, trainees also need to become aware and make use of their positive emotions in a planful way (e.g., reinforcing client’s suc- cess by celebrating new learning). Besides promoting change, this type of process work can raise the client’s awareness about the impact that he or she has on others, which may decrease his or her feelings of helplessness and social disconnection (McCullough, 2000, 2006).
Therapist Variables
In addition to attention to emotions during sessions, training of evidence-based practitioners should involve the exploration of therapist variables, which account for roughly 6% to 9% of vari- ance in treatment outcome (Addis et al., 2006; Kim, Wampold, & Bolt, 2006). Because therapy is contextual and modified to fit the particular client, therapist qualities such as being flexible, honest, alert, and warm, which are associated with the alliance (Ackerman & Hilsenroth, 2001, 2003), should be modeled and supported. It is further important to address the trainee’s belief that the therapeutic approach is viable (Kendall & Beidas, 2007), which is also con- sistent with Wampold’s (2007) postulation that therapist’s expec- tation that the therapy has value is a necessary ingredient in promoting change.
Both therapist and relationship variables can be assessed via observation and with standardized instruments (Beidas & Kendall, 2010), the combination of which can protect against common cognitive biases (e.g., favoring information that confirms hypoth- eses about the trainee or client), highlight clinically relevant areas that may not emerge from one strategy alone, and provide data to use in the context of research. Thus, an EBP approach indicates that trainees should learn to develop and monitor the therapeutic alliance and relevant therapist variables using observation, stan- dardized measures, and feedback from supervision, and to inte- grate these multiple sources of evidence into an ongoing case conceptualization and treatment plan throughout the course of treatment. In addition, coursework can be used to teach these recommended approaches for developing clinical expertise, such as an intervention course for teaching principles of change and relevant client and therapist variables, as well as a Clinical Practi- cum for teaching approaches to case conceptualization and the development of the therapeutic alliance, consistent with arguments to promote learning with the use of multiple methods (blended learning; Cucciare, Weingardt, & Villafranca, 2008). Supervision is also a major outlet in which to model and teach these evidence- based activities that can move students from clinical experience to clinical expertise.
Supervision
The primary roles of a clinical supervisor are to (1) ensure that treatment services are implemented effectively, ethically, with integrity, with appreciation of diversity, and informed by empirical evidence; and (2) develop the trainee’s clinical competencies (Fal- ender & Shafranske, 2004). Supervision provides an excellent
128 HERSHENBERG, DRABICK, AND VIVIAN
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
opportunity for “active” learning (e.g., role plays, coaching, feed- back) that are advantageous over more “passive” learning strate- gies (e.g., didactic lectures) (Beidas & Kendall, 2010; Drabick & Goldfried, 2000). Indeed, the case-specific mentoring that occurs in supervision provides an excellent opportunity to strengthen the skill set, and spirit, of EBP, particularly by integrating research into practice, as well as creating opportunities for practice-based research.
Although emerging as a growing area of interest, supervision has been relatively neglected empirically (Reiser & Milne, in press; Watkins, 2011). This dearth of attention may stem, in part, from lack of formal training in models of supervision among most supervisors, which perpetuates the notion that supervision is more of an art than a science (Falender & Shafranske, 2004; Kavanagh et al., 2003; Scott, Ingram, Vitanza, & Smith, 2000). The accu- mulating data are encouraging, as the evidence for the validity of supervision (i.e., that it leads to expected outcomes) is growing (Helge Rønnestad & Ladany, 2006; Watkins, 2011). Indeed, em- pirical research demonstrates that supervision can benefit super- visees, particularly with regard to knowledge of treatment, skill acquisition, self-efficacy, supervisee’s therapeutic alliance (Hilsenroth, Defife, Blagys, & Ackerman, 2006; Watkins, 2011), and client outcomes such as decreased dropout rates, working alliance, and symptom reduction (Bambling, King, Raue, Schweitzer, & Lambert, 2006; Callahan, Almstrom, Swift, Borja, & Heath, 2009; Stein & Lambert, 1995). Further, research has identified potentially harmful aspects of supervision, including dismissing supervisees’ ideas and emotions (Gray, Ladany, Walker, & Ancis, 2001) and ignoring or avoiding responsibility for behaviors that lead to supervisor-supervisee conflict (Nelson & Friedlander, 2001). These findings likely parallel the goals of therapeutic change (parallel process; see Helge Rønnestad & Ladany, 2006) and consequently reflect similar processes such as the centrality of the supervisor-supervisee relationship, raising awareness of problematic supervisee behaviors, creating opportu- nities for corrective experiences, managing negative emotions effectively, and promoting skill acquisition and autonomy devel- opment.
Recent advances also have been made with regard to supervi- sory assessments of trainee competence, which is one vehicle for evaluating students’ development of expertise. Although compe- tency assessments were previously somewhat idiosyncratic and program specific, recent efforts resulted in more systematic at- tempts to derive consensus models of competence-based training. In fact, together with other training councils, the Association of Psychology Postdoctoral and Internship Centers cosponsored a conference in 2002 examining competencies in professional psy- chology, leading to a systematic effort to define competencies across key domains and developmental levels (Falender et al., 2004; Rodolfa et al., 2005). Particularly for predoctoral programs, major advantages of this approach include a de-emphasis on some- what arbitrarily defined “clinical hours” and increased standard- ization across programs (Cellucci & Reports, 2010). Such stan- dardization also provides a basis from which to conduct research on the reliability and validity of these ratings (Rings, Genuchi, Hall, Angelo, & Cornish, 2009). This approach provides ample guidance for research efforts, though much work remains to be done. Nevertheless, the EBP framework provides a useful foun- dation for conceptualizing gaps in our knowledge of supervision
and supervisory assessments and for conducting research to ad- dress these gaps.
Not only does the evidence base with regard to supervision demonstrate advances in the field that bridge the gap between science and practicum activities, but, most germane to this paper, provide methodologies to enhance an EBP approach within indi- vidual supervisory relationships. For example, Hilsenroth, DeFife, and colleagues (2006) demonstrated that trainees significantly increased use of psychodynamic-interpersonal techniques over the course of treatment with ongoing psychodynamic supervision, vis-à-vis independent clinical ratings made with the Comparative Psychotherapy Process Scale (CPPS; Hilsenroth, Blagys, Acker- man, Bonge, & Blais, 2005), a 20-item descriptive measure de- signed to assess therapist activity and techniques. This type of work is important, as it demonstrates objectively rated increases in competence in a particular set of behaviors and can be adapted within an individual supervisory relationship. For example, to encourage trainees to think about the “internal validity” of their treatment, increase awareness of their actual (in contrast to per- ceived) behavior, and provide a more objective way of assessing competence, an exercise could involve both supervisor and super- visee rating the trainee’s audiotaped or videotaped session using the CPPS to compare their understanding of the interventions used in session. We hypothesize that as training increases, (a) scores will increase on the CPPS, indicating greater use of characteristic aspects of the therapeutic orientation selected; (b) the interrater reliability between trainee self-assessment and supervisor assess- ment will increase; and (c) opportunities to test whether increasing CPPS scores and interrater reliability are associated with improve- ment in client outcome (validity) will arise. Given the mismatch between self-perception and actual behavior in evaluating fidelity to a treatment (Beidas & Kendall, 2010; Carroll, Martino, & Rounsaville, 2010), such an activity may be a fruitful way to increase trainee self-awareness and reduce the discrepancy be- tween perceived and actual behavior over time. Notably, such a strategy involves the use of videotaping or audiotaping therapy sessions, rather than relying on trainee self-report, as well as providing adherence ratings, both of which have been suggested as guidelines for training (Calhoun et al., 1998).
In addition to the use of actual recordings, we agree with suggestions to have more than one supervisor make competency assessments (e.g., based on adherence ratings during recorded sessions), given the systematic biases that supervisors have been shown to exhibit (Gonsalvez & Freestone, 2007; Gonsalvez & McLeod, 2008; Lazar & Mosek, 1993). In a related vein, we also suggest making both absolute and relative competency ratings to track progress over time and to compare ability to peers, respec- tively. Another possible way to assess clinical competency is client response to treatment. We recognize that, in isolation, this ap- proach may reflect idiosyncrasies in client’s symptom levels, rather than therapist skill, and may underemphasize client changes that are not indexed by standardized measures. As such, we do not espouse a particular reliance on client outcome as a proxy for supervisory or trainee effectiveness (Helge Rønnestad & Ladany, 2006). However, consistent with EBP and integrating multiple methods to obtain the best evidence, the use of client outcome data (vis-à-vis ongoing assessment ratings) can provide one method to assess therapist effectiveness. In sum, in an evidence-based ap- proach to training, the performance of a trainee may be based on
129GRADUATE TRAINING PROGRAMS
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
his or her skills in delivering treatment, standardized competency ratings, and clients’ treatment outcomes.
Overall, to enhance the science within the art of supervision, we encourage supervisors to model a nonjudgmental stance of empir- ical curiosity. For example, a supervisor can model this approach by encouraging supervisees to consult the literature, collaborating about hypotheses, and, as discussed previously, collecting data with supervisees from their sessions. In other words, consistent with the evidence-based value of training supervisees to be life- long learners, supervisors can encourage literature searches and hypothesis testing, rather than assuming the role of expert with the “correct” answer (Falender et al., 2004; Hunsley, 2007). Further, an evidence-based approach to supervision suggests that at least a subset of supervisory sessions may be audio- or video-recorded, analyzed, and critiqued (anonymously) vis-à-vis structured evalu- ation questionnaires, either by supervisees, colleagues, or indepen- dent agencies so that supervisors also have an opportunity to receive feedback (Gonsalvez & McLeod, 2008) and to identify discrepancies between perceived and actual behavior (Beidas & Kendall, 2010). We believe that engaging in these activities will increase the scientific zeitgeist of supervision, thereby continuing to instill in trainees the value of an EBP approach. Even more, these supervisory activities can provide a model from which train- ees can extrapolate to their own clinical work, consequently further reducing the gap between what gets taught in the class hour and what takes place during the therapy hour. These recommendations for supervision are consistent with the EBP framework in that we (a) expect supervisors and supervisees to obtain the best available evidence, (b) are interested in clinical expertise for both supervisor and supervisee, and (c) should take into consideration individual characteristics, values, and preferences (of both trainee and client) in determining how supervision should proceed. It is to this last point regarding client characteristics that we now turn.
Relevant Client Characteristics
As much as 40% of the variance in treatment outcome, or more, may be due to client variables (Addis et al., 2006). Thus, part of training in EBP must involve teaching trainees to integrate key client variables into their case conceptualization and treatment plan. Although a range of client characteristics are likely impor- tant, in the following sections, we focus on individual difference variables, expectations for and willingness to engage in treatment, and client values and preferences.
It goes without saying that clients’ presenting problems are heterogeneous. Individuals with the same disorder may exhibit different symptom constellations; further, those who have the same disorder may have experienced different developmental pathways and risk processes (equifinality), and those with the same risk factors (e.g., abuse) may experience very different outcomes (mul- tifinality; Cicchetti & Rogosch, 1996). As discussed previously, consistent with a developmental psychopathology perspective, EBP thus involves learning to approach the multifaceted “evi- dence,” which involves training in biological, cognitive, emo- tional, developmental, contextual, and cultural domains that may influence individuals’ functioning, course, and response to inter- vention. For example, the work of Westen and colleagues high- lights patterns of personality functioning that predict significant variability in adaptive and psychiatric functioning, above and
beyond DSM–IV (Diagnostic and Statistical Manual of Mental Disorders, 4th edition) diagnoses, that have direct implications for treatment targets (e.g., perfectionism, emotional constriction, and emotional dysregulation across the eating disorders; Hershenberg, Novotny, & Westen, 2006; Westen & Harnden-Fischer, 2001).
Additionally, knowledge of client characteristics included in controlled clinical trials, such as rates of comorbidity in targeted populations, may help a trainee to choose the most appropriate treatment and to adopt realistic and evidence-based expectations for client’s response to treatment. For example, one of the largest randomized clinical trials for the treatment of chronic depression demonstrated that about one-third of the sample also had a history of anxiety disorders, one-third a history of alcohol/substance abuse disorders, and two-thirds received a diagnosis for coexisting per- sonality disorders (Keller et al., 2000). Thus, when assessing a client who may be seeking therapy for a long-standing mood disorder such as depression, a trainee should know to assess for the presence of concomitant psychological issues that may affect the client’s response to treatment (even if these problems are not identified by the client as targets for treatment), and to learn to tailor treatment plans to anticipate and flexibly accommodate these additional issues as they become salient.
Further, awareness of the effect sizes of controlled outcome studies with targeted populations also can be used to promote therapist awareness of how much change can be attained (on average) with a particular treatment for a particular population and over how many “dosages” of therapy. Consistent with our recom- mendation to collect ongoing assessment data (e.g., through online software packages; see Lambert, in press; Youn et al., in press), aggregate effect sizes of outcome data gathered from clients in psychology training clinics can be compared with those of clinical trials (Minami et al., 2008; Ogles, Lambert, & Fields, 2002), or response to treatment can be compared with functioning in non- clinical populations (Kraus, Seligman, & Jordan, 2005; Youn, et al., in press) via benchmarking approaches. On an individualized level, trainees can compare their clients to expected recovery curves and examine client increases or decreases (measured in standardized scores) across relevant psychosocial and adaptive functioning domains. Statistically examining discrepancies be- tween individual clients and average or expected trajectories of change may enhance trainee’s attention to client characteristics that either promote or hinder the speed of change in therapy and, again, enhance the trainee’s ability to flexibly address possible discrepancies in a planful way. Ideally, as part of selecting the appropriate treatment based on the best research evidence, client characteristics could be used to determine which treatment (or components of treatment) may be appropriate for a particular client and in which circumstances (e.g., Chorpita et al., 2005; Roth et al., 1996). Unfortunately, as a field, we are far from meeting the goal of treatment matching; thus, we recommend selecting treatments based on both evidence and client characteristics, with careful attention to issues related to diversity and cultural differences.
In addition to more presumably static client variables such as personality style, we believe it is important that all trainees learn to form hypotheses about clients’ dynamic patterns of interper- sonal functioning and to monitor how those patterns are evidenced in the therapeutic relationship as part of their case conceptualiza- tion (McCullough, 2000, 2006). Such patterns may be conducive to therapy (e.g., the “good” client who completes her homework to
130 HERSHENBERG, DRABICK, AND VIVIAN
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
please others) or may interfere with goals of therapy (e.g., the client who has unresolved issues with a parent and projects these issues onto the therapist), and these patterns may be linked to maintaining factors in the client’s presenting problems. Given the robust associations between interpersonal functioning and psycho- pathology, we encourage programs to teach trainees to examine the interpersonal patterns of the client; to consider how the therapeutic relationship may potentially parallel, reinforce, or exacerbate mal- adaptive interpersonal patterns (see, e.g., Levenson & Strupp, 1997; McCullough, 2000); and to obtain supervision to determine how to support a more adaptive pattern of relating in interpersonal relationships.
Similarly, trainees must learn to assess and address process- related variables that affect the timing and course of treatment. For example, to address readiness and motivation to change, we rec- ommend that students learn the transtheoretical model of change (Prochaska & DiClemente, 1982) and suggest the possibility of training in motivational interviewing as one technique to address these stages of change (Miller & Rollnick, 1991). Further, because of the impact of client expectancies on treatment outcome (Arnkoff, Glass, & Shapiro, 2002), as we discussed earlier, we also recommend learning to directly address these expectancies at the onset of treatment (Callahan, Aubuchon-Endsley, et al., 2009), preparing clients for their roles in therapy, and developing a collaborative approach (DeFife & Hilsenroth, 2011), which may increase rates of client retention (Swift & Callahan, 2008b).
In terms of client characteristics, values, and preferences, train- ees should learn to work with clients as shared decision-makers and recognize clients’ roles as stake-holders in therapy, which requires the development of a treatment plan that is sensitive and responsive to clients’ values (e.g., cultural diversity, spirituality, involvement of family) and preferences (e.g., acceptability of treatment, timing and frequency of intervention, use of medica- tions, individual and/or group therapy; Bauer, 2007; Spring, 2007). One issue that requires attention is how to assess such values and preferences; indeed, there is a dearth of user-friendly, validated instruments for identifying client values and preferences that will inform the decisions that we make (Spring, 2007). However, such information will influence not only case conceptualization and treatment planning, but also trainees’ development of clinical expertise through their learning to assess and apply information about client values and preferences flexibly and effectively (Bauer, 2007). Training consequently should prepare students for attending to these individual client characteristics, as well as quantifying and monitoring these variables to evaluate how these factors affect treatment course and relevant outcomes.
Overall, we recognize that training programs likely differ in the extent to which client characteristics are incorporated into case conceptualization and intervention. For example, clinics training in predominantly interpersonal approaches may consider relevant personality and interpersonal variables that manifest themselves in the “here and now” of the client–therapist interaction as one of the most salient aspects of active intervention, whereas more cognitive behaviorally oriented approaches may largely reflect on progress that occurs outside of session. Regardless of where the focus of change takes place (e.g., inside or outside of the session), an EBP approach includes the incorporation of client variables into the case conceptualization and treatment plan, which may directly impact the development of a strong alliance. We hypothesize that
if assessing for and responding to client variables is reinforced in a training program, trainees might view responding to client am- bivalence and even resistance as part of the intervention, rather than something standing in the way of it. However, this is ulti- mately an empirical question, and tools for assessing and moni- toring these variables will be useful for evaluating this hypothesis. In sum, trainees must not only learn what type of disorder the client has, but also know what kind of client has the disorder (Reed, Kihlstrom, & Messer, 2006). As such, training in EBP requires that trainees increase their own awareness, assessment, and implementation of knowledge related to client variables that will affect the timing, style, and content of their intervention.
Conclusions
EBP is the integration of the best available research with clinical expertise in the context of client characteristics, values, and pref- erences. Consistent with the spirit of EBP, we have provided suggestions for methodologies to increase our ability as a field to train students to function as empirically informed and clinically sensitive clinicians and clinical researchers, with a particular em- phasis on clinical training (summarized in Table 1).
A few caveats must be kept in mind. First, because training is contextual, trainees and supervisors are embedded within larger systems. As such, the adoption of EBPs into training needs to be accepted by all faculty; similar to a key client variable such as resistance (Sanders & Murphy-Brennan, 2010), motivation to change may need to be addressed. Second, further empirical study is needed to determine the appropriate focus of training content (Beidas & Kendall, 2010; Calhoun et al., 1998; McFall, 1991), including “how” it gets taught (e.g., use of multiple approaches to learning), as well as “what,” “how much” (e.g., duration), and “when” (e.g., sequence, spacing) (Chu, 2008; Cucciare et al., 2008).
Notably, in addition to documenting clinical notes and conduct- ing single-case studies, there are numerous opportunities for psy- chotherapy research that can be incorporated into the clinical setting to answer these questions and to assess how well specific training activities are meeting these goals. For example, data collected through coursework, assessment, and supervision can be included in the clinic’s database and used to determine training variables that are associated with increases in trainee competency and client outcome. The overall goal would be to support a research infrastructure within the training clinic, to collect data that could be used to evaluate particular components of treatment or training, and to provide one resource that could be combined across different training clinics, consistent with Practice Research Networks (Borkovec, 2004; Drabick & Goldfried, 2000).
The bigger issue in the field is the chasm between research and practice, though this gap is increasingly shrinking and may be seen from either an optimistic or pessimistic lens (Teachman et al., in press). We believe that there are ample opportunities within the current climate to train students to act and think in ways that will reduce the polarization both now and over time. Indeed, doing so will permit trainees to begin their own careers with this way of thinking and behaving firmly embedded in their approach to clin- ical practice and clinical research, and to train others to adopt this approach and associated strategies. We write this training piece with incredible optimism, given the bastion of resources at our
131GRADUATE TRAINING PROGRAMS
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
disposal. We hope that our suggestions contribute to others’ think- ing about ways to facilitate training in EBP and to further identify opportunities for bridging clinical research and clinical practice.
References
Ackerman, S. J., & Hilsenroth, M. J. (2001). A review of therapist char- acteristics and techniques negatively impacting the therapeutic alliance. Psychotherapy: Theory, Research, Practice, Training, 38, 171–185. doi:10.1037/0033–3204.38.2.171.
Ackerman, S. J., & Hilsenroth, M. J. (2003). A review of therapist char- acteristics and techniques positively impacting the therapeutic alliance. Clinical Psychology Review, 23, 1–33. doi: 10.1016/S0272- 7358(02)00146 – 0
Addis, M. E., Cardemil, E. V., Duncan, B. L., & Miller, S. D. (2006). Does manualization improve therapy outcomes? In J. C. Norcross, L. E. Beutler & R. F. Levant (Eds.), Evidence-based practices in mental health: Debate and dialogue on the fundamental questions (pp. 131– 160). Washington, DC: American Psychological Association. doi: 10.1037/11265-003
Allen, L. B., McHugh, R. K., & Barlow, D. H. (2008). Emotional disor- ders: A unified protocol. In D. H. Barlow (Ed.), Clinical handbook of psychological disorders: A step-by-step treatment manual (4th ed., pp. 216 –249). New York, NY: Guilford Press.
Arnkoff, D. B., Glass, C. R., & Shapiro, S. J. (2002). Expectations and preferences. In J. C. Norcross (Ed.), Psychotherapy relationships that work: Therapist contributions and responsiveness to patients (pp. 335– 356). New York, NY: Oxford University Press.
Bambling, M., King, R., Raue, P., Schweitzer, R., & Lambert, W. (2006). Clinical supervision: Its influence on client-rated working alliance and client symptom reduction in the brief treatment of major depression. Psychotherapy Research, 16, 317–331. doi:10.1080/10503300500268524
Bauer, R. M. (2007). Evidence-based practice in psychology: Implications for research and research training. Journal of Clinical Psychology, 63, 685– 694. doi:10.1002/jclp.20374
Beidas, R. S., & Kendall, P. C. (2010). Training therapists in evidence- based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17, 1–30. doi: 10.1111/j.1468-2850.2009.01187.x
Binder, J. L., & Strupp, H. H. (1997). “Negative Process”: A recurrently discovered and underestimated facet of therapeutic process and outcome in the individual psychotherapy of adults. Clinical Psychology: Science and Practice, 4, 121–139. doi:10.1111/j.1468-2850.1997.tb00105.x
Borckardt, J. J., Nash, M. R., Murphy, M. D., Moore, M., Haw, D., & O’Neil, P. (2008). Clinical practice as natural laboratory for psychother- apy research: A guide to case-based time-series analysis. American Psychologist, 63, 77–95. doi:10.1037/0003-066X.63.2.77
Borkovec, T. D. (2002). Training clinic research and the possibility of a national training clinics practice research network. The Behavior Ther- apist, 25, 98 –103.
Borkovec, T. D. (2004). Research in training clinics and practice research net- works: A Route to the Integration of Science and Practice. Clinical Psychology: Science and Practice, 11, 211–215. doi:10.1093/clipsy.bph073
Boswell, J. F., Nelson, D. A., Nordberg, S. S., McAleavey, A. A., & Castonguay, L. G. (2010). Competency in integrative psychotherapy: Perspectives on training and supervision. Psychotherapy Theory, Re- search, Practice, Training, 47, 3–11. doi:10.1037/a0018848
Calhoun, K. S., Moras, K., Pilkonis, P. A., & Rehm, L. P. (1998). Empirically supported treatments: Implications for training. Journal of Consulting and Clinical Psychology, 66, 151–162. doi:10.1037/0022- 006X.66.1.151
Callahan, J. L., Almstrom, C. M., Swift, J. K., Borja, S. E., & Heath, C. J. (2009). Exploring the contribution of supervisors to intervention out- comes. Training and Education in Professional Psychology, 3, 72–77. doi:10.1037/a0014294
Callahan, J. L., Aubuchon-Endsley, N., Borja, S. E., & Swift, J. K. (2009). Pretreatment expectancies and premature termination in a training clinic environment. Training and Education in Professional Psychology, 3, 111–119. doi: 10.1037/a0012901
Callahan, J. L., & Hynan, M. T. (2005). Models of psychotherapy outcome: Are they applicable in training clinics? Psychological Services, 2, 65– 69. doi:10.1037/1541-1559.2.1.65
Carroll, K. M., Martino, S., & Rounsaville, B. J. (2010). No train, no gain? Clinical Psychology: Science and Practice, 17, 36 – 40. doi:10.1111/ j.1468-2850.2009.01190.x
Castonguay, L. G., & Beutler, L. E. (2006). Principles of therapeutic change that work. New York, NY: Oxford University Press.
Castonguay, L. G., Goldfried, M. R., Wiser, S., Raue, P. J., & Hayes, A. M. (1996). Predicting the effect of cognitive therapy for depression: A study of unique and common factors. Journal of Consulting and Clinical Psychology, 64, 497–504. doi:10.1037/0022-006X.64.3.497
Cellucci, T., & Reports, M. (2010). A competencies evaluation and tracking system for pre-doctoral practicum training. Poster presented at the American Psychological Association Annual Convention, San Diego, CA.
Chambless, D. L., Crits-Christoph, P., Wampold, B. E., Norcross, J. C., Lambert, M. J., Bohart, A. C., . . . Johannsen, B. E. (2006). What should be validated? In J. C. Norcross, L. E. Beutler & R. F. Levant (Eds.), Evidence-based practices in mental health: Debate and dialogue on the fundamental questions (pp. 191–256). Washington, DC: American Psy- chological Association. doi:10.1037/11265-005
Chorpita, B., Daleiden, E., & Weisz, J. (2005). Identifying and selecting the common elements of evidence based interventions: A distillation and matching model. Mental Health Services Research, 7, 5–20. doi: 10.1007/s11020-005–1962-6
Chu, B. C. (2008). Empirically supported training approaches: The who, what, and how of disseminating psychological interventions. Clinical Psychology: Science and Practice, 15, 308 –312. doi:10.1111/j.1468- 2850.2008.00142.x
Cicchetti, D., & Rogosch, F. A. (1996). Equifinality and multifinality in developmental psychopathology. Development and Psychopathology, 8, 597– 600. doi:10.1017/S0954579400007318
Clarkin, J. F., & Levy, K. N. (2003). Influence of client variables on psychotherapy. In M. J. Lambert (Ed.), Handbook of psychotherapy and behavior change (5th ed.). New York, NY: Wiley & Sons.
Collins, F. L., Leffingwell, T. R., & Belar, C. D. (2007). Teaching evidence-based practice: Implications for psychology. Journal of Clin- ical Psychology, 63, 657– 670. doi:10.1002/jclp.20378
Cucciare, M. A., Weingardt, K. R., & Villafranca, S. (2008). Using blended learning to implement evidence-based psychotherapies. Clinical Psy- chology: Science and Practice, 15, 299 –307. doi:10.1111/j.1468- 2850.2008.00141.x
Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science, 243, 1668 –1674. doi:10.1126/science.2648573
DeFife, J. A., & Hilsenroth, M. J. (2011). Starting off on the right foot: Common factor elements in early psychotherapy process. Journal of Psychotherapy Integration, 21, 172–191. doi: 10.1037/a0023889
DiLillo, D., & McChargue, D. (2007). Implementing elements of evidence- based practice into scientist–practitioner training at the University of Nebraska-Lincoln. Journal of Clinical Psychology, 63, 671– 684. doi: 10.1002/jclp.20375
Drabick, D. A., & Goldfried, M. R. (2000). Training the scientist– practitioner for the 21st century: Putting the bloom back on the rose. Journal of Clinical Psychology, 56, 327–340. doi:10.1002/(SICI)1097- 4679(200003)56:3�327::AID-JCLP9�3.0.CO;2-Y
Drabick, D. A., & Kendall, P. C. (2010). Developmental psychopathology and the diagnosis of mental health problems among youth. Clinical Psychology: Science and Practice, 17, 272–280. doi:10.1111/j.1468- 2850.2010.01219.x
Drabick, D. A., & Steinberg, L. (2011). Developmental psychopathology.
132 HERSHENBERG, DRABICK, AND VIVIAN
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
In B. Brown & M. Prinstein (Eds.), Encyclopedia of Adolescence (Vol. 3, pp. 136 –142). San Diego: Academic Press. doi:10.1016/B978-0-12- 373951-3.00109-5
Duncan, B. L., Miller, S. D., Reynolds, L., Sparks, J., Claud, D., Brown, J., & Johnson, L. D. (2003). The session rating scale: Psychometric prop- erties of a ‘working’ alliance scale. Journal of Brief Therapy, 3, 3–12.
Eells, T. D. (2007). Handbook of psychotherapy case formulation. New York, NY: The Guilford Press.
Falender, C. A., Cornish, J. A. E., Goodyear, R., Hatcher, R., Kaslow, N. J., Leventhal, G., . . . Grus C. (2004). Defining competencies in psychology supervision: A consensus statement. Journal of Clinical Psychology, 60, 771–785. doi:10.1002/jclp.20013
Falender, C. A., & Shafranske, E. P. (2004). Clinical supervision: A competency-based approach. Washington, DC: American Psychological Association. doi:10.1037/10806-000
Falzon, L., Davidson, K. W., & Bruns, D. (2010). Evidence searching for evidence-based psychology practice. Professional Psychology: Re- search and Practice, 41, 550 –557. doi:10.1037/a0021352
Gard, G., Tremblay, G., DiLillo, D., & Pantesco, V. (2002). Facilitating research in training clinics: Aspiring to the scientist-practitioner ideal. The Behavior Therapist, 25, 103–106.
Garfield, S. L. (1994). Research on client variables in psychotherapy. In A. E. Bergin & S. L. Garfield (Eds.), Handbook of psychotherapy and behavior change (4th ed.). New York, NY: Wiley & Sons.
Goldfried, M. R., & Davila, J. (2005). The role of relationship and technique in therapeutic change. Psychotherapy: Theory, Research, Practice, Training, 42, 421– 430. doi:10.1037/0033–3204.42.4.421
Goldfried, M. R., & Wolfe, B. E. (1996). Psychotherapy practice and research: Repairing a strained alliance. American Psychologist, 51, 1007–1016. doi:10.1037/0003-066X.51.10.1007
Goldfried, M. R. (1980). Toward the delineation of therapeutic change principles. American Psychologist, 35, 991–999. doi:10.1037/0003- 066X.35.11.991
Gonsalvez, C. J., & Freestone, J. (2007). Field supervisors’ assessments of trainee performance: Are they reliable and valid? Australian Psycholo- gist, 42, 23–32. doi: 10.1080/00050060600827615
Gonsalvez, C. J., & McLeod, H. J. (2008). Toward the science-informed practice of clinical supervision: The Australian context. Australian Psy- chologist, 43, 79 – 87. doi:10.1080/00050060802054869
Gray, L. A., Ladany, N., Walker, J. A., & Ancis, J. R. (2001). Psychotherapy trainees’ experience of counterproductive events in supervision. Journal of Counseling Psychology, 48, 371–383. doi:10.1037/0022-0167.48.4.371
Harmon, C., Hawkins, E. J., Lambert, M. J., Slade, K., & Whipple, J. S. (2005). Improving outcomes for poorly responding clients: The use of clinical support tools and feedback to clients. Journal of Clinical Psy- chology, 61, 175–185. doi:10.1002/jclp.20109
Hart, D., & Marmorstein, N. R. (2009). Neighborhoods and genes and everything in between: Understanding adolescent aggression in social and biological contexts. Development and Psychopathology, 21, 961– 973. doi:10.1017/S0954579409000510
Helge Rønnestad, M., & Ladany, N. (2006). The impact of psychotherapy training: Introduction to the special section. Psychotherapy Research, 16, 261–267. doi: 10.1080/10503300600612241
Henry, W. P., Strupp, H. H., Butler, S. F., Schacht, T. E., & Binder, J. L. (1993). Effects of training in time-limited dynamic psychotherapy: Changes in therapist behavior. Journal of Consulting and Clinical Psy- chology, 61, 434 – 440. doi:10.1037/0022-006X. 61.3.434
Hershenberg, R., Novotny, C., & Westen, D. (2006). Personality subtypes and eating disorders: Replication and validation of a taxonomy. Emory University.
Hilsenroth, M. J., Blagys, M. D., Ackerman, S. J., Bonge, D. R., & Blais, M. A. (2005). Measuring psychodynamic-interpersonal and cognitive- behavioral techniques: Development of the comparative psychotherapy process scale. Psychotherapy: Theory, Research, Practice, Training, 42, 340 –356. doi:10.1037/0033–3204.42.3.340
Hilsenroth, M. J., & Cromer, T. D. (2007). Clinician interventions related to alliance during the initial interview and psychological assessment. Psychotherapy: Theory, Research, Practice, Training;Psychotherapy: Theory, Research, Practice, Training, 44, 205–218. doi:10.1037/0033– 3204.44.2.205
Hilsenroth, M. J., Defife, J. A., Blagys, M. D., & Ackerman, S. J. (2006). Effects of training in short-term psychodynamic psychotherapy: Changes in graduate clinician technique. Psychotherapy Research, 16, 293–305. doi:10.1080/10503300500264887
Hunsley, J., & Mash, E. J. (2007). Evidence-based assessment. Annual Review of Clinical Psychology, 3, 29 –51. doi:10.1146/annurev- .clinpsy.3.022806.091419
Hunsley, J. (2007). Training Psychologists for Evidence-Based Practice. Canadian Psychology, 48, 32– 42. doi: 10.1037/cp2007005a
Jensen, P. S., & Hoagwood, K. (1997). The book of names: DSM-IV in context. Development and Psychopathology, 9, 231–249. doi:10.1017/ S0954579497002034
Kavanagh, D. J., Spence, S. H., Strong, J., Wilson, J., Sturk, H., & Crown, N. (2003). Supervision practices in allied mental health: A staff survey. Mental Health Services Research, 5, 187–195. doi:10.1023/A:1026223517172
Keller, M. B., McCullough, J. P., Klein, D. N., Arnow, B., Dunner, D. L., Gelenberg, A. J., . . . Zajecka J. (2000). A Comparison of nefazodone, the cognitive behavioral-analysis system of psychotherapy, and their combination for the treatment of chronic depression. New England Journal of Medicine, 342, 1462–1470. doi:10.1056/NEJM200005183422001
Kendall, P. C., & Beidas, R. S. (2007). Smoothing the trail for dissemi- nation of evidence-based practices for youth: Flexibility within fidelity. Professional Psychology: Research and Practice, 38, 13–20. doi: 10.1037/0735–7028.38.1.13
Kendjelic, E. M., & Eells, T. D. (2007). Generic psychotherapy case formulation training improves formulation quality. Psychotherapy: Theory, Research, Prac- tice, Training, 44, 66–77. doi: 10.1037/0033–3204.44.1.66
Kim, D., Wampold, B. E., & Bolt, D. M. (2006). Therapist effects in psychotherapy: A random-effects modeling of the National Institute of Mental Health Treatment of Depression Collaborative Research Pro- gram data. Psychotherapy Research, 16, 161–172. doi: 10.1080/ 10503300500264911
Kraus, D. R., Seligman, D., & Jordan, J. R. (2005). Validation of a behavioral health treatment outcome and assessment tool designed for naturalistic settings: The Treatment Outcome Package. Journal of Clin- ical Psychology, 61, 285–314. doi:10.1002/jclp.20084
Kuyken, W., Padesky, C. A., & Dudley, R. (2009). Collaborative case conceptualization: Working effectively with clients in cognitive- behavioral therapy. New York, NY: The Guilford Press.
Lambert, M. J. (in press). Helping clinicians to use and learn from research-based systems: The OQ-analyst. Psychotherapy.
Lazar, A., & Mosek, A. (1993). The Influence of the field instructor- student relationship on evaluation of students’ practice. The Clinical Supervisor, 11, 111–120. doi: 10.1300/J001v11n01_08
Levenson, H., & Strupp, H. H. (1997). Cyclical maladaptive patterns: Case formulation in time-limited dynamic psychotherapy. In T. D. Eells (Ed.), Handbook of psychotherapy case formulation (pp. 84 –115). New York, NY: Guilford Press.
Mash, E. J., & Hunsley, J. (2005). Evidence-based assessment of child and adolescent Disorders: Issues and challenges. Journal of Clinical Child & Ado- lescent Psychology, 34, 362–379. doi: 10.1207/s15374424jccp3403_1
McCullough, J. P. (2000). Treatment for chronic depression: Cognitive behavioral analysis system of psychotherapy. New York, NY: The Guilford Press.
McCullough, J. P. (2006). Treating chronic depression with disciplined personal involvement: Cognitive behavioral analysis system of psycho- therapy (CBASP). New York, NY: Springer Publishing Company, Inc.
McFall, R. M. (1991). Manifesto for a science of clinical psychology. Clinical Psychologist, 44, 75– 88.
133GRADUATE TRAINING PROGRAMS
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
Miller, S. D., Duncan, B. L., Brown, J., Sorrell, R., & Chalk, M. B. (2006). Using outcome to inform and improve treatment outcomes. Journal of Brief Therapy, 5, 5–22.
Miller, W. R., & Rollnick, S. (1991). Motivational interviewing: Preparing people to change addictive behavior. New York, NY: Guilford Press
Miller, W. R., Yahne, C. E., Moyers, T. B., Martinez, J., & Pirritano, M. (2004). A randomized trial of methods to help clinicians learn motiva- tional interviewing. Journal of Consulting and Clinical Psychology, 72, 1050 –1062. doi:10.1037/0022-006X.72.6.1050
Minami, T., Davies, D. R., Tierney, S. C., Bettmann, J. E., McAward, S. M., Averill, L. A., . . . Wampold, B. E (2009). Preliminary evidence on the effectiveness of psychological treatments delivered at a university counseling center. Journal of Counseling Psychology, 56, 309 –320. doi: 10.1037/a0015398
Minami, T., Wampold, B. E., Serlin, R. C., Hamilton, E. G., Brown, G. S., & Kircher, J. C. (2008). Benchmarking the effectiveness of psychother- apy treatment for adult depression in a managed care environment: A preliminary study. Journal of Consulting and Clinical Psychology, 76, 116 –124. doi:10.1037/0022-006X. 76.1.116
Nathan, P. E., & Gorman, J. E. (Eds.). (1998). A guide to treatments that work. New York, NY: Oxford University Press.
Nelson, M. L., & Friedlander, M. L. (2001). A close look at conflictual supervisory relationships: The trainee’s perspective. Journal of Coun- seling Psychology, 48, 384 –395. doi:10.1037/0022– 0167.48.4.384
Ogles, B. M., Lambert, M. J., & Fields, S. A. (2002). Essentials of outcome assessment. New York, NY: John Wiley & Sons, Inc.
Page, A. C., Stritzke, W. G. K., & McLean, N. J. (2008). Toward science- informed supervision of clinical case formulation: A training model and supervision method. Australian Psychologist, 43, 88 –95. doi: 10.1080/ 00050060801994156
Page, A. C., & Stritzke, W. G. K. (2006). Clinical psychology for trainees: A foundations of science-informed practice. Cambridge, United King- dom: Cambridge University Press. doi:10.1017/CBO9780511607363
Prochaska, J. O., & DiClemente, C. C. (1982). Transtheoretical therapy: Toward a more integrative model of change. Psychotherapy: Theory, Research & Practice, 19, 276 –288. doi:10.1037/h0088437
Reed, G. M., Kihlstrom, J. F., & Messer, S. B. (2006). What qualifies as evidence of effective practice? In J. C. Norcross, L. E. Beutler & R. F. Levant (Eds.), Evidence-based practices in mental health: Debate and dialogue on the fundamental questions (pp. 13–55). Washington, DC: American Psychological Association. doi:10.1037/11265-001
Reiser, R., & Milne, D. (In press). Supervising cognitive-behavioral psy- chotherapy: Pressing needs, impressing possibilities. Journal of Contem- porary Psychotherapy.
Rings, J. A., Genuchi, M. C., Hall, M. D., Angelo, M-A., & Cornish, J. A. (2009). Is there consensus among predoctoral internship training direc- tors regarding clinical supervision competencies? A descriptive analysis. Training and Education in Professional Psychology, 3, 140 –147. doi: 10.1037/a0015054
Rodolfa, E., Bent, R., Eisman, E., Nelson, P., Rehm, L., & Ritchie, P. (2005). A cube model for competency development: Implications of psychology educators and regulators. Professional Psychology: Re- search and Practice, 36, 347– 454. doi:10.1037/0735-7028.36.4.347
Roth, A., Fonagy, P., Parry, G., Target, M., & Woods, R. (1996). What works for whom? A critical review of psychotherapy research: New York, NY: Guilford Press.
Rutter, M., & Sroufe, L. A. (2000). Developmental psychopathology: Concepts and challenges. Development and Psychopathology, 12, 265– 296. doi:10.1017/S0954579400003023
Safran, J. D., Muran, J. C., Samstag, L. W., & Stevens, C. (2001). Repairing alliance ruptures. Psychotherapy: Theory, Research, Practice, Training, 38, 406 – 412. doi:10.1037/0033–3204.38.4.406
Safran, J. D., & Muran, J. C. (2000). Resolving therapeutic alliance ruptures: Diversity and integration. Journal of Clinical Psychology, 56,
233–243. doi:10.1002/(SICI)1097-4679(200002)56:2�233::AID- JCLP9�3.0.CO;2-3
Sanders, M. R., & Murphy-Brennan, M. (2010). Creating conditions for success beyond the professional training environment. Clinical Psychology: Science and Practice, 17, 31–35. doi:10.1111/j.1468-2850.2009.01189.x
Sauer, E., & Huber, D. (2007). Implementing the boulder model of training in a psychology training clinic. Journal of Contemporary Psychother- apy, 37, 221–228. doi:10.1007/s10879-007-9057-x
Scott, K. J., Ingram, K. M., Vitanza, S. A., & Smith, N. G. (2000). Training in supervision: A survey of current practices. The Counseling Psychol- ogist, 28, 403– 422. doi:10.1177/0011000000283007
Spring, B. (2007). Evidence-based practice in clinical psychology: What it is, why it matters; what you need to know. Journal of Clinical Psychol- ogy, 63, 611– 631. doi:10.1002/jclp.20373
Stein, D. M., & Lambert, M. J. (1995). Graduate training in psychotherapy: Are therapy outcomes enhanced? Journal of Consulting and Clinical Psychology, 63, 182–196. doi:10.1037/0022-006X. 63.2.182
Steinberg, L., & Avenevoli, S. (2000). The role of context in the develop- ment of psychopathology: A conceptual framework and some specula- tive propositions. Child Development, 71, 66 –74. doi:10.1111/1467- 8624.00119
Stiles, W. B., Barkham, M., Mellor-Clark, J., & Connell, J. (2007). Effec- tiveness of cognitive-behavioural, person-centred, and psychodynamic therapies in UK primary-care routine practice: Replication in a larger sample. Psychological Medicine, 38, 677– 688.
Stricker, G. (1992). The relationship of research to clinical practice. Amer- ican Psychologist, 47, 543–549. doi:10.1037/0003-066X.47.4.543
Swets, J. A., Dawes, R. M., & Monahan, J. (2000). Psychological science can improve diagnostic decisions. Psychological Science in the Public Interest, 1, 1–26. doi:10.1111/1529-1006.001
Swift, J. K., & Callahan, J. L. (2008a). A delay discounting measure of great expectations and the effectiveness of psychotherapy client decision making. Professional Psychology: Research and Practice, 39, 581–588. doi:10.1037/0735-7028.39.6.581
Swift, J. K., & Callahan, J. L. (2008b). Decreasing treatment drop-out thru pre-treatment education. Poster presentation at the 2008 OPA Annual Convention. Tulsa, OK.
Teachman, B. A., Drabick, D. A., Hershenberg, R., Vivian, D., Wolfe, B. E., & Goldfried, M. R. (In press). Bridging the gap between clinical research and clinical practice: Introduction to the special section. Psychotherapy
Wampold, B. E. (2007). Psychotherapy: The humanistic (and effective) treatment. American Psychologist, 62, 857– 873. doi:10.1037/0003- 066X.62.8.857
Watkins, C. E. (2011). Psychotherapy supervision since 1909: Some friendly observations about its first century. Journal of Contemporary Psychotherapy, 41, 57– 67. doi:10.1007/s10879-010-9152-2
Weinberger, J. (1995). Common factors aren’t so common: The common factors dilemma. Clinical Psychology: Science and Practice, 2, 45– 69. doi:10.1111/j.1468-2850.1995.tb00024.x
Westen, D., & Harnden-Fischer, J. (2001). Personality profiles in eating disorders: Rethinking the distinction between Axis I and Axis II. American Journal of Psychiatry, 165, 547–562. doi:10.1176/appi.ajp.158.4.547
Wolf, A., Goldfried, M. R., & Muran, J. C. (Eds.). (2012). Transforming negative reactions to clients: From frustration to compassion. Washing- ton, DC: American Psychological Association.
Wolfe, B. E. (In press). Two chair dialogue between my research head and my therapist head. Psychotherapy.
Youn, S. J., Kraus, D., & Castonguay, L. G. (In press). The treatment outcome package: Facilitating practice and clinically relevant research. Psychotherapy.
Received January 29, 2012 Accepted January 29, 2012 �
134 HERSHENBERG, DRABICK, AND VIVIAN
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
