Tutor: MathematicsExpert Only

Enhancing Psychotherapy Process With Common Factors Feedback: A Randomized, Clinical Trial

Andrew S. McClintock, Matthew R. Perlman, Shannon M. McCarrick, Timothy Anderson, and Lina Himawan

Ohio University

In this study, we developed and tested a common factors feedback (CFF) system. The CFF system was designed to provide ongoing feedback to clients and therapists about client ratings of three common factors: (a) outcome expectations, (b) empathy, and (c) the therapeutic alliance. We evaluated the CFF system using randomized, clinical trial (RCT) methodology. Participants: Clients were 79 undergradu- ates who reported mild, moderate, or severe depressive symptoms at screening and pretreatment assessments. These clients were randomized to either: (a) treatment as usual (TAU) or (b) treatment as usual plus the CFF system (TAU � CFF). Both conditions entailed 5 weekly sessions of evidence-based therapy delivered by doctoral students in clinical psychology. Clients completed measures of common factors (i.e., outcome expectations, empathy, therapeutic alliance) and outcome at each session. Clients and therapists in TAU � CFF received feedback on client ratings of common factors at the beginning of Sessions 2 through 5. When surveyed, clients and therapists indicated that that they were satisfied with the CFF system and found it useful. Multilevel modeling revealed that TAU � CFF clients reported larger gains in perceived empathy and alliance over the course of treatment compared with TAU clients. No between-groups effects were found for outcome expectations or treatment outcome. These results imply that our CFF system was well received and has the potential to improve therapy process for clients with depressive symptoms.

Public Significance Statement In this study, we developed a system that provides ongoing feedback to clients and therapists about what is transpiring in therapy. Results suggest that the feedback system may help to improve the process of treatment for clients with depressive symptoms.

Keywords: common factors, feedback, empathy, alliance, randomized clinical trial

A growing body of research attests to the utility and effectiveness of outcome feedback (Connolly Gibbons et al., 2015; De Jong et al., 2014; Shimokawa, Lambert, & Smart, 2010). In outcome feedback systems, client progress is monitored and reviewed by therapists (and, in some cases, by clients as well) to guide ongoing treatment (Lam- bert, 2007). Specifically, these systems collect distress/symptomatol- ogy data from clients on a routine basis, and then compare these data with norms or expected treatment responses (see Lambert, 2007; Lutz et al., 2006). When a client is off-track (i.e., is projected to have a relatively poor treatment response), the therapist is alerted and is then typically provided with strategies for improving quality of care (Lambert et al., 2004; Miller, Duncan, Sorrell, & Brown, 2005).

Although outcome feedback has demonstrated efficacy (e.g., Shimokawa et al., 2010), there is undoubtedly room for improve- ment. Effects for outcome feedback systems are often only small or medium in size and, in some samples, are nonsignificant (Con- nolly Gibbons et al., 2015; De Jong et al., 2014; Shimokawa et al., 2010). In a recent study, Connolly Gibbons et al. (2015) found that 64% of clients who received treatment with outcome feedback did not achieve clinically significant change. Clearly, modifications to these systems are warranted.

One novel approach is to utilize process-based feedback. Pro- cess feedback may be advantageous for several reasons. First, there is evidence from educational psychology (e.g., Zimmerman & Kitsantas, 1997) that the development of a skill (e.g., consis- tently hitting a bull’s-eye on a dartboard) is enhanced through a focus on process (e.g., the mechanics of dart-throwing). From this, it stands to reason that the development of psychological well- being may be enhanced by focusing on the therapeutic processes that foster well-being. Second, certain treatment modalities (e.g., humanistic and psychodynamic therapies) do not target symptoms per se and thus may be more compatible with a process feedback system than an outcome/symptom-based feedback system. Third, whereas therapists may view outcome feedback as evaluative and threatening (Boswell, Krauss, Miller, & Lambert, 2015), therapists

This article was published Online First January 23, 2017. Andrew S. McClintock, Matthew R. Perlman, Shannon M. McCarrick,

Timothy Anderson, and Lina Himawan, Department of Psychology, Ohio University.

The ideas and data reported in this article have not been previously disseminated.

Correspondence concerning this article should be addressed to Andrew S. McClintock, 264 Porter Hall, Athens, OH 45701. E-mail: am248310@ ohio.edu

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

Journal of Counseling Psychology © 2017 American Psychological Association 2017, Vol. 64, No. 3, 247–260 0022-0167/17/$12.00 http://dx.doi.org/10.1037/cou0000188

247

may be more receptive to feedback about what is transpiring in therapy. Thus, process feedback has the potential to be more widely implemented. Fourth, a process feedback system could yield information that is actionable and immediately useful. For example, disagreement about treatment tasks could be readily addressed by exploring discrepancies between the implemented techniques and the client’s perceptions about which techniques should be implemented.

An exemplary system that integrates process and outcome feed- back is the Partners for Change Outcome Management System (PCOMS; Miller et al., 2005; Duncan, 2012). PCOMS monitors the therapeutic alliance (i.e., agreement on therapeutic goals and tasks in the context of a positive affective bond; Bordin, 1979) at every session, enabling therapists to identify and repair alliance ruptures on an ongoing basis. Although the effectiveness of PCOMS is well documented (e.g., Duncan, 2012), it is unclear whether PCOMS’ effectiveness is because of outcome feedback, process feedback, or both. Indeed, no research to date has exam- ined the efficacy of process feedback in and of itself.

To build a process feedback system that could be widely im- plemented, it seems prudent to track processes that are common across treatment approaches (i.e., “common factors”). Common factors account for the lion’s share of outcome variance (�50%), more so than theory-specific techniques (�15%) and extrathera- peutic factors (�25%) (Cuijpers et al., 2012; Lambert, 2013). In a landmark text, Wampold and Imel (2015) highlighted three spe- cific common factors that drive change in psychotherapy: (a) client’s outcome expectations, (b) a genuinely empathic connec- tion between client and therapist, and (c) the therapeutic alliance. Outcome expectations, empathy, and the alliance are discussed in the following sections to highlight their suitability for inclusion in a process feedback system.

Outcome Expectations

Outcome expectations are anticipatory beliefs about a treat- ment’s personal efficacy (Constantino, Ametrano, & Greenberg, 2012). A recent meta-analysis (Constantino, Glass, Arnkoff, Ame- trano, & Smith, 2011) that included 8,016 clients across 46 inde- pendent samples revealed that client outcome expectations ac- counted for a significant, albeit modest, percentage (1.4%) of outcome variance. It is worth noting that this association was derived predominantly from studies that assessed outcome expec- tations before or very early in treatment. An alternative approach is to conceptualize outcome expectations as a dynamic process, wherein the client’s expectations are influenced by the developing client-therapist relationship, the credibility of the treatment ratio- nale, the effectiveness of early treatment procedures, and so forth. That is, according to this approach, outcome expectations may evolve over the course of therapy and thus should be measured beyond the first few sessions. Underscoring the utility of monitor- ing outcome expectations over the course of treatment, Newman and Fisher (2010) found that a midtreatment assessment of expec- tancy/credibility accounted for nearly 40% of the variance in therapeutic change.

Empathy

Empathy is a complex, interactional process involving three temporal stages: (a) the therapist’s attunement to the client’s

experience, (b) the therapist’s communication about the client’s experience, and (c) the client’s receipt of the empathic communi- cation (Barrett-Lennard, 1981; MacFarlane, Anderson, & Mc- Clintock, 2015). A focus on the third stage is particularly impor- tant because client’s perceptions of therapist empathy may have the largest effect on outcome (Elliott, Bohart, Watson, & Green- berg, 2011); a meta-analysis of 38 studies (Elliott et al., 2011) showed that client-perceived empathy accounted for over 10% of outcome variance.

Alliance

A related construct is the therapeutic alliance, which refers to the collaborative, working relationship between client and thera- pist. Bordin (1979) conceptualized the alliance as involving three components: goals, tasks, and bond. The goals component is the level of agreement between client and therapist on the objectives of treatment (e.g., anxiety reduction). The tasks component is the level of client–therapist agreement on the techniques (e.g., cogni- tive restructuring, dream interpretation) used to attain treatment goals. Finally, the bond is the degree of emotional connection (e.g., care, liking, trust) between client and therapist. In a meta-analysis of 112 studies, Horvath, Del Re, Flückiger, and Symonds (2011) found that client-rated alliance accounted for about 8% of outcome variance.

Current Research

In contrast to outcome feedback systems, we developed a sys- tem that focuses exclusively on psychotherapy process. We se- lected outcome expectations, empathy, and the alliance for routine monitoring because these processes: (a) are common across treat- ment approaches, (b) are emphasized in Wampold and Imel’s (2015) widely influential model of therapeutic change, and (c) are among the strongest predictors of treatment success.

We anticipated that the provision of common factors feedback (CFF) would help therapists to identify poor process. Indeed, therapists do not always share their client’s perceptions of thera- peutic process, as evidenced by relatively weak correlations be- tween therapist-rated process and client-rated/observer-rated pro- cess (Cecero, Fenton, Frankforter, Nich, & Caroll, 2001; Greenberg, Watson, Elliott, & Bohart, 2001). Not only did we want to assist therapists in identifying poor process, but we also wanted to help therapists to intervene in ways that would improve that process. Therefore, we created a manual detailing evidence- based strategies for enhancing outcome expectations (e.g., Con- stantino et al., 2012; Swift & Derthick, 2013), empathy (e.g., Bohart & Greenberg, 1997; Bruce, Shapiro, & Constantino, & Manber, 2010; Dowell & Berman, 2013), and the alliance (e.g., Hill & O’Brien, 1999; Safran & Muran, 2000; Safran & Muran, 2006), and through prestudy training and ongoing supervision, encouraged study therapists to employ these strategies when com- mon factor ratings were suboptimal.

The effects of the CFF system were tested using randomized, clinical trial (RCT) methodology. Given the exploratory nature of this research, we enrolled clinical analogues who reported at least a mild level of depressive symptoms on two separate occasions. These participants were randomly assigned to either treatment as usual (TAU) or TAU plus the CFF system (TAU � CFF). The

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

248 MCCLINTOCK ET AL.

CFF system monitored client ratings of outcome expectations, empathy, and the alliance and provided feedback on this informa- tion to clients and therapists in order to facilitate an open discus- sion about the therapeutic process. Our CFF system fed back information to both clients and therapists because there is some evidence that the provision of feedback to the client-therapist dyad is more effective than the provision of feedback to the therapist alone (De Jong et al., 2014) and because the provision of feedback to the client might increase the client’s sense of agency in treat- ment (see De Jong et al., 2014; Flückiger et al., 2012; Zuroff et al., 2007).

We hypothesized that clients in TAU � CFF would report greater increases in outcome expectations, empathy, and the alli- ance over the course of therapy, compared with clients in TAU. Because common factors are purportedly therapeutic (see Wampold & Imel, 2015) and thus improvements in the common factors should lead to better outcomes, we further hypothesized that clients in TAU � CFF would report greater decreases in depressive symptoms and greater increase in psychological well- being over the course of therapy, compared with clients in TAU.

Method

Participants

Clients. Seventy-nine undergraduates at a Midwestern university met inclusion criteria and were randomized to a treatment condition (see Procedure). These participants were either in their freshman (59.5%), sophomore (25.3%), junior (5.1%), or senior year (10.1%) of college, with a mean age of 19.3 years (SD � 3.0). Most (82.3%) identified as female. About 81.0% identified as White/Caucasian, 5.1% as Black or African American, 3.8% as American Indian or Alaska Native, 3.8% as Multiracial, 2.5% as Hispanic or Latino/ Latina, 2.5% as Asian or Asian American, and 1.3% as Middle Eastern. About 13.9% were currently receiving psychological or phar- macological treatment at the pretreatment assessment. Participants reported a mean BDI-II score at pretreatment (23.68; SD � 8.21) that fell in the moderate depression range (31 participants reported mild depression, 27 reported moderate depression, and 21 reported severe depression; see Beck et al., 1996).

Therapists. Client participants received treatment from one of six doctoral students in a clinical psychology training program. All therapist participants had completed graduate-level assessment and treatment courses and were involved in practicum/traineeship as- sociated with the training program. Therapists had acquired a mean of 313.17 face-to-face clinical hours (SD � 261.31) by the start of the study. Three therapists were male, and three were female. Therapists had a mean age of 26.00 years (SD � 2.19), and all identified as White/Caucasian. With regard to theoretical orienta- tion, three therapists identified as cognitive– behavioral, two iden- tified as integrative/eclectic, and one identified as humanistic.

Measures

Outcome expectations. The Outcome Expectations Question- naire (OEQ; Constantino, McClintock, McCarrick, Anderson, & Himawan, 2016) is a recently developed, 10-item measure of client outcome expectations. Each item reflects a facet of treatment outcome about which clients may form expectations (example

item: “My self-esteem”). Items are rated on a 7-point Likert scale ranging from (0) “I expect no improvement,” to (6) “I expect very substantial improvements.” Exploratory and confirmatory factor analyses (Constantino et al., 2016) of the OEQ items supported a two-factor solution, with one factor pertaining to the specific problems that bring the client to treatment (example item: “My distress about the problems that brought me to treatment”), and the second factor pertaining to more global issues (example item: “My sense of purpose”). These two factors have been found to be strongly correlated (rs ranged from 0.60 to 0.71; Constantino et al., 2016). We used total OEQ scores (sum of all items) in the current study. Akin to the original research (Constantino et al., 2016), the OEQ demonstrated good internal consistency in the present study (Cronbach’s alpha � .93 at pretreatment).

Empathy. The Barrett-Lennard Relationship Inventory- Empathy Scale (BLRI-E; Barrett-Lennard, 2015) is the most widely used client rated measure of empathy (Elliott et al., 2011). The 16 BLRI-E items (example item: “My counselor usually senses or realizes what I am feeling”) are rated on a 6-point Likert scale ranging from (�3) “No, I strongly feel that it is not true” to (3) “Yes, I strongly feel that it is true.” A total BLRI-E score is derived by taking the mean of all items (after reverse-scoring eight items). Past research has established the internal consistency, test–retest reliability, convergent/divergent validity, and predictive validity of the BLRI-E (see Barrett-Lennard, 2015). The BLRI-E exhibited acceptable internal consistency in the current study (Cronbach’s alpha � .73 after Session 1).

Therapeutic alliance. The Working Alliance Inventory-Short Form Revised (WAI-SR; Hatcher & Gillaspy, 2006) is a widely used 12-item measure of the therapeutic alliance. Each item (ex- ample item: “I feel that the things I do in therapy will help me to accomplish the changes that I want.”) is rated on a 5-point Likert scale ranging from 1 (seldom) to 5 (always) and loads onto one of three factors: goals (i.e., agreement on the goals of therapy), tasks (i.e., agreement on the tasks of therapy), and bond (i.e., the emotional connection between client and therapist). The measure has demonstrated excellent reliability, a clean factor structure, convergent validity, and predictive validity (Hatcher & Gillaspy, 2006; McClintock, Anderson, & Petrarca, 2015). The total score (used for analyses) is calculated by summing all items. The inter- nal consistency of the WAI-SR was high in the present sample (Cronbach’s alpha � .88 after Session 1).

Depression. The Beck Depression Inventory-II (BDI-II; Beck, Steer, & Brown, 1996) is the most widely used measure of depressive symptoms. The measure features 21 items representing depressive symptoms. Respondents rate the presence of each symptom on a 4-point Likert scale. An example item is “Sadness” with response options (0) “I do not feel sad,” (1) “I feel sad much of the time,” (2) “I am sad all of the time,” (3) “I am so sad or unhappy that I can’t stand it.” BDI-II total scores (sum of all items) can be categorized in the following ranges: minimal (0 –13), mild (14 –19), moderate (20 –28), and severe (29 – 63). The BDI-II has sound psychometric properties in both clinical and nonclinical samples (Beck et al., 1996). The BDI-II demonstrated good inter- nal consistency in the current study (Cronbach’s alpha � .84 at pretreatment).

Psychological well-being. The Schwartz Outcome Scale-10 (SOS-10; Blais et al., 1999) is a 10-item self-report measure of psychological well-being. The SOS-10 was developed using clas-

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

249COMMON FACTORS FEEDBACK

sical test theory and Rasch item analysis and has been employed extensively to assess the effectiveness of mental health treatments. Each item features a 7-point Likert scale ranging from 0 (never) to 6 (nearly all of the time). Sample items include “I have confidence in my ability to sustain important relationships” and “I am gener- ally satisfied with my psychological health.” The SOS-10 is scored by summing the 10 items (higher scores indicate better well- being). The measure has demonstrated good internal consistency, test–retest reliability, and convergent/discriminant validity (Hag- gerty, Blake, Naraine, Siefert, & Blais, 2010; Young, Waehler, Laux, McDaniel, & Hilsenroth, 2003). The SOS-10 exhibited high internal consistency in the present research (Cronbach’s alpha � .84 at pretreatment).

Client satisfaction survey. We developed a brief client satis- faction survey to evaluate client perceptions about the CFF system. Because we were concerned that clients might find the completion of measures burdensome, we asked clients to rate the degree to which they enjoyed the completion of measures at the end of each session using a 7-point Likert scale (1 � not at all, 7 � very much). In addition, clients were asked about the degree to which the feedback reports helped improve treatment on a 7-point Likert scale (1 � not at all, 7 � very much). Clients completed the satisfaction survey at the end of their treatment.

Therapist satisfaction survey. We also developed a brief therapist satisfaction survey to evaluate the degree to which ther- apists were satisfied with the CFF system and found it useful. Clinicians were asked to rate their satisfaction on a 5-point Likert scale (1 � dissatisfied, 5 � completely satisfied) and the utility of

the CFF system on a 5-point Likert scale (1 � not useful, 5 � very useful). Therapists completed the satisfaction survey at the end of the research project.

Procedures

This study was conducted in the psychology department of a large Midwestern university during the 2015–2016 academic year. Institutional review board approval was obtained, and all ethical standards were followed; no adverse events were reported during the study. To be consistent with research on outcome feedback (e.g., see Connolly Gibbons et al., 2015), we aimed to recruit a sample of 75–100 participants. See Figure 1 for a procedure flowchart.

We recruited undergraduates with depressive symptoms via the psychology department’s Web based screening system. Specifi- cally, we administered the BDI-II in the screening system (n � 1862) and recruited only those who scored in the mild range or higher (i.e., �14; see Beck et al., 1996). These students reporting mild, moderate, or severe depressive symptoms (n � 463) were given a vague description of the study (titled “A Study of Psycho- therapy”) and were offered time slots; participation in the RCT was on a first-come, first-serve basis.

The RCT was conducted in a psychotherapy laboratory on the university’s campus. Students arrived to the laboratory individually and all provided informed consent (n � 95). The OEQ, BDI-II, and SOS-10 were then administered for the pretreatment assessment. At this pretreatment assessment, 13 participants did not score in the mild

40 completed at least two sessions 32 completed all five sessions

29 completed at least two sessions 24 completed all five sessions

Assessed for eligibility via screening assessment (n=1862) 463 scored 14 or higher on BDI-II and were eligible for study

Excluded (n=16) 13 scored below 14 on BDI-II 3 exhibited active suicidality, mania,

or psychosis

Analyzed (n=35) Excluded from analysis (n=0)

Allocated to TAU (n=44)

Analyzed (n=44) Excluded from analysis (n=0)

Randomized (N=79)

Assessed for eligibility via pretreatment assessment (n=95)

Enrollment

Alloca�on

Analysis

Allocated to TAU+CFF (n=35)



 

Figure 1. Procedure flow chart.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

250 MCCLINTOCK ET AL.

range or higher on the BDI-II (i.e., score �14). These 13 participants were immediately deemed ineligible (and no additional data were collected) to maintain the integrity of a symptomatic sample. There- fore, to be eligible for the RCT, participants had to score in the mild range or higher on the BDI-II at both the screening assessment and pretreatment assessment (time between these assessments ranged from 3 days to 8 weeks). Participants were also excluded for exhib- iting or reporting active suicidality, mania, or psychosis (n � 3). All excluded participants were referred to local mental health providers. The remaining 79 participants were randomly assigned to either TAU (n � 44) or TAU � CFF (n � 35); the first author determined condition assignment using a table of random numbers.

Therapists were crossed with treatment condition (to balance therapist skill across conditions; see Heppner, Wampold, Owen, Thompson, & Wang, 2016). Therapists were assigned clients based on mutual availability, but they did not know which condi- tion a client was assigned to until after the pretreatment assess- ment. Therapists’ beliefs about the effectiveness of each treatment condition were assessed after they were trained to use the CFF system; on a 5-point Likert scale (1 � ineffective, 5 � highly effective), therapists reported a mean rating of 3.83 (SD � 0.75) for TAU and a mean rating of 4.50 (SD � 0.55) for TAU � CFF. Therapists participated in weekly, group supervision to discuss individual cases and to maintain adherence to the CFF system. Supervision was provided by a licensed clinical psychologist with over 25 years of clinical experience.

Both TAU and TAU � CFF entailed five, 50-min individual treatment sessions delivered once per week. The treatments were limited to five sessions because of department restrictions on the use of the subject pool. Five session treatments have been shown to be effective in past research (e.g., McClintock, Anderson, & Cranston, 2015). To increase external validity, therapists selected from a range of evidence-based treatment approaches based on their theoretical orientation, case conceptualization, and supervisor input. The following treatment approaches were used in TAU: cognitive– behavioral (50%), emotion-focused (22%), mindfulness/acceptance- based (13%), client-centered (13%), and interpersonal (3%). The following treatment approaches were used in TAU � CFF: cognitive– behavioral (46%), emotion-focused (31%), mindfulness/acceptance- based (12%), client-centered (8%), and interpersonal (4%). As previ- ously noted, the OEQ, BDI-II, and SOS-10 were administered at pretreatment (i.e., before the first session). The OEQ, BLRI-E, and WAI-SR were administered to clients after the first session. The OEQ, BLRI-E, WAI-SR, BDI-II, and SOS-II were administered to clients after sessions two through five.

In total, clients attended a mean of 4.13 sessions (SD � 1.48, range � 1–5 sessions). Clients dropped from the study for a variety of reasons (e.g., study too cumbersome, no longer interested in therapy, etc.); 69 (40 in TAU and 29 in TAU � CFF) completed at least two sessions, and 56 (32 in TAU and 24 in TAU � CFF) completed all five sessions. Clients who completed all treatment sessions were compensated with $10 and five course credits (par- tial credit was awarded for partial participation).

CFF System. The CFF system was a novel procedure devel- oped for the current research. The CFF system monitors client ratings of three common factors (i.e., outcome expectations, em- pathy, and alliance) and provides feedback on this information to clients and therapists in order to facilitate an open discussion about

the therapeutic process and to help therapists to make adjustments when process is suboptimal.

In Session 1 or the beginning of Session 2 in TAU � CFF, therapists described the three common factors (outcome expecta- tions, empathy, and alliance) and provided a jargon-free rationale for using the CFF system (e.g., “Each of these components is strongly related to treatment success, and so by maximizing these components in our treatment, we might be able to maximize your improvement as well”). Clients were told that their ratings of these factors would be reviewed and discussed in session.

As mentioned, clients completed the OEQ, BLRI-E, and WAI-SR after each session. For TAU � CFF clients, these ratings were entered into an Excel spreadsheet (Microsoft, 2013) that visually depicted the client’s ratings in a line graph relative to percentile-based tracks. The tracks, derived from normative data (Anderson, Patterson, McClintock, & Song, 2013; Barrett- Lennard, 2015; Constantino et al., 2016), are color-coded green (highest 33% of scores in normative data), yellow (middle 33% of scores), and red (lowest 33% of scores). High, middle, and low tracks were created for each of the following variables: outcome expectations (i.e., OEQ scores), empathy (i.e., BLRI-E scores), alliance (i.e., WAI-SR-Total scores), goals facet of the alliance (i.e., WAI-SR-Goals scores), tasks facet of the alliance (WAI-SR- Tasks scores), and the bond facet of the alliance (i.e., WAI-SR- Bond scores). In this way, clients and therapists could view the client’s common factors ratings over time (i.e., within-client change) as well as relative to normative data (i.e., between clients). A screen shot of the Excel output is presented in Figure 2, showing a client’s alliance scores by the beginning of the fifth session.

In addition to the Excel graphs, a common factors enhancement manual was created that details the general principles underlying the CFF system and the specific strategies that can be employed to enhance outcome expectations, empathy, and the alliance.1 In creating this manual, we drew heavily from existing strategies and guidelines (e.g., Bohart & Greenberg, 1997; Bruce et al., 2010; Constantino et al., 2012; Dowell & Berman, 2013; Safran & Muran, 2000; Safran & Muran, 2006; Swift & Derthick, 2013). Prior to study initiation, therapists were asked to read the manual and to role-play the discussion of client common factors ratings and the delivery of common factors enhancement strategies. Im- portantly, while the viewing of client data in the TAU condition was forbidden, therapists were not forbidden from using the com- mon factors enhancement strategies with their TAU clients.

Therapists were instructed to review the outcome expectations, empathy, and alliance graphs with their TAU � CFF clients at the beginning of Sessions 2–5 and to initiate an exploration of the client’s perspective, particularly when ratings were suboptimal (i.e., in the yellow or red tracks). TAU � CFF data discussions were designed to be collaborative between client and therapist; clients were invited to share their perceptions of therapeutic pro- cesses, and therapists were instructed to validate the client’s per- ceptions while employing techniques— adapted to the individual needs of the client—to bolster outcome expectations, empathy, and/or the therapeutic alliance. For example, a therapist could intervene with a client reporting low WAI-SR scores by exploring

1 For a copy of the common factors enhancement manual, please contact first author.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

251COMMON FACTORS FEEDBACK

and attempting to repair alliance ruptures (see Safran & Muran, 2000).

CFF system adherence. Near the end of the research project, therapists were surveyed about their adherence to the CFF system. Therapists were first asked how frequently they discussed the feedback with TAU � CFF clients; on a 5-point Likert scale (1 � never, 5 � always), therapists reported a mean rating of 4.67 (SD � 0.82). Therapists were also asked how much time they spent, in an average TAU � CFF session, discussing the feedback; with the options 0 –1 min, 1–5 min, 5–10 min, 10 –20 min, and �20 min, three therapists reported 1–5 min, and the other three therapists reported 5–10 min. Finally, therapists were asked about the extent to which the feedback influenced their interven- tion strategy; on a 5-point Likert scale (1 � not at all, 5 � substantially), therapists reported a mean rating of 3.67 (SD � 0.52).

Plan of Analysis

Correlations (r) were used to investigate associations between study measures. To assess differences on demographic and pre- treatment data between conditions, independent samples t tests and chi-square tests of independence were used. An independent sam- ples t test and a logistic regression were used to evaluate differ- ences in drop out/number of sessions attended. To assess clinically significant change, we identified participants who evidenced a 20% reduction in BDI-II scores and fell in the nonclinical range on the BDI-II (i.e., score �13) by the end of treatment (see Borkovec, Newman, Pincus, & Lytle, 2002; McClintock et al., 2015; Roemer, Orsillo, & Salters-Pedneault, 2008). Satisfaction ratings were an- alyzed with descriptive statistics.

To model changes in process (i.e., WAI-SR, BLRI-E and OEQ) and outcome (i.e., BDI-II and SOS-10) measures, a three-level hierarchical linear model (HLM) was used for each measure with sessions nested within clients and clients nested within therapists. Thus, within-client variability was modeled at Level 1, the between-client and within-therapist variability was modeled at

Level 2, and the between-therapist variability was modeled at Level 3. Time/session variables were entered as Level 1 predictors. The time/session variables were centered at the first session (i.e., first session coded as 0, second session coded as 1, etc.). Because randomization to treatment conditions occurred at the client level, treatment condition was entered as a Level 2 predictor. Treatment condition was centered at TAU (i.e., TAU coded as 0, TAU � CFF coded as 1). Analyses did not include Level 3 predictors.

For each process/outcome measure, an unconditional growth curve was fitted first to investigate whether scores changed sig- nificantly over time. These unconditional growth curves only included time/session variable(s) as predictor(s). If a time/session predictor was significant (i.e., significant change in scores over time), then treatment condition was added as a Level 2 predictor to investigate whether the change over time differed between TAU and TAU � CFF.

To account for the different shapes that the growth curve might take, four different unconditional growth curves were fitted to the data, and the best model was obtained by comparing the informa- tion criteria (i.e., Akaike Information Criteria [AIC] and Bayesian Information Criteria [BIC]). The four unconditional growth curves were as follows: (a) a linear unconditional growth curve (i.e., a model with only a linear term of session number included as the Level 1 predictor) to assess the possibility that scores decrease or increase at a constant rate over time; (b) a log unconditional growth curve (i.e., a model with only a log of session number included as the Level 1 predictor) to assess the possibility that scores decrease or increase at a faster rate during the early ses- sions, then decrease or increase at a slower rate during the later sessions; (c) a quadratic unconditional growth curve (i.e., a model with linear and quadratic terms of session number as the Level 1 predictors) to assess the possibility that scores first decrease over time then increase or first increase then decrease; and (5) a cubic unconditional quadratic growth curve (i.e., a model with linear, quadratic, and cubic terms of session number as the Level 1 predictors) to assess the possibility that scores decrease first over

Figure 2. Example of feedback graph. See the online article for the color version of this figure.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

252 MCCLINTOCK ET AL.

time, then increase before decreasing again or increase first, then decrease before increasing again.

For illustration purposes, the model fitted for the linear uncon- ditional growth curve is provided below:

Level 1:

(Measure)tij � �0ij � �1ij(Session)tij � etij

Level 2:

�0ij � �00j � r0ij �1ij � �10j � r1ij

Level 3:

�00j � �000 � u00j �10j � �100 � u10j

The complete model:

(Measure)tij � �000 _ �100(Session)tij � [u00j(Session)tij � r0ij

� r1ij(Session)tij � etij]

In the previous model, (Measure)tij is the process/outcome measure (i.e., BDI-II, SOS-10, WAI-SR, BLRI-E, or OEQ) at time t for client i seeing therapist j; because Session was centered at the first session (i.e., immediately before first session for BDI-II, SOS-10 and OEQ and immediately after first session for BLRI-E and WAI-SR), �000 is the average of the scores at the first session; and �100 is the rate of change of the scores over one unit of time (i.e., session). A significant �000 means that the average of the scores at the first session is significantly different than zero. A significant �100 means that the scores change significantly over time (i.e., the rate of change of the scores is significantly different than zero). The parameters inside the brackets are the random effects: etij is the session variability within a client; r0ij and r1ij are client variability within a therapist around �000 and �100, respectively; and u00j and u10j are therapist variability around �000 and �100, respectively. In the beginning of the model fitting, �000 and �100 were treated as random effects at both Levels 2 and 3. However, when there was an indication that the model was overspecified, these random effects were dropped one by one starting from the highest level, until the model fit properly.

Also for illustration purposes, the linear model fitted with treat- ment condition (TC) as a Level 2 predictor is provided here:

Level 1:

(Measure)tij � �0ij � �1ij(Session)tij � etij

Level 2:

�0ij � �00j � �01j(TC)ij � r0ij �1ij � �10j � �11j(TC)ij � r1ij

Level 3:

�00j � �000 � u00j �01j � �010 �10j � �100 � u10j �11j � �110

The complete model:

(Measure)tij � �000 � �010(TC)ij � �100(Session)tij

� �110(TC)ij(Session)tij � [u00j � u10j(Session)tij

� r0ij � r1ij(Session)tij � etij]

In the previous model, (Measure)tij is the process/outcome measure at time t for client i seeing therapist j; because Session was centered at the first session and Treatment Condition was centered at TAU, �000 is the average of the TAU scores at the first session; �010 is the effect of TAU � CFF on �000; �100 is the rate of change of TAU scores over one unit of time (i.e., session); and �110 is the effect of TAU � CFF on �100. A significant �000 means that the average of the TAU scores at the first session is significantly different than zero; a significant �010 means that the average of TAU � CFF scores at the first session is significantly different than that of TAU; a significant �100 means that the rate of change of TAU scores is significantly different than zero; and a significant �110 means that the rate of change of TAU � CFF scores is significantly different than that of TAU. The parameters inside the brackets are the random effects as described previously. In the beginning of the model fitting, �000 and �100 were also treated as random effects at both Levels 2 and 3. However, when there was an indication that this model was over- specified, these random effects were dropped one by one starting from the highest level, until the model fit properly.

Because the main goal of the study was to investigate whether there was a significant difference in the rate of change of the process/ outcome scores over time between clients in the TAU and TAU � CFF conditions, the focus of the study was �110. Because clients were randomized into the two conditions, we did not expect that the two conditions would significant differ in average scores at the first session (i.e., �010).

Results

Preliminary Analyses

Data were evaluated and found to be within normal limits in regards to outliers and degree of normality. Correlations (r) between study measures at first session are presented in Table 1. Correlation size ranged from trivial (e.g., correlation between BLRI-E and SOS- 10) to large (e.g., correlation between BLRI-E and WAI-SR), al- though even the large correlations were not so large as to suggest measure redundancy. At pretreatment, TAU participants and TAU � CFF participants did not significantly differ (p � .05) on any of the demographic and pretreatment data, implying that randomization was successful. An independent samples t test showed that TAU partici- pants and TAU � CFF participants did not significantly differ (p � .05) on the number of sessions attended. Similarly, a logistic regres- sion showed that TAU and TAU � CFF did not significant differ (p � .05) in the number of participants who dropped out (i.e., did not complete all five sessions). Of the 79 enrolled participants (TAU n � 44, TAU � CFF n � 35), 45.6% (TAU n � 21, TAU � CFF n � 15) achieved clinically significant change (i.e., evidenced a 20% reduc- tion in BDI-II scores and fell in the nonclinical range on the BDI-II by the end of treatment).

Client Satisfaction Ratings

At the end of treatment, clients in TAU � CFF were asked about the degree to which they enjoyed the completion of measures at the end of each session; clients reported a mean rating of 5.15 (SD � 1.35) on a 7-point Likert scale (1 � not at all, 7 � very much). Clients in TAU � CFF were also asked about the degree to which the feedback reports helped improve treatment; clients reported a mean

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

253COMMON FACTORS FEEDBACK

rating of 5.63 (SD � 1.15) on a 7-point Likert scale (1 � not at all, 7 � very much).

Therapist Satisfaction Ratings

At the end of the research project, therapists were asked about their level of satisfaction with the CFF system; therapists reported a mean rating of 4.17 (SD � 0.75) on a 5-point Likert scale (1 � dissatisfied, 5 � completely satisfied). Therapists were also asked about the degree to which they found the CFF system to be useful; therapists reported a mean rating of 4.00 (SD � 0.89) on a 5-point Likert scale (1 � not useful, 5 � very useful).

Between-Group Effects on Process and Outcome

A three-level HLM was fitted for each process and outcome variable. Comparison of AIC and BIC showed that BDI-II, SOS- 10, and OEQ were best represented with linear unconditional growth curves, while WAI-SR and BLRI-E were best represented with log unconditional growth curves. The final unconditional growth curves were: (1) BDI-II

(BDI-II)tij � �000 � �100(Session)tij � [u00j � u10j(Session)tij � r0ij

� r1ij(Session)tij � etij]

(2) SOS-10

(SOS-10)tij � �000 � �100(Session)tij � [u10j(Session)tij � r0ij

� r1ij(Session)tij � etij]

(3) WAI-SR

(WAI-SR)tij � �000 � �100(LogSession)tij

� [r0ij � r1ij(LogSession)tij � etij]

(4) BLRI-E

(BLRI-E)tij � �000 � �100(LogSession)tij � [u10j(LogSession)tij

� r0ij � r1ij(LogSession)tij � etij]

(5) OEQ

(OEQ)tij � �000 � �100(Session)tij � [r0ij � r1ij(Session)tij � etij]

Results indicated that the average of the scores at the first session (i.e., �000) and the rate of change over time/session (i.e., �100) were

significantly different than zero. As expected, BDI-II scores decreased over time, while SOS-10, WAI-SR, BLRI-E, and OEQ scores in- creased over time. Over one unit of time/session, BDI-II scores decreased by 2.74 points, SOS-10 scores increased by 2.42 points, WAI-SR scores increased by 5.75 points, BLRI-E scores increased by 0.32 points, and OEQ scores increased by 2.21 points. A summary of the unconditional growth curve results is presented in Table 2.

In the next set of analyses, treatment condition was entered as a Level 2 predictor. The final conditional growth curves were: (1) BDI-II

(BDI-II)tij � �000 � �010(TC)ij � �100(Session)tij

� �110(TC)ij(Session)tij � [u00j � u10j(Session)tij

� r0ij � r1ij(Session)tij � etij]

(2) SOS-10

(SOS-10)tij � �000 � �010(TC)ij � �100(Session)tij

� �110(TC)ij(Session)tij � [u10j(Session)tij � r0ij

� r1ij(Session)tij � etij]

(3) WAI-SR

(WAI-SR)tij � �000 � �010(TC)ij � �100(LogSession)tij

� �110(TC)ij(LogSession)tij � [u10j(LogSession)tij

� r0ij � r1ij(LogSession)tij � etij]

(4) BLRI-E

(BLRI-E)tij � �000 � �010(TC)ij � �100(LogSession)tij

� �110(TC)ij(LogSession)tij � [u10j(LogSession)tij

� r0ij � r1ij(LogSession)tij � etij]

(5) OEQ

(OEQ)tij � �000 � �010(TC)ij � �100(Session)tij

� �110(TC)ij(Session)tij � [r0ij � r1ij(Session)tij � etij]

Results indicated that for each process/outcome measure, the average of TAU scores at the first session (i.e., �000) was signif- icantly different than zero. For each process/outcome measure, the effect of the TAU � CFF condition on �000 (i.e., �010) was not significant. This implies that, as would be expected given random-

Table 1 Means (SDs) and Correlations for Study Measures at First Session (N � 79)

Study measures M (SD) BDI-II SOS-10 WAI-SR BLRI-E OEQ

BDI-II 23.68 (8.21) �.65��� �.17 .22 .03 SOS-10 31.26 (8.32) .19 �.02 .08 WAI-SR 43.60 (7.83) .63��� .48���

BLRI-E 1.61 (.50) .43���

OEQ 39.42 (11.23)

Note. BDI-II � Beck Depression Inventory-II (before first session); SOS-10 � Schwartz Outcome Scale-10 (before first session); WAI-SR � Working Alliance Inventory-Short Form Revised (after first session); BLRI-E � Barrett-Lennard Relationship Inventory-Empathy Scale (after first session); OEQ � Outcome Expectations Questionnaire (after first session). ��� p � .001.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

254 MCCLINTOCK ET AL.

ization, TAU and TAU � CFF did not significantly differ in process/outcome scores at the first session.

Results also indicated that for each process/outcome mea- sure, the rate of change of TAU scores over time/session (i.e., �100) was significantly different than zero; directions of change were as expected (i.e., BDI-II scores decreased and SOS-10, WAI-SR, BLRI-E, and OEQ scores increased). Over one unit of time/session in the TAU condition, BDI-II scores decreased by 2.46 points, SOS-10 scores increased by 2.18 points, WAI-SR scores increased by 4.63 points, BLRI-E scores increased by 0.22 points, and OEQ scores increased by 2.01 points.

The effect of the TAU � CFF condition on �100 (i.e., �110) was significant for WAI-SR and BLRI-E. That is, while TAU and TAU � CFF did not significantly differ in the rates of change of BDI-II, SOS-10, and OEQ scores, the two conditions significantly differed in the rates of change of WAI-SR and BLRI-E scores. Specifically, participants in TAU � CFF re- ported greater increases in WAI-SR and BLRI-E scores relative to TAU participants (over one unit of time/session, WAI-SR scores increased by 2.61 points more in TAU � CFF relative to TAU and BLRI-E scores increased by 0.20 points more in TAU � CFF relative to TAU). A summary of the conditional growth curve results is presented in Table 3. Figures 3 and 4 depict mean BLRI-E and WAI-SR scores at each session for TAU and TAU � CFF.

We calculated the proportions of variance explained at Level 1 coefficients (i.e., 0ij and 1ij) by treatment condition, above and beyond the time/session variable. Treatment condition ac- counted for 0.59%, 0.47%, 0.49%, 1.01%, and 0.36% of the variance in 0ij for BDI-II, SOS-10, WAI-SR, BLRI-E and OEQ scores, respectively, and accounted for 2.77%, 0.81%, 9.01%, 13.67%, and 0.70% of the variance in 1ij for BDI-II, SOS-10, WAI-SR, BLRI-E, and OEQ scores, respectively.

Discussion

The present research marks the first attempt to develop and evaluate a common factors feedback (CFF) intervention. Re- sults suggest that our CFF system holds promise. Clients and therapists reported satisfaction with the CFF system and en- dorsed its utility. Multilevel modeling showed that, while there were no between-groups effects on client ratings of outcome expectations, depressive symptoms, and psychological well- being, treatment condition had a medium-to-large sized effect (see Lambert, 2013) on empathy (accounted for about 13.7% of variability) and alliance ratings (accounted for about 9.0% of the variability). Specifically, clients who received treatment with CFF reported greater increases in perceived empathy and alliance over the course of treatment relative to clients who received treatment as usual. These results imply that our brief feedback intervention, which on average took less than 10 min to implement per session, may have enhanced the process of psychotherapy.

Although we did not assess how the CFF system produced these results, we can speculate about potential mechanisms. Outcome feedback systems, on which our CFF system is based, are effective in part because they help to identify patients who are at risk for treatment failure. Research shows that, unaided, therapists are relatively poor in identifying off-track patientsT

ab le

2 E

st im

a te

s o f

th e

U n co

n d it

io n a l

G ro

w th

C u rv

es P

a ra

m et

er s

W it

h S E

s

M ea

su re

s

F ix

ed ef

fe ct

s co

ef fi

ci en

ts (S

E )

R an

do m

ef fe

ct s

va ri

an ce

co m

po ne

nt s

(S E

)

� 0 0 0

� 1 0 0

V ar

(u 0 0

j) V

ar (u

1 0

j) C

ov (u

0 0

j,u 1 0

j) V

ar (r

0 ij )

V ar

(r 1 ij )

C ov

(r 0 ij ,r

1 ij )

e t ij

B D

I- II

23 .4

9� � �

(1 .0

41 )

� 2.

74 � � �

(. 39

2) .6

7 (4

.6 54

) .4

3 (.

46 3)

� .5

3 (1

.0 35

) 66

.1 1�

� �

(1 2.

79 8)

3. 51

� � �

(. 94

3) �

3. 94

(2 .5

86 )

13 .6

5� � �

(1 .4

43 )

S O

S -1

0 30

.9 5�

� �

(. 92

1) 2.

42 � �

(. 36

1) .1

7 (.

35 1)

57 .7

4� � �

(1 0.

67 1)

5. 00

� � �

(1 .1

95 )

� 4.

50 (2

.6 43

) 13

.1 4�

� �

(1 .3

95 )

W A

I- S

R 43

.4 8�

� �

(. 88

3) 5.

75 � � �

(. 60

7) 51

.1 4�

� �

(9 .8

27 )

16 .9

0� � �

(4 .5

82 )

� 11

.4 8�

(5 .4

08 )

12 .0

3� � �

(1 .2

76 )

B L

R I-

E 1.

60 � � �

(. 05

6) .3

2� �

(. 06

6) .0

1 (.

01 5)

.1 8�

� �

(. 04

1) .0

7� �

(. 02

4) �

.0 2

(. 02

4) .0

7� � �

(. 00

8) O

E Q

36 .0

6� � �

(1 .2

53 )

2. 21

� � �

(. 32

6) 11

3. 16

� � �

(1 9.

74 4)

6. 19

� � �

(1 .3

31 )

� 11

.1 5�

� (3

.9 75

) 19

.1 0�

� �

(1 .6

91 )

N o te

. B

D I-

II �

B ec

k D

ep re

ss io

n In

ve nt

or y-

II ;

S O

S -1

0 �

S ch

w ar

tz O

ut co

m e

S ca

le -1

0; W

A I-

S R

� W

or ki

ng A

ll ia

nc e

In ve

nt or

y- S

ho rt

F or

m R

ev is

ed ;

B L

R I-

E �

B ar

re tt

-L en

na rd

R el

at io

ns hi

p In

ve nt

or y-

E m

pa th

y S

ca le

; O

E Q

� O

ut co

m e

E xp

ec ta

ti on

s Q

ue st

io nn

ai re

. F

or in

te rp

re ta

ti on

of �

0 0 0 ,

� 1 0 0 ,

u 0 0

j, u 1

0 j,

r 0 ij ,

r 1 ij

an d

e t ij ,

pl ea

se se

e “P

la n

of A

na ly

si s”

se ct

io n.

� �

p �

.0 1.

� �

� p

� .0

01 .

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

255COMMON FACTORS FEEDBACK

(Hannan et al., 2005), and so any tool that detects these patients serves a critical need. This same concept may translate to process-based feedback systems. Specifically, given that therapist-rated process is relatively weakly correlated with both client-rated and observer-rated process (Cecero et al., 2001; Greenberg et al., 2001), it stands to reason that poor process is frequently missed. Our CFF system might thus be useful be- cause it helps therapists perform an otherwise challenging task—the identification of poor process.

Once poor process is recognized, therapists are presumably in a better position to tailor their behavior to the specific needs of the client (see therapist responsiveness; Stiles, Honos-Webb, & Surko, 1998). For example, therapists could respond to low empathy ratings by not only exploring areas of misunderstand- ing but by also increasing reflective listening and validation. This adaptation of behavior to match the client’s needs would likely improve process and the client’s perceptions of that process.

Although this discussion focuses on therapist behavior, it is critical that we not lose sight of the important role that clients can play in process development. Flückiger and colleagues (2012) showed that clients who are explicitly encouraged to be proactive participants in their treatment tend to report greater improvements in the alliance relative to clients who do not receive this encouragement. Thus, it could be that CFF facili- tates increased client agency and engagement in treatment, which in turn improves process (see also Ryan & Deci, 2008; Zuroff et al., 2007). Examination of these and other mecha- nisms should be a high priority for future research.

Our finding that CFF influenced empathy and alliance but not treatment outcome ratings may, at first blush, appear inconsis- tent with common factors theory. That is, if common factors are therapeutic, then an improvement in the common factors should coincide with an improvement in treatment outcomes. A num- ber of factors could explain our nonsignificant effects on treat- ment outcome (as well as on outcome expectations). First, our study was underpowered for the purpose of detecting small between-groups effects. A second explanation is that our short treatment length (i.e., five sessions) may have precluded some between-groups effects. Indeed, the benefits of outcome feed- back are relatively minimal over the first few sessions (De Jong et al., 2014). Not only could treatment length be an issue, but the amount of time devoted to feedback discussion may have been insufficient; feedback was discussed, on average, less than 10 min per session, and so a greater focus on feedback may be needed to maximize its effects. Alternatively, the more time spent on process leaves less time for the client’s presenting issue, and so an increased focus on process may unwittingly attenuate treatment outcome effects. Yet another explanation for our nonsignificant effects pertains to our analog sample; by enrolling undergraduates with only moderate depressive symp- toms (M BDI-II score � 23.68), floor/ceiling effects could have contributed to our nonsignificant findings. It could also be the case that feedback effects are specific to the variables that are monitored; outcome feedback may primarily affect outcome, and process feedback may primarily affect process. Future research is needed to determine whether our nonsignificant findings reflect Type II errors or reflect inherent limitations of the feedback system.T

ab le

3 E

st im

a te

s o f

th e

C o n d it

io n a l

G ro

w th

C u rv

es P

a ra

m et

er s

W it

h S E

s

M ea

su re

s

F ix

ed ef

fe ct

s co

ef fi

ci en

ts (S

E )

R an

do m

ef fe

ct s

va ri

an ce

co m

po ne

nt s

(S E

)

� 0 0 0

� 0 1 0

� 1 0 0

� 1 1 0

V ar

(u 0 0

j) V

ar (u

1 0

j) C

ov (u

0 0

j,u 1 0

j) V

ar (r

0 ij )

V ar

(r 1 ij )

C ov

(r 0 ij ,r

1 ij )

e t ij

B D

I- II

23 .0

8� � �

(1 .3

60 )

.9 1

(1 .9

81 )

� 2.

46 � �

(. 48

0) �

.6 0

(. 56

7) .8

1 (4

.7 02

) .4

8 (.

49 6)

� .6

4 (1

.0 83

) 65

.7 3�

� �

(1 2.

74 4)

3. 41

� � �

(. 92

7) �

3. 67

(2 .5

58 )

13 .6

3� � �

(1 .4

41 )

S O

S -1

0 30

.5 5�

� �

(1 .2

27 )

.9 1

(1 .8

53 )

2. 18

� �

(. 47

5) .5

0 (.

63 8)

.2 5

(. 40

9) 57

.4 7�

� �

(1 0.

61 7)

4. 96

� � �

(1 .1

86 )

� 4.

69 (2

.6 51

) 13

.1 1�

� �

(1 .3

91 )

W A

I- S

R 43

.0 5�

� �

(1 .1

86 )

.9 8

(1 .7

72 )

4. 63

� � �

(. 78

2) 2.

61 �

(1 .1

83 )

50 .8

9� � �

(9 .7

76 )

15 .3

8� � �

(4 .2

82 )

� 11

.8 8�

(5 .2

63 )

11 .9

7� � �

(1 .2

65 )

B L

R I-

E 1.

58 � � �

(. 07

5) .0

3 (.

11 2)

.2 2�

(. 08

4) .2

0� (.

08 7)

.0 2

(. 01

8) .1

8� � �

(. 04

0) .0

6� �

(. 02

2) �

.0 2

(. 02

3) .0

7� � �

(. 00

8) O

E Q

35 .5

6� � �

(1 .6

76 )

1. 13

(2 .5

18 )

2. 01

� � �

(. 43

5) .4

5 (.

65 5)

11 2.

75 � � �

(1 9.

67 5)

6. 15

� � �

(1 .3

22 )

� 11

.2 0�

� (3

.9 60

) 19

.0 8�

� �

(1 .6

88 )

N o te

. B

D I-

II �

B ec

k D

ep re

ss io

n In

ve nt

or y-

II ;

S O

S -1

0 �

S ch

w ar

tz O

ut co

m e

S ca

le -1

0; W

A I-

S R

� W

or ki

ng A

ll ia

nc e

In ve

nt or

y- S

ho rt

F or

m R

ev is

ed ;

B L

R I-

E �

B ar

re tt

-L en

na rd

R el

at io

ns hi

p In

ve nt

or y-

E m

pa th

y S

ca le

; O

E Q

� O

ut co

m e

E xp

ec ta

ti on

s Q

ue st

io nn

ai re

. F

or in

te rp

re ta

ti on

of �

0 0 0 ,

� 0 1 0 ,

� 1 0 0 ,

� 1 1 0 ,

u 0 0 j,

u 1 0 j,

r 0 ij ,

r 1 ij

an d

e t ij ,

pl ea

se se

e “P

la n

of A

na ly

si s”

se ct

io n.

� p

� .0

5. � �

p �

.0 1.

� � �

p �

.0 01

.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

256 MCCLINTOCK ET AL.

There are additional points to be made regarding this study’s limitations. Although the array of implemented evidence-based therapies increases the generalizability of findings, generaliz- ability is limited by the short therapy duration, participant compensation, and use of a mostly White and female analog sample. A different pattern of results could have emerged, for instance, with clients who are more difficult to engage in therapy (e.g., clients with personality disorders, chronic depres- sion, etc.) and who require longer-term treatment (see De Jong et al., 2014). It is also worth noting that study therapists were involved in the design of the CFF system and, as such, may have had a strong allegiance (see De Jong, van Sluis, Nugter, Heiser, & Spinhoven, 2012; Falkenström, Markowitz, Jonker, Philips, & Holmqvist, 2013) to the TAU � CFF condition. Demand characteristics and social desirability bias may have influenced the present results as well; TAU � CFF clients knew their data would be reviewed and discussed with their therapist, and so they may have overreported process quality. This con- cern is somewhat mitigated, however, by Reese et al.’s (2013) finding that alliance scores are not influenced by the presence of a therapist or the knowledge that one’s scores would be reviewed by the therapist. Nevertheless, in light of these short- comings, we recommend that future investigations (a) enroll demographically diverse, treatment-seeking participants; (b) employ longer treatments; and (c) evaluate the CFF system relative to an outcome-based feedback system.

The CFF system developed in the present study represents a novel synthesis of outcome feedback systems and the common factors literature. Our CFF system monitors three common factors (i.e., outcome expectations, empathy, and alliance) over the course of therapy, visually presents these ratings—relative to normative data—to clients and therapists, and provides use- ful, empirically based strategies for improving suboptimal pro- cess. This approach has a number of strengths. First, since identifying poor process is the first step in repairing process, the CFF system fills a vital role in providing both a signal for off-track process and a context for collaboratively addressing concerns. Second, the CFF system yields targeted, actionable information that has direct implications for treatment planning. For example, low ratings on the tasks component of the alliance can be readily addressed by exploring discrepancies between the implemented techniques and the client’s perceptions about which techniques should be implemented in therapy. A third strength of the CFF system is that it focuses on factors common across treatments and thus could be useful in a wide range of settings and contexts. Finally, whereas outcome feedback is often met with fear and mistrust (Boswell, Kraus, Miller, & Lambert, 2015), feedback about what is simply transpiring in therapy might be more palatable for therapists and thus has the potential to be widely implemented. We are hopeful that our CFF system will advance outcome feedback and common fac-

Figure 3. Mean empathy scores over time for TAU and TAU � CFF. Note. BLRI-E � Barrett-Lennard Relationship Inventory-Empathy Scale; TAU � Treatment as usual; TAU � CFF � Treatment as usual plus common factors feedback. See the online article for the color version of this figure.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

257COMMON FACTORS FEEDBACK

tors literatures and will improve the care provided to psycho- therapy clients.

References

Anderson, T., Patterson, C. L., McClintock, A. S., & Song, X. (2013). Factorial and predictive validity of the expectations about counseling- brief (EAC-B) with clients seeking counseling. Journal of Counseling Psychology, 60, 496 –507. http://dx.doi.org/10.1037/a0034222

Barrett-Lennard, G. T. (1981). The empathy cycle: Refinement of a nuclear concept. Journal of Counseling Psychology, 28, 91–100. http://dx.doi .org/10.1037/0022-0167.28.2.91

Barrett-Lennard, G. T. (2015). The Relationship Inventory: A complete resource and guide. West Sussex, United Kingdom: Wiley.

Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Beck Depression Inventory (2nd ed.). San Antonio, TX: The Psychological Corporation.

Blais, M. A., Lenderking, W. R., Baer, L., deLorell, A., Peets, K., Leahy, L., & Burns, C. (1999). Development and initial validation of a brief mental health outcome measure. Journal of Personality Assessment, 73, 359 –373. http://dx.doi.org/10.1207/S15327752JPA7303_5

Bohart, A. C., & Greenberg, L. S. (1997). Empathy reconsidered: New directions in psychotherapy. Washington, DC: American Psychological Association. http://dx.doi.org/10.1037/10226-000

Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research, & Practice, 16, 252–260. http://dx.doi.org/10.1037/h0085885

Borkovec, T. D., Newman, M. G., Pincus, A. L., & Lytle, R. (2002). A component analysis of cognitive-behavioral therapy for generalized anx- iety disorder and the role of interpersonal problems. Journal of Consult-

ing and Clinical Psychology, 70, 288 –298. http://dx.doi.org/10.1037/ 0022-006X.70.2.288

Boswell, J. F., Kraus, D. R., Miller, S. D., & Lambert, M. J. (2015). Implementing routine outcome monitoring in clinical practice: Benefits, challenges, and solutions. Psychotherapy Research, 25, 6 –19. http://dx .doi.org/10.1080/10503307.2013.817696

Bruce, N., Shapiro, S. L., Constantino, M. J., & Manber, R. (2010). Psychotherapist mindfulness and the psychotherapy process. Psycho- therapy: Theory, Research, Practice, Training, 47, 83–97. http://dx.doi .org/10.1037/a0018842

Cecero, J. J., Fenton, L. R., Frankforter, T. L., Nich, C., & Caroll, K. M. (2001). Focus on therapeutic alliance: The psychometric properties of six measures across three instruments. Psychotherapy: Theory, Research, Practice, Training, 38, 1–11. http://dx.doi.org/10.1037/0033- 3204.38.1.1

Connolly Gibbons, M. B., Kurtz, J. E., Thompson, D. L., Mack, R. A., Lee, J. K., Rothbard, A., . . . Crits-Christoph, P. (2015). The effectiveness of clinician feedback in the treatment of depression in the community mental health system. Journal of Consulting and Clinical Psychology, 83, 748 –759. http://dx.doi.org/10.1037/a0039302

Constantino, M. J., Ametrano, R. M., & Greenberg, R. P. (2012). Clinician interventions and participant characteristics that foster adaptive patient expectations for psychotherapy and psychotherapeutic change. Psycho- therapy, 49, 557–569. http://dx.doi.org/10.1037/a0029440

Constantino, M. J., Glass, C. R., Arnkoff, D. B., Ametrano, R. M., & Smith, J. Z. (2011). Expectations. In J. Norcross (Ed.), Psychotherapy relationships that work: Evidence-based responsiveness (2nd ed., pp. 181–192). New York, NY: Oxford University Press. http://dx.doi.org/ 10.1093/acprof:oso/9780199737208.003.0018

Figure 4. Mean alliance scores over time for TAU and TAU � CFF. Note. WAI-SR � Working Alliance Inventory-Short Form Revised; TAU � Treatment as usual; TAU � CFF � Treatment as usual plus common factors feedback. See the online article for the color version of this figure.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

258 MCCLINTOCK ET AL.

Constantino, M. J., McClintock, A. S., McCarrick, S. M., Anderson, T., & Himawan, L. (2016). Outcome Expectations Questionnaire. Manuscript in preparation.

Cuijpers, P., Driessen, E., Hollon, S. D., van Oppen, P., Barth, J., & Andersson, G. (2012). The efficacy of non-directive supportive therapy for adult depression: A meta-analysis. Clinical Psychology Review, 32, 280 –291.

De Jong, K., Timman, R., Hakkaart-Van Roijen, L., Vermeulen, P., Kooiman, K., Passchier, J., & Van Busschbach, J. (2014). The effect of outcome monitoring feedback to clinicians and patients in short and long-term psychotherapy: A randomized controlled trial. Psychotherapy Research, 24, 629 – 639. http://dx.doi.org/10.1080/10503307.2013 .871079

De Jong, K., van Sluis, P., Nugter, M. A., Heiser, W. J., & Spinhoven, P. (2012). Understanding the differential impact of outcome monitoring: Therapist variables that moderate feedback effects in a randomized clinical trial. Psychotherapy Research, 22, 464 – 474. http://dx.doi.org/ 10.1080/10503307.2012.673023

Dowell, N. M., & Berman, J. S. (2013). Therapist nonverbal behavior and perceptions of empathy, alliance, and treatment credibility. Journal of Psychotherapy Integration, 23, 158 –165. http://dx.doi.org/10.1037/ a0031421

Duncan, B. L. (2012). The partners for change outcome management system (PCOMS): The heart and soul of change project. Canadian Psychology, 53, 93–104. http://dx.doi.org/10.1037/a0027762

Elliott, R., Bohart, A. G., Watson, J. C., & Greenberg, L. S. (2011). Empathy. In J. Norcross (Ed.), Psychotherapy relationships that work: Evidence-based responsiveness (2nd ed., pp. 89 –108). New York, NY: Oxford University Press. http://dx.doi.org/10.1093/acprof:oso/ 9780199737208.003.0006

Falkenström, F., Markowitz, J. C., Jonker, H., Philips, B., & Holmqvist, R. (2013). Can psychotherapists function as their own controls? Meta- analysis of the crossed therapist design in comparative psychotherapy trials. The Journal of Clinical Psychiatry, 74, 482– 491. http://dx.doi .org/10.4088/JCP.12r07848

Flückiger, C., Del Re, A. C., Wampold, B. E., Znoj, H., Caspar, F., & Jörg, U. (2012). Valuing clients’ perspective and the effects on the therapeutic alliance: A randomized controlled study of an adjunctive instruction. Journal of Counseling Psychology, 59, 18 –26. http://dx.doi.org/10 .1037/a0023648

Greenberg, L. S., Watson, J. C., Elliott, R., & Bohart, A. C. (2001). Empathy. Psychotherapy: Theory, Research, Practice, Training, 38, 380 –384. http://dx.doi.org/10.1037/0033-3204.38.4.380

Haggerty, G., Blake, M., Naraine, M., Siefert, C., & Blais, M. A. (2010). Construct validity of the Schwartz Outcome Scale-10: Comparisons to interpersonal distress, adult attachment, alexithymia, the five-factor model, romantic relationship length and ratings of childhood memories. Clinical Psychology & Psychotherapy, 17, 44 –50.

Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., & Sutton, S. W. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology, 61, 155–163. http://dx.doi.org/10.1002/jclp.20108

Hatcher, R. L., & Gillaspy, J. A. (2006). Development and validation of a revised short version of the Working Alliance Inventory. Psychotherapy Research, 16, 12–25. http://dx.doi.org/10.1080/10503300500352500

Heppner, P. P., Wampold, B. E., Owen, J., Thompson, M. N., & Wang, K. T. (2016). Research design in counseling. Boston, MA: Cengage Learning.

Hill, C. E., & O’Brien, K. M. (1999). Helping skills: Facilitating explo- ration, insight, and action. Washington, DC: American Psychological Association.

Horvath, A. O., Del Re, A. C., Flückiger, C., & Symonds, D. (2011). Alliance in individual psychotherapy. In J. Norcross (Ed.), Psychother- apy relationships that work: Evidence-based responsiveness (2nd ed.,

pp. 25– 69). New York, NY: Oxford University Press. http://dx.doi.org/ 10.1093/acprof:oso/9780199737208.003.0002

Lambert, M. J. (2007). Presidential address: What we have learned from a decade of research aimed at improving psychotherapy outcome in rou- tine care. Psychotherapy Research, 17, 1–14. http://dx.doi.org/10.1080/ 10503300601032506

Lambert, M. J. (2013). The efficacy and effectiveness of psychotherapy. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (pp. 169 –218). Oxford, England: Wiley.

Lambert, M. J., Whipple, J. L., Harmon, C., Shimokawa, K., Slade, K., & Christofferson, C. (2004). Clinical support tools manual. Provo, UT: Department of Psychology, Brigham Young University.

Lutz, W., Lambert, M. J., Harmon, S. C., Tschitsaz, A., Schürch, E., & Stulz, N. (2006). The probability of treatment success, failure and duration: What can be learned from empirical data to support decision making in clinical practice? Clinical Psychology & Psychotherapy, 13, 223–232. http://dx.doi.org/10.1002/cpp.496

MacFarlane, P., Anderson, T., & McClintock, A. S. (2015). Empathy from the client’s perspective: A grounded theory analysis. Psychotherapy Research. Advance online publication. http://dx.doi.org/10.1080/ 10503307.2015.1090038

McClintock, A. S., Anderson, T., & Cranston, S. (2015). Mindfulness therapy for maladaptive interpersonal dependency: A preliminary ran- domized controlled trial. Behavior Therapy, 46, 856 – 868. http://dx.doi .org/10.1016/j.beth.2015.08.002

McClintock, A. S., Anderson, T., & Petrarca, A. (2015). Treatment expec- tations, alliance, session positivity, and outcome: An investigation of a three-path mediation model. Journal of Clinical Psychology, 71, 41– 49. http://dx.doi.org/10.1002/jclp.22119

Microsoft. (2013). Microsoft Excel. Redmond, WA: The Microsoft Cor- poration.

Miller, S. D., Duncan, B. L., Sorrell, R., & Brown, G. S. (2005). The partners for change outcome management system. Journal of Clinical Psychology, 61, 199 –208. http://dx.doi.org/10.1002/jclp.20111

Newman, M. G., & Fisher, A. J. (2010). Expectancy/credibility change as a mediator of cognitive behavioral therapy for generalized anxiety disorder: Mechanism of action or proxy for symptom change? Interna- tional Journal of Cognitive Therapy, 3, 245–261. http://dx.doi.org/10 .1521/ijct.2010.3.3.245

Reese, R. J., Gillaspy, J. A., Jr., Owen, J. J., Flora, K. L., Cunningham, L. C., Archie, D., & Marsden, T. (2013). The influence of demand characteristics and social desirability on clients’ ratings of the therapeu- tic alliance. Journal of Clinical Psychology, 69, 696 –709. http://dx.doi .org/10.1002/jclp.21946

Roemer, L., Orsillo, S. M., & Salters-Pedneault, K. (2008). Efficacy of an acceptance-based behavior therapy for generalized anxiety disorder: Evaluation in a randomized controlled trial. Journal of Consulting and Clinical Psychology, 76, 1083–1089. http://dx.doi.org/10.1037/ a0012720

Ryan, R. M., & Deci, E. L. (2008). A self-determination theory approach to psychotherapy: The motivational basis for effective change. Canadian Psychology, 49, 186 –193. http://dx.doi.org/10.1037/a0012753

Safran, J. D., & Muran, J. C. (2000). Negotiating the therapeutic alliance: A relational treatment guide. New York, NY: Guilford Press.

Safran, J. D., & Muran, J. C. (2006). Resolving therapeutic impasses: A training DVD. Santa Cruz, CA: Custom-flix.com.

Shimokawa, K., Lambert, M. J., & Smart, D. W. (2010). Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology, 78, 298 –311. http://dx .doi.org/10.1037/a0019247

Stiles, W. B., Honos-Webb, L., & Surko, M. (1998). Responsiveness in psychotherapy. Clinical Psychology: Science and Practice, 5, 439 – 458. http://dx.doi.org/10.1111/j.1468-2850.1998.tb00166.x

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

259COMMON FACTORS FEEDBACK

Swift, J. K., & Derthick, A. O. (2013). Increasing hope by addressing clients’ outcome expectations. Psychotherapy, 50, 284 –287. http://dx .doi.org/10.1037/a0031941

Wampold, B. E., & Imel, Z. E. (2015). The great psychotherapy debate: The evidence for what makes psychotherapy work (2nd ed.). New York, NY: Routledge.

Young, J. L., Waehler, C. A., Laux, J. M., McDaniel, P. S., & Hilsenroth, M. J. (2003). Four studies extending the utility of the Schwartz Outcome Scale (SOS-10). Journal of Personality Assessment, 80, 130 –138. http:// dx.doi.org/10.1207/S15327752JPA8002_02

Zimmerman, B. J., & Kitsantas, A. (1997). Development phases in self- regulation: Shifting from process goals to outcome goals. Journal of

Educational Psychology, 89, 29 –36. http://dx.doi.org/10.1037/0022- 0663.89.1.29

Zuroff, D. C., Koestner, R., Moskowitz, D. S., McBride, C., Marshall, M., & Bagby, M. R. (2007). Autonomous motivation for therapy: A new common factor in brief treatments for depression. Psychotherapy Re- search, 17, 137–147. http://dx.doi.org/10.1080/10503300600919380

Received August 20, 2016 Revision received November 7, 2016

Accepted November 7, 2016 �

Members of Underrepresented Groups: Reviewers for Journal Manuscripts Wanted

If you are interested in reviewing manuscripts for APA journals, the APA Publications and Communications Board would like to invite your participation. Manuscript reviewers are vital to the publications process. As a reviewer, you would gain valuable experience in publishing. The P&C Board is particularly interested in encouraging members of underrepresented groups to participate more in this process.

If you are interested in reviewing manuscripts, please write APA Journals at [email protected]. Please note the following important points:

• To be selected as a reviewer, you must have published articles in peer-reviewed journals. The experience of publishing provides a reviewer with the basis for preparing a thorough, objective review.

• To be selected, it is critical to be a regular reader of the five to six empirical journals that are most central to the area or journal for which you would like to review. Current knowledge of recently published research provides a reviewer with the knowledge base to evaluate a new submission within the context of existing research.

• To select the appropriate reviewers for each manuscript, the editor needs detailed information. Please include with your letter your vita. In the letter, please identify which APA journal(s) you are interested in, and describe your area of expertise. Be as specific as possible. For example, “social psychology” is not sufficient—you would need to specify “social cognition” or “attitude change” as well.

• Reviewing a manuscript takes time (1– 4 hours per manuscript reviewed). If you are selected to review a manuscript, be prepared to invest the necessary time to evaluate the manuscript thoroughly.

APA now has an online video course that provides guidance in reviewing manuscripts. To learn more about the course and to access the video, visit http://www.apa.org/pubs/authors/review- manuscript-ce-video.aspx.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

260 MCCLINTOCK ET AL.