Psychotherapists’ Outcomes with White and Racial/Ethnic Minority Clients: First, the Good News Journal of Counseling Psychology

Psychotherapists’ Outcomes With White and Racial/Ethnic Minority Clients: First, the Good News

Jeffrey A. Hayes, Andrew A. McAleavey, Louis G. Castonguay, and Benjamin D. Locke Pennsylvania State University

The purposes of this study were to (a) investigate whether psychotherapists differ in their effectiveness with clients, (b) determine whether disparities exist within therapists’ caseloads in their outcomes with White and racial and ethnic minority (REM) clients, (c) explore therapist factors that might contribute to observed therapist effects, and (d) identify whether treatment outcomes varied for REM and White clients. A sample of 3,825 clients seen by 251 therapists at 45 college counseling centers completed the Counseling Center Assessment of Psychological Symptoms at the beginning and end of individual psychotherapy. Therapists differed in their effectiveness at reducing general distress across clients, and evidence was found for disparities within therapists’ caseloads in their effectiveness with REM and White clients. Effect sizes were small. Disparities within therapists’ caseloads were not a function of any therapist variable that was studied. Therapy outcomes were similar for White and REM clients. Therapist multicultural competence can, and should, be considered in terms of measurable outcomes across client racial/ethnic groups. It is possible to identify multiculturally expert therapists who evidence competence with both REM and White clients and who might serve as models from whom the field could learn.

Keywords: therapist effects, psychotherapy outcomes, multicultural competence, disparities

In the United States, racial and ethnic minority (REM)1 indi- viduals experience mental health problems at rates that are similar to or greater than nonminorities, although REM persons utilize mental services less often than do European Americans (Santiago & Miranda, 2014; Smith, Rodriguez, & Bernal, 2011). REM individuals may be hesitant to seek psychotherapy because of cultural mistrust, racial and ethnic norms related to self-reliance and privacy, and doubts about the availability of culturally com- petent services (Barksdale & Molock, 2009; Duncan, 2003; Hayes, Youn, et al., 2011; Whaley, 2001).

Research to date does not offer a clear picture as to whether REM persons’ doubts about receiving culturally competent psy- chotherapy are well-founded. When REM clients receive services from European American therapists, some studies suggests that they may have difficulty forming an alliance with their therapists (Flückiger et al., 2013) and are at increased risk for premature termination, although findings regarding premature termination are mixed (Kearney, Draper, & Baron, 2005; Maramba & Hall, 2002; Owen, Imel, Adelson, & Rodolfa, 2012; Shin et al., 2005; Terrell & Terrell, 1984; Wade & Bernstein, 1991). A recent meta-analysis of randomized clinical trials (Ünlü Ince, Riper, van ’t Hof, &

Cuijpers, 2014) found that the proportion of REM clients in a study did not moderate effect sizes in the treatment of depression, although a trend in the data (p � .10) was reported such that treatment effect size was inversely related to the proportion of REM clients. It is important to note, however, that this meta- analysis did not directly examine the effectiveness of psychother- apy for REM and White clients within individual studies. Lambert et al. (2006) did conduct such a study and found no differences in outcome as a function of client ethnicity among students receiving services at a university counseling center. Research suggests that providing culturally adapted treatment, such as conducting therapy in clients’ native language and using metaphors that fit clients’ worldviews, may improve outcomes with REM clients (Benish, Quintana, & Wampold, 2011; Griner & Smith, 2006). Based on their synthesis of the literature regarding culturally competent practices, Smith et al. (2011) recommended that therapists use interventions that are consistent with clients’ values; express em- pathy and high regard for REM clients in building a working alliance; demonstrate flexibility and openness with REM clients; communicate in culturally sensitive ways—preferably in the cli- ent’s native language; and exercise caution not to equate cultural differences with deficits.

Therapist Effects in the Provision of Culturally Competent Psychotherapy

Although it has been established that some psychotherapists produce better outcomes than others (Baldwin & Imel, 2013; Crits-Christoph & Mintz, 1991; Kim, Wampold, & Bolt, 2006;

1 The term REM refers to individuals in the United States who identify their race or ethnicity as other than White, European American, or Cau- casian.

Jeffrey A. Hayes, Department of Educational Psychology, Counseling, and Special Education, Pennsylvania State University; Andrew A. McAle- avey and Louis G. Castonguay, Department of Psychology, Pennsylvania State University; Benjamin D. Locke, Center for Counseling and Psycho- logical Services, Pennsylvania State University.

Funding for this study was provided by a grant from the American Psychological Foundation.

Correspondence concerning this article should be addressed to Jeffrey A. Hayes, 307 Cedar Building, Penn State University, University Park, PA 16802. E-mail: [email protected]

Journal of Counseling Psychology © 2016 American Psychological Association 2016, Vol. 63, No. 3, 261–268 0022-0167/16/$12.00 http://dx.doi.org/10.1037/cou0000098

261

Kraus, Castonguay, Boswell, Nordberg, & Hayes, 2011; Laska, Smith, Wislocki, Minami, & Wampold, 2013; Okiishi, Lambert, Nielsen, & Ogles, 2003), research on differential therapist effec- tiveness with REM clients is in its infancy. To date, research on therapist effectiveness with REM clients has been plagued by several consistent problems. Early studies tended to rely on client, therapist, and observer perceptions of therapists’ cultural compe- tence rather than directly testing the assumption that more cultur- ally competent therapists would produce better outcomes with REM clients (Wampold & Brown, 2005). Furthermore, therapists’ self-perceived cultural competence has been found to be unrelated to client and observer ratings of therapists’ cultural competence (Fuertes et al., 2006; Worthington, Mobley, Franks, & Tan, 2000), as well as to client improvement in therapy (Larrison, Schoppel- rey, Hack-Ritzo, & Korr, 2011). In a similar vein, across thera- pists, client ratings of therapist cultural competence have been found to be unrelated to clients’ perceived gains from therapy; within therapists, however, client ratings of therapist cultural com- petence have been found to predict clients’ perceived gains from therapy (Owen, Leach, Wampold, & Rodolfa, 2011).

More recently, research has begun to emerge that examines therapist cultural competence directly as a function of REM client outcomes rather than relying on therapist, client, or observer estimates of therapist cultural competence. Again, the assumption underlying these studies is that therapists who are more culturally competent will produce better outcomes with REM clients than will less culturally competent therapists. Unfortunately, this emerging body of research tends to be characterized by small samples derived from a single site. For example, the Owen et al. (2011) study included 31 therapists from one university counseling center. In another study on therapist effectiveness with REM clients (Imel et al., 2011), there were only 13 therapists, all of whom worked at the same agency. Imel et al. (2011) found that some of the 13 therapists were more effective with REM clients and others were more effective with White clients, where effec- tiveness was defined in terms of clients’ reduced cannabis use. Similar results were obtained in a study of 36 therapists and 228 clients at a single university training clinic (Hayes, Owen, & Bieschke, 2014). Outcome in this study was defined in terms of reduction in client distress as measured by the OQ-45. Some of the therapists in this study saw as few as two REM clients, however, and all were graduate students, which further limited the general- izability of the findings. Larrison et al. (2011) studied a larger sample of therapists (n � 62) and clients (n � 551; 25% of whom were Black) from 13 community mental health centers. They found evidence of differential therapist effectiveness with Black and White clients. Based on a measure of psychological symptoms and functioning, 12 therapists had better outcomes for White clients than Black clients, 24 therapists had better outcomes for Black clients than White clients, and 26 therapists had comparable outcomes for Black and White clients. Unfortunately, some ther- apists saw as few as one Black client, and the median number of Black clients per therapist was only two, undermining the robust- ness of the study’s findings.

Investigating therapists’ cultural competence requires a large data set with sufficient numbers of both therapists and REM clients, optimally from multiple locations to enhance external validity. As Baldwin and Imel (2013, p. 280) have indicated, “When the number of patients per therapist is small, estimates of

therapist mean outcomes will be less reliable . . . which will make comparisons among therapists suspect.” With regard to studying REM clients in particular, the National Institute of Mental Health (2006) has stated that it is important to “identify whether and to what extent . . . outcomes may hold for members of particular racial/ethnic, racial, geographic, or age groups. Research must include large enough representations of these populations in order to determine treatment effectiveness with greater accuracy” (p. 7). The Center for Collegiate Mental Health (CCMH) has generated data that may meet these demands. CCMH is a national research- practice network involving more than 290 college and university counseling centers that administer standardized assessment instru- ments and pool their data each academic year (CCMH, 2012; Hayes, Locke, & Castonguay, 2011). The resulting data set is large enough to permit the complex statistical analyses needed to detect therapist effects in outcome studies when data are nested (e.g., multiple administrations of an instrument within clients who are nested within therapists). In addition, the size of the CCMH data set makes it possible to study samples large enough to yield robust findings, and the multisite nature of CCMH enhances the external validity of its data.

Capitalizing on the strengths of data gathered by CCMH, then, this study had several aims. To begin, we sought to determine whether therapist effects exist in the provision of individual psy- chotherapy to clients at college counseling centers. The primary outcome variable that we were interested in was general psycho- logical distress reported by clients. Consistent with previous re- search, we expected that therapist effects would be identified in terms of general competence (i.e., differences among therapists in outcomes across all clients) and that the magnitude of these effects would be approximately 7%, as has been found in prior naturalistic studies (Baldwin & Imel, 2013). In addition to identifying therapist effects in general competence, we expected to find therapist effects in cultural competence. That is, we predicted that the difference in outcomes between White and REM clients would vary across therapists, as has been found in prior studies (Hayes et al., 2014; Imel et al., 2011; Larrison et al., 2011). Furthermore, we were interested in identifying particular therapist variables that might account for differential therapist effectiveness with REM and White clients. Along these lines, CCMH routinely gathers data on eight therapist factors: gender, race/ethnicity, age, highest degree, professional discipline, years of experience, staff position, and theoretical orientation. There is little extant research to suggest which, if any, of these factors might contribute to successful outcomes with REM clients. In fact, there is a long history of research indicating that many of these therapist factors, which are all rather distal to the actual therapy process, do not meaningfully affect outcome (Beutler, Machado, & Neufeldt, 1994; Tracey, Wampold, Lichtenberg, & Goodyear, 2014; Wampold & Brown, 2005). That being said, most research on these therapist factors has been conducted with White clients and therapists, and it is not clear whether findings from these studies would generalize to a more racially and ethnically diverse sample. Therefore, we took an exploratory approach in examining the extent to which the eight therapist factors might contribute to explaining therapist variability in psychotherapy outcomes associated with clients’ race and eth- nicity.

262 HAYES, MCALEAVEY, CASTONGUAY, AND LOCKE

Method

Participants

The data for this project were gathered by CCMH during the 2010 –2011 and 2011–2012 academic years. This data set included 161,335 clients of 3,599 therapists from 122 counseling centers. Prior to analysis, however, this data set was reduced to meet several inclusion criteria: (a) To prevent analyses of multiple courses of therapy for the same client during the 2-year data collection period, only the first course of therapy was considered. Consistent with Minami et al. (2009), a client was presumed to have started a second course of therapy if 90 days or more had elapsed between sessions; all subsequent data was considered part of an ensuing course of therapy and was not included in analysis. (b) The client must have attended at least one session of individual, face-to-face psychotherapy, counseling, or a clinical intake. This excluded clients who attended counseling centers strictly for as- sessment or psychiatric or case management services, for instance. Clients were further required to have had more than half of their attended appointments be individual face-to-face appointments (including clinical intakes), to ensure that the included clients were primarily attending counseling centers for counseling. (c) To main- tain a focus in this study on individual treatment, clients were excluded if they attended any group therapy appointments during their first course of therapy. (d) Clients must have completed the Counseling Center Assessment of Psychological Symptoms (either the CCAPS-34 or CCAPS-62; see the Measures section) at least twice during their first course of therapy. The first of these CCAPS must have occurred within 14 days of the first attended session, and the final CCAPS must have been completed within 14 days of the final attended treatment appointment. (e) Finally, because this study focused on therapist effects, it was required that a single therapist must have seen the client more than half of the times that the client attended the counseling center for any purpose during their first course of therapy. A further exclusion criterion was established for therapists: Only data from therapists who had seen at least three White clients and at least three REM clients were retained. This resulted in a final data set of 3,825 clients seen by 251 therapists from 45 counseling centers.

Clients. The mean age of the clients was 22.8 years. A total of 17.9% (n � 662) of the clients classified themselves as “fresh- men,” 19.2% (n � 712) as “sophomore,” 21.8% (n � 810) as “junior,” 26.3% (n � 974) as “senior,” 13.6% (n � 506) as “graduate student,” and 1.2% (n � 45) as “other.” Almost two thirds of the clients (65.5%; n � 2,443) were women, 34.1% (n � 1,271) were men, and 0.2% (n � 8) identified as transgender. A total of 8.5% identified as African American (n � 318), 6.8% as Asian American (n � 252), 8.8% as Hispanic/Latino/a (n � 327), 66.9% as Caucasian/White (n � 2489), 4.6% as multiracial (n � 171), 0.7% as American Indian/Alaskan native (n � 26), and 0.2% as Native Hawaiian or Pacific Islander (n � 8). A total of 131 clients (3.5%) identified their race or ethnicity as “other.”

Therapists. Descriptive data were provided by 103 of the 251 therapists in the sample. Of these 103 therapists, 66 were women, 35 were men, and one was transgender. Therapists’ mean age was 41.1 years (SD � 10.9, range � 23 to 63). Most therapists (64%) identified their race or ethnicity as White/Caucasian. Approxi- mately three quarters of the therapists were professional staff

members and the rest were therapist trainees (nine psychology interns, seven postdoctoral trainees, and nine other trainees). Most of the therapists (n � 62) were doctoral-level psychologists and 29 were master’s-level clinicians. Therapists were primarily human- istic or cognitive– behavioral in their theoretical orientations.

Dyads. The total number of clients per therapist in this data set ranged from six to 72, although most therapists had between six and 17 clients (M � 15.3, SD � 10.0, Mdn � 12). The number of REM clients per therapist ranged from 3 to 27, though most therapists had between 3 and 9 REM clients (M � 5.3, SD � 3.3, Mdn � 4).

Measures

CCAPS-62 (Locke et al., 2011). The CCAPS-62 is a 62-item measure designed to assess a range of psychological symptoms applicable to the college population. It has eight subscales: De- pression, Generalized Anxiety, Social Anxiety, Eating Concerns, Substance Use, Hostility, Academic Distress, and Family Con- cerns. The subscales have demonstrated acceptable internal con- sistency and retest reliability estimates, as well as initial evidence of convergent validity (Locke et al., 2011; McAleavey et al., 2012). In addition, 20 of the items constitute the Distress Index (DI), a measure of general distress (CCMH, 2012) that has items from Depression, Generalized Anxiety, Social Anxiety, Hostility, and Academic Distress, and strongly correlates with the Outcome Questionnaire-45.2 (Lambert et al., 1996). The DI was the primary outcome measure in this study. In this sample, Cronbach’s alpha for the DI was 0.93.

Based on pilot study data, the CCAPS appears to be reliable for REM clients; internal consistency estimates for African American, Asian American, and Latino clients were at least .80 for each subscale. In addition, evidence from a separate validation study suggests that the factor structure of the CCAPS is stable across ethnic groups, and the CCAPS evidences construct validity for REM clients. To be specific, large correlations were found be- tween CCAPS subscales and theoretically similar instruments (e.g., Beck Depression Inventory, Eating Attitudes Test) in sepa- rate analyses for African American, Asian American, Latino, and European American clients (Hayes, Edens, & Locke, 2010). In addition, data were gathered in 2010 from more than 21,000 students in the general campus populations of 43 colleges to establish nonclinical norms for students of various ethnicities. These norms have been used to generate reliable change indices for use in college counseling centers. Finally, as mentioned earlier, a 34-item version of the CCAPS was developed to facilitate assessment of treatment progress and outcome by easing the time constraints associated with multiple administrations of an instru- ment. In addition to demonstrating solid psychometrics qualities (Locke et al., 2012), this 34-item version contains all 20 items of the DI, which is therefore measurable through either the short or long forms.

Standardized Data Set (SDS). The SDS gathers cultural and demographic information in separate versions for both clients and therapists. For both the client and therapist versions of the SDS, participants identify their gender, age, and their race/ethnicity; categories for race/ethnicity are “African American/Black,” “American Indian or Alaskan Native,” “Asian American/Asian,” “Hispanic/Latino/a,” “Native Hawaiian or Pacific Islander,” “Mul-

263THERAPIST EFFECTS WITH REM CLIENTS

tiracial,” “White” or “Other.” These categories are consistent with federal guidelines for measuring individuals’ race and ethnicity (http://www.whitehouse.gov/fedreg_1997standards). The therapist version of the SDS also measures highest professional degree, years of experience, professional discipline (e.g., counseling psy- chology, clinical psychology), staff position (e.g., intern, staff psychologist), and theoretical orientation. Theoretical orientation is measured with five items from the Development of Psychother- apists Common Core Questionnaire (Orlinsky et al., 1991), which asks therapists to rate, on a 1 (low) to 5 (high) scale, how much their current practice is guided by each of five approaches to therapy: cognitive, behavioral, humanistic, psychodynamic, and systems. These items allow therapists to endorse several orienta- tions as influences.

Procedure

Each counseling center collected standardized data on clients and therapists. Therapists’ SDS data were collected annually through a CCMH liaison at each counseling center, deidentified, and sent to CCMH upon therapists’ consent. Clients’ SDS data were collected prior to the first counseling session. CCAPS data were collected from clients according to procedures established at each counseling center. Most centers administered the CCAPS-62 immediately prior to the client’s initial appointment and adminis- tered the CCAPS-34 subsequently, either on a regular time interval (e.g., every 2 weeks) or a regular session interval (e.g., every session) as a routine part of clinical practice. Only the first and last CCAPS administrations were used in the present analyses. Unique codes were assigned to clients and therapists so that they could be paired for each client appointment within clients’ electronic med- ical records. Clients who consented to contributing data to CCMH had their data deidentified and uploaded monthly to CCMH, where data were cleaned, stored, and analyzed.

Results

All analyses were conducted using R (R Core Team, 2013) and the lme4 package (version 1.0 –5; Bates, Maechler, Bolker, & Walker, 2013). The general strategy was mixed-effects linear modeling, which is sometimes also called hierarchical linear mod-

eling, in order to account for nonindependence of multiple clients seen by the same therapist. First, a random intercepts-only model (that is, a model with random intercepts for therapists as the sole predictor beyond a residual) was conducted in order to determine whether therapists differ meaningfully from one another on their clients’ final DI scores. This preliminary model indicated that a small amount of variance between therapists existed. The intra- class correlation estimate for therapists was 0.039, meaning that 3.9% of the variability in outcome on clients’ final DI scores was attributable to differences between therapists. A similar effect was found when controlling for pretreatment DI score, which may better represent the effect of therapists (Wampold & Brown, 2005); the ICC in that case was 0.032. This finding supported Hypothesis 1, that a therapist effect would be detected, although the magnitude of the effect was smaller than expected.

We then proceeded to Model 1. In Model 1, clients’ pretreat- ment DI score and REM status were included as fixed effects, and the therapist random effect was retained. The complete mixed effect model was

Yij � �00 � �10 � (DIpre)ij � �20 � (R ⁄ EMij) � b0j � eij,

where Yij is the estimated final DI score, �00 is the overall mean, �10 is the fixed effect of pretreatment DI score (DIpre), �20 is the fixed effect of client REM status relative to White status (i.e., the difference between White and REM clients’ final DI, represented by a dummy-coded variable with White � 0 and REM � 1), b0j is the therapist level random intercept, and eij is the residual variance. Results are shown in Table 1. For the fixed effects, pretreatment DI significantly predicted final DI, whereas REM status did not, indicating that White and REM clients did not differ in treatment outcome, on average.

We then proceeded to Model 2, which included all effects from Model 1 while including a random effect for the REM slope:

Yij � �00 � �10 � (DIpre)ij � �20 � (R ⁄ EMij) � b0j � b1j � (R ⁄ EMij)eij.

The additional b1j parameter represents the difference between therapist effectiveness with White and REM clients. A significance test of this variable therefore examines whether therapist compe- tence is general (applying to all clients) or differs based on race/ ethnicity of clients (a therapist has better outcomes with one group

Table 1 Parameters of Model 1 and Model 2

Model 1 Model 2

Estimate 95% CI Estimate 95% CI

Fixed effects Intercept (�00) 0.192 [0.140, 0.246]

a 0.190 [0.143, 0.238]a

Pretreatment DI (�10) 0.609 [0.586, 0.636] a 0.609 [0.584, 0.634]a

REM status (�20) �0.002 [�0.042, 0.041] �0.001 [�0.046, 0.044] Random effects

Intercept (b0j) 0.1067 [0.073, 0.133] a 0.082 [0.042, 0.119]a

REM status (b1j) — — 0.100 [0.036, 0.178] a

Residual (eij) 0.5886 [0.576, 0.603] a 0.586 [0.572, 0.600]a

Note. N � 3,825 clients, 251 therapists. Confidence intervals (CIs) computed using parametric bootstrapping. Random effects and CIs are reported as standard deviations. The overall likelihood ratio test indicated that Model 2 is superior to Model 1, p � .023. a 95% CI does not include zero.

264 HAYES, MCALEAVEY, CASTONGUAY, AND LOCKE

than the other). The results of Model 2 are also presented in Table 1. A likelihood ratio test compared the overall model fits of Model 1 and Model 2. The test was significant, �2(2) � 6.50, p � .039, indicating that Model 2 is a significant improvement over Model 1. That is, in support of Hypothesis 2, therapist effectiveness is not best modeled as a single distribution, but is better modeled by two separate effects: therapist effectiveness with White clients and therapist effectiveness with REM clients. Although some thera- pists did not evidence disparate outcomes for their White and REM clients, others demonstrated significantly discrepant effects.

We then extracted the estimated values for b0j and b1j in Model 2 for secondary analysis. In Model 2, the estimates of b0j can be interpreted as the impact of a given therapist on final DI scores for White clients, and b1j represents the difference between therapists’ effectiveness with White and REM clients. Because b1j is actually a difference value, the sum of b0j and b1j represents the total estimated impact of therapist on posttreatment DI scores for REM clients, controlling for pretreatment DI score. This sum, rather than b1j itself, is the estimated therapist effectiveness for REM clients. An additional measure of effectiveness thought to be important was the marginal effectiveness for each therapist across White and REM clients. This quantity is the average of b0j and b0j � b1j. This marginal effectiveness represents therapists’ combined effective- ness with White and REM clients, and might be a good indicator of multicultural competence. To estimate the approximate size of differences between therapists, we ordered therapists by their marginal effectiveness and chose to examine the 15 most effective and 15 least effective therapists, and included the middle 15 therapists as well. Figure 1 displays the pre–post DI change scores for their White and REM clients on Cohen’s d scale.

A series of analyses were conducted using the available thera- pist characteristics to predict b1j, therapist differential effective- ness with REM versus White clients. No therapist demographics variables were significant predictors of b1j, including gender, F(2, 99) � 0.99, p � .37; race/ethnicity, F(7, 94) � 1.02, p � .42 (also alternatively simplified to White and REM therapists, F[1, 100] �

0.51, p � .48); highest completed degree, F(10, 91) � 1.00, p � .45; discipline of highest completed degree, F(5, 96) � 1.15, p � .34; type of staff position in the counseling center, F(5, 96) � 0.12, p � .99; and number of years licensed, F(1, 64) � .02, p � .903. Items measuring therapists’ theoretical orientation were not sig- nificant predictors when entered as individual predictors, nor as simultaneous predictors in a multiple regression: cognitive, F(1, 86) � 1.32, p � .25, R2 � .02; behavioral, F(1, 83) � 0.01, p � .94, R2 � .00; humanistic, F(1, 86) � 0.60, p � .44, R2 � .01; psychoanalytic/dynamic, F(1, 86) � 0.17, p � .69, R2 � .00; systems, F(1, 84) � 0.211, p � .65, R2 � .00; all orientations combined, F(5, 78) � 0.36, p � .87, R2 � .02. Thus, none of these therapist characteristics significantly predicted estimates of differ- ential effectiveness with REM clients versus White clients.

Discussion

The findings from this study add to the growing body of em- pirical literature on therapist effects with REM clients. Similar to Larrison et al. (2011), who found that therapists differed in their effectiveness at reducing psychological symptoms with REM cli- ents, as well as other previous research (Hayes et al., 2014; Imel et al., 2011; Owen et al., 2012), we found differential therapist effectiveness related to clients’ race and ethnicity. In particular, results from this study indicated that therapists varied in their effectiveness at reducing psychological symptoms among clients, and that this variability was partially due to clients’ racial/ethnic heritage.

Although we expected to find differences in therapists’ general effectiveness at reducing client distress, the magnitude of these effects were surprising. Therapists accounted for only 3.9% of the variance in outcome across all clients, which was small for a naturalistic setting in which the size of therapist effects tends to exceed those found in efficacy studies (Baldwin & Imel, 2013). What might account for this relatively small therapist effect? One possibility is the composition of the sample. Previous research has suggested that therapist effects are accentuated when clients have more severe concerns at intake (Saxon & Barkham, 2012). That is, therapists produce more similar outcomes with clients who have moderate concerns, but differences in therapist competencies be- come magnified when they work with more difficult clients. This makes sense clinically. When clients are especially challenging, it matters more who their therapist is because the consequences are magnified. Whereas most therapists produce good outcomes (e.g., Kraus et al., 2011), the effects of better therapists are highlighted when clients are more difficult to work with, as are the effects of relatively worse therapists. Given the fact that the sample in this study consisted of college students, most of whom are functioning at a relatively high level, many may not be clinically complex and challenging enough to produce pronounced therapist effects.

At the same time, it could be that REM clients present more difficulties for some therapists, especially those whose compe- tency working with REM clients is not well developed (Comas- Díaz, 2014). Therapist effects, then, will be magnified when there is increased variability in therapists’ multicultural competence and clients are diverse along ethnic and racial lines. In this study, roughly one third of the clients were REM individuals. By select- ing a large sample of therapists who worked with at least three REM clients and three White clients, we essentially oversampled

Figure 1. Client pre–post effect sizes with the 15 most effective, 15 least effective, and 15 middle therapists working with REM clients. The dashed line represents the grand mean for change in this sample. REM � racial and ethnic minority.

265THERAPIST EFFECTS WITH REM CLIENTS

for REM clients to generate estimates of therapist effects that we thought would be more stable than in studies with smaller numbers therapists and fewer REM clients per therapist (e.g., Hayes et al., 2014; Imel et al., 2011; Larrison et al., 2011). Interestingly, Lar- rison and Schoppelrey (2011) found a large therapist effect in a naturalistic study with 19 REM and 79 White clients in a commu- nity mental health setting. They reported that 29% of variance in client outcomes could be attributed to therapists, though the mag- nitude of this effect was likely inflated due to the small number of therapists (n � 14) in the study.

On the whole, the data suggest that some therapists evidenced better outcomes with REM clients than White clients, and vice versa. These therapists seem to possess culture-specific expertise with one racial/ethnic group that, for reasons that could not be identified in this study, do not extend to their work with the other group. This finding replicated previous research regarding differ- ential therapist effectiveness with REM and White clients across a variety of settings. For instance, Hayes et al. (2014) found that therapist trainees showed variable effectiveness across REM and White clients in reducing clients’ psychological distress. Similarly, Imel et al. (2011) reported that therapists’ effectiveness at reducing adolescent cannabis use varied according to the clients’ ethnicity, and Larrison et al. (2011) detected therapist effects associated with client race in 13 community mental health centers. Based on this collection of studies, then, there does seem to be unique variability in therapist effectiveness that is associated with the client’s race or ethnicity. These findings would support continuing or increasing the emphasis in many graduate psychology programs on multicul- tural training. Future researchers could examine the comparable effectiveness of different models of multicultural training (e.g., a curriculum infused with a multicultural focus vs. stand-alone courses) on psychotherapy outcomes, and investigate the specific components of training (e.g., experiential, didactic, clinical) that promote cultural competence. It would also be useful for educa- tion, training, and supervisory purposes for future studies to iden- tify which multicultural competencies (e.g., awareness, knowl- edge, skills) are related to client outcome.

Whereas the data were best modeled by accounting for both general therapist competence and specific cultural competence, it is also important to point out that, in this data, there appears to be a close correspondence between the outcomes of clients of the same therapists, regardless of client race or ethnicity. We would suggest that the binary discourse that has characterized the litera- ture (general effectiveness vs. culture-specific effectiveness) to this point be replaced. Instead, as indicated by this study, thera- pists’ general competence and cultural competence are both im- portant considerations. Thus, psychotherapy training, for students as well as professionals, should incorporate principles that promote change across all clients as well as within specific ethnic and racial groups (Comas-Díaz, 2014). As reflected in Figure 1, a number of therapists were highly effective with White clients and were par- ticularly effective with REM clients. These multiculturally expert therapists might serve as valuable educators and role models from whom other therapists could learn.

It is also important to note that, on average, REM clients experienced as much reduction in psychological symptoms as did White clients. This clinically encouraging finding replicates an earlier study of counseling center clients (Lambert et al., 2006). As it is the case in many studies on therapist effects, unfortunately, we

can only speculate as to variables that are associated with client change and that might account for why some therapists were more effective than others. In particular, the CCMH data set did not contain variables that provided insight into underlying therapist factors associated with differential outcomes among REM and White clients. As is so often true of research on therapist effects, we only had access to relatively surface-level variables such as gender, age, and professional discipline that reveal little about the actual mechanisms of change in psychotherapy (Beutler et al., 1994). It is much more difficult to study constructs that would yield more valuable information about working effectively with REM clients, such as cultural humility (Comas-Díaz, 2014), man- aging one’s cognitive and emotional reactions to REM clients (Gelso & Mohr, 2001), or communicating skillfully about a cul- turally relevant topic in a cross-racial dyad (Chang & Berk, 2009; Gaztambide, 2012; Hook, Davis, Owen, Worthington, & Utsey, 2013). In that context, a particularly encouraging finding worthy of replication was reported by Larrison et al. (2011), who discovered that the number of positive experiences and relationships that a therapist has with individuals from ethnic and racial groups dis- similar to her or his own predicted differential outcomes between Black and White clients.

One of the strengths of the present study relative to other studies of therapist effectiveness with REM clients was its large sample. That being said, the minimum number of REM clients per therapist was only three, potentially yielding imprecise estimates of thera- pists’ effectiveness with REM clients. Future research with more REM clients per therapist will be needed not only to replicate findings from this study but also to investigate therapist effects with specific REM groups rather than combining all REM clients into one category. Subsequent studies might also profitably exam- ine therapist effects when taking into consideration clients’ mul- tiple, intersecting cultural characteristics, such as sexual orienta- tion, religion, and ethnicity. It also will be important to determine the extent to which findings from this study generalize to settings other than college and university counseling centers.

Limitations notwithstanding, this study overcame typical prob- lems in previous research on therapist effects with REM clients by gathering data on a relatively large number of therapists and clients from multiple sites. We also found that client distress at the end of treatment was similar for REM and White clients. The study also replicated and extended prior research in identifying therapist effects in a naturalistic setting and determining that variability in therapist effectiveness was associated with clients’ race and eth- nicity. Future researchers would be advised not to investigate demographic therapist predictors of effectiveness with REM cli- ents that are distal to the process of therapy but to examine more proximal psychological therapist variables.

References

Baldwin, S. A., & Imel, Z. E. (2013). Therapist effects. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (6th ed., pp. 258 –297). Hoboken, NJ: Wiley.

Barksdale, C. L., & Molock, S. D. (2009). Perceived norms and mental health help seeking among African American college students. The Journal of Behavioral Health Services & Research, 36, 285–299. http:// dx.doi.org/10.1007/s11414-008-9138-y

266 HAYES, MCALEAVEY, CASTONGUAY, AND LOCKE

Bates, D., Maechler, M., Bolker, B., & Walker, S. (2013). lme4: Linear mixed-effects models using Eigen and S4. R package version 1.0 –5. Retrieved from http://CRAN.R-project.org/package�lme4

Benish, S. G., Quintana, S., & Wampold, B. E. (2011). Culturally adapted psychotherapy and the legitimacy of myth: A direct-comparison meta- analysis. Journal of Counseling Psychology, 58, 279 –289. http://dx.doi .org/10.1037/a0023626

Beutler, L. E., Machado, P. P. P., & Neufeldt, S. A. (1994). Therapist variables. In A. E. Bergin & S. L. Garfield (Eds.), Handbook of psy- chotherapy and behavior change (4th ed., pp. 229 –269). New York, NY: Wiley.

Center for Collegiate Mental Health. (2012). CCAPS 2012 technical man- ual. University Park, PA: Author.

Chang, D. F., & Berk, A. (2009). Making cross-racial therapy work: A phenomenological study of clients’ experiences of cross-racial therapy. Journal of Counseling Psychology, 56, 521–536. http://dx.doi.org/ 10.1037/a0016905

Comas-Díaz, L. (2014). Multicultural psychotherapy. In F. T. L. Leong, L. Comas-Díaz, G. C. Nagayama Hall, V. C. McLoyd, & J. E. Trimble (Eds.), APA handbook of multicultural psychology, Vol. 2: Applications and training (pp. 419 – 441). Washington, DC: American Psychological Association. http://dx.doi.org/10.1037/14187-024

Crits-Christoph, P., & Mintz, J. (1991). Implications of therapist effects for the design and analysis of comparative studies of psychotherapies. Journal of Consulting and Clinical Psychology, 59, 20 –26. http://dx.doi .org/10.1037/0022-006X.59.1.20

Duncan, L. E. (2003). Black male college students’ attitudes toward seeking psychological help. Journal of Black Psychology, 29, 68 – 86. http://dx.doi.org/10.1177/0095798402239229

Flückiger, C., Del Re, A. C., Horvath, A. O., Symonds, D., Ackert, M., & Wampold, B. E. (2013). Substance use disorders and racial/ethnic mi- norities matter: A meta-analytic examination of the relation between alliance and outcome. Journal of Counseling Psychology, 60, 610 – 616. http://dx.doi.org/10.1037/a0033161

Fuertes, J. N., Stracuzzi, T. I., Bennett, J., Scheinholtz, J., Mislowack, A., Hersh, M., & Cheng, D. (2006). Therapist multicultural competency: A study of therapy dyads. Psychotherapy (Chicago, Ill.), 43, 480 – 490. http://dx.doi.org/10.1037/0033-3204.43.4.480

Gaztambide, D. J. (2012). Addressing cultural impasses with rupture resolution strategies: A proposal and recommendations. Professional Psychology: Research and Practice, 43, 183–189. http://dx.doi.org/ 10.1037/a0026911

Gelso, C. J., & Mohr, J. J. (2001). The working alliance and the transfer- ence/countertransference relationship: Their manifestation with racial/ ethnic and sexual orientation minority clients and therapists. Applied & Preventive Psychology, 10, 51– 68. http://dx.doi.org/10.1016/S0962- 1849(05)80032-0

Griner, D., & Smith, T. B. (2006). Culturally adapted mental health intervention: A meta-analytic review. Psychotherapy (Chicago, Ill.), 43, 531–548. http://dx.doi.org/10.1037/0033-3204.43.4.531

Hayes, J. A., Edens, A., & Locke, B. D. (2010). Cultural validity of the CCAPS-62. Paper presented at the annual conference of American Psychological Association, San Diego, CA.

Hayes, J. A., Locke, B. D., & Castonguay, L. G. (2011). The Center for Collegiate Mental Health: Practice and research working together. Jour- nal of College Counseling, 14, 101–104. http://dx.doi.org/10.1002/j .2161-1882.2011.tb00265.x

Hayes, J. A., Owen, J. J., & Bieschke, K. J. (2014). Therapist differences in symptom change with racial/ethnic minority clients. Psychotherapy (Chicago, Ill.). Advance online publication.

Hayes, J. A., Youn, S. J., Castonguay, L. G., Locke, B. D., McAleavey, A. A., & Nordberg, S. S. (2011). Rates and predictors of counseling center utilization among college students of color. Journal of College

Counseling, 14, 105–116. http://dx.doi.org/10.1002/j.2161-1882.2011 .tb00266.x

Hook, J. N., Davis, D. E., Owen, J., Worthington, E. L., & Utsey, S. O. (2013). Cultural humility: Measuring openness to culturally diverse clients. Journal of Counseling Psychology, 60, 353–366. http://dx.doi .org/10.1037/a0032595

Imel, Z. E., Baldwin, S., Atkins, D. C., Owen, J., Baardseth, T., & Wampold, B. E. (2011). Racial/ethnic disparities in therapist effective- ness: A conceptualization and initial study of cultural competence. Journal of Counseling Psychology, 58, 290 –298. http://dx.doi.org/ 10.1037/a0023284

Kearney, L. K., Draper, M., & Barón, A. (2005). Counseling utilization by ethnic minority college students. Cultural Diversity and Ethnic Minority Psychology, 11, 272–285. http://dx.doi.org/10.1037/1099-9809.11.3.272

Kim, D., Wampold, B. E., & Bolt, D. M. (2006). Therapist effects in psychotherapy: A random effects modeling of the National Institute of Mental Health Treatment of Depression Collaborative Research Pro- gram data. Psychotherapy Research, 16, 161–172. http://dx.doi.org/ 10.1080/10503300500264911

Kraus, D. R., Castonguay, L., Boswell, J. F., Nordberg, S. S., & Hayes, J. A. (2011). Therapist effectiveness: Implications for accountability and patient care. Psychotherapy Research, 21, 267–276. http://dx.doi.org/ 10.1080/10503307.2011.563249

Lambert, M. J., Burlingame, G. M., Umphress, V., Hansen, N. B., Ver- meersch, D. A., Clouse, G. C., & Yanchar, S. C. (1996). The reliability and validity of the Outcome Questionnaire. Clinical Psychology & Psychotherapy, 3, 249 –258. http://dx.doi.org/10.1002/(SICI)1099- 0879(199612)3:4�249::AID-CPP106�3.0.CO;2-S

Lambert, M. J., Smart, D. W., Campbell, M. P., Hawkins, E. J., Harmon, C., & Slade, K. L. (2006). Psychotherapy outcome as measured by the OQ-45 in African American, Asian/Pacific Islander, Latino/a, and Na- tive American clients compared with matched Caucasian clients. Journal of College Student Psychotherapy, 20, 17–29. http://dx.doi.org/10.1300/ J035v20n04_03

Larrison, C. R., & Schoppelrey, S. L. (2011). Therapist effects on dispar- ities experienced by minorities receiving services for mental illness. Research on Social Work Practice, 21, 727–736. http://dx.doi.org/ 10.1177/1049731511410989

Larrison, C. R., Schoppelrey, S. L., Hack-Ritzo, S., & Korr, W. S. (2011). Clinician factors related to outcome differences between black and white patients at CMHCs. Psychiatric Services, 62, 525–531. http://dx.doi.org/ 10.1176/ps.62.5.pss6205_0525

Laska, K. M., Smith, T. L., Wislocki, A. P., Minami, T., & Wampold, B. E. (2013). Uniformity of evidence-based treatments in practice? Therapist effects in the delivery of cognitive processing therapy for PTSD. Journal of Counseling Psychology, 60, 31– 41. http://dx.doi.org/10.1037/ a0031294

Locke, B. D., Buzolitz, J. S., Lei, P. W., Boswell, J. F., McAleavey, A. A., Sevig, T. D., . . . Hayes, J. A. (2011). Development of the Counseling Center Assessment of Psychological Symptoms-62 (CCAPS-62). Jour- nal of Counseling Psychology, 58, 97–109. http://dx.doi.org/10.1037/ a0021282

Locke, B. D., McAleavey, A. A., Zhao, Y., Lei, P.-W., Hayes, J. A., Castonguay, L. G., . . . Lin, Y.-C. (2012). Development and initial validation of the Counseling Center Assessment of Psychological Symptoms-34 (CCAPS-34). Measurement and Evaluation in Counsel- ing and Development, 45, 151–169. http://dx.doi.org/10.1177/ 0748175611432642

Maramba, G. G., & Hall, G. C. (2002). Meta-analyses of ethnic match as a predictor of dropout, utilization, and level of functioning. Cultural Diversity and Ethnic Minority Psychology, 8, 290 –297. http://dx.doi .org/10.1037/1099-9809.8.3.290

McAleavey, A. A., Nordberg, S. S., Hayes, J. A., Castonguay, L. G., Locke, B. D., & Lockard, A. J. (2012). Clinical validity of the Coun-

267THERAPIST EFFECTS WITH REM CLIENTS

seling Center Assessment of Psychological Symptoms-62 (CCAPS-62): Further evaluation and clinical applications. Journal of Counseling Psy- chology, 59, 575–590.

Minami, T., Davies, D. R., Tierney, S. C., Bettmann, J. E., McAward, S. M., Averill, L. A., . . . Wampold, B. E. (2009). Preliminary evidence on the effectiveness of psychological treatments delivered at a university counseling center. Journal of Counseling Psychology, 56, 309 –320. http://dx.doi.org/10.1037/a0015398

National Institute of Mental Health. (2006). Roadmap for the future. Rockville, MD: Author.

Okiishi, J., Lambert, M. J., Nielsen, S. L., & Ogles, B. M. (2003). Waiting for supershrink: An empirical analysis of therapist effects. Clinical Psychology & Psychotherapy, 10, 361–373. http://dx.doi.org/10.1002/ cpp.383

Orlinsky, D. E., Gerin, P., Davis, J., Davis, M., Ambuhl, H., Willutzski, U., . . . Iahns, J. F. (1991). The international study of the development of psychotherapists: Conceptual organizational of the common core ques- tionnaires. Unpublished manuscript, University of Chicago, IL.

Owen, J., Imel, Z., Adelson, J., & Rodolfa, E. (2012). “No-show”: Ther- apist racial/ethnic disparities in client unilateral termination. Journal of Counseling Psychology, 59, 314 –320. http://dx.doi.org/10.1037/ a0027091

Owen, J., Leach, M. M., Wampold, B., & Rodolfa, E. (2011). Client and therapist variability in clients’ perceptions of their therapists’ multicul- tural competencies. Journal of Counseling Psychology, 58, 1–9. http:// dx.doi.org/10.1037/a0021496

R Core Team. (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from http://www.R-project.org/

Santiago, C. D., & Miranda, J. (2014). Progress in improving mental health services for racial-ethnic minority groups: A ten-year perspective. Psy- chiatric Services, 65, 180 –185. http://dx.doi.org/10.1176/appi.ps .201200517

Saxon, D., & Barkham, M. (2012). Patterns of therapist variability: Ther- apist effects and the contribution of patient severity and risk. Journal of Consulting and Clinical Psychology, 80, 535–546. http://dx.doi.org/ 10.1037/a0028898

Shin, S., Chow, C., Camacho-Gonsalves, T., Levy, R., Allen, I. E., & Leff, H. S. (2005). A meta-analytic review of racial-ethnic matching for

African American and Caucasian American clients and clinicians. Jour- nal of Counseling Psychology, 52, 45–56. http://dx.doi.org/10.1037/ 0022-0167.52.1.45

Smith, T. B., Rodriguez, M. M. D., & Bernal, G. (2011). Culture. In J. C. Norcross (Ed.), Psychotherapy relationships that work (2nd ed., pp. 316 –335). New York, NY: Oxford University Press. http://dx.doi.org/ 10.1093/acprof:oso/9780199737208.003.0016

Terrell, F., & Terrell, S. (1984). Race of counselor, client sex, cultural mistrust level, and premature termination from counseling among Black students. Journal of Counseling Psychology, 31, 371–375. http://dx.doi .org/10.1037/0022-0167.31.3.371

Tracey, T. J. G., Wampold, B. E., Lichtenberg, J. W., & Goodyear, R. K. (2014). Expertise in psychotherapy: An elusive goal? American Psychol- ogist, 69, 218 –229. http://dx.doi.org/10.1037/a0035099

Ünlü Ince, B., Riper, H., van ’t Hof, E., & Cuijpers, P. (2014). The effects of psychotherapy on depression among racial-ethnic minority groups: A metaregression analysis. Psychiatric Services, 65, 612– 617. http://dx .doi.org/10.1176/appi.ps.201300165

Wade, P., & Bernstein, B. L. (1991). Culture sensitivity training and counselor’s race: Effects on Black female clients’ perceptions and attri- tion. Journal of Counseling Psychology, 38, 9 –15. http://dx.doi.org/ 10.1037/0022-0167.38.1.9

Wampold, B. E., & Brown, G. S. (2005). Estimating variability in out- comes attributable to therapists: A naturalistic study of outcomes in managed care. Journal of Consulting and Clinical Psychology, 73, 914 –923. http://dx.doi.org/10.1037/0022-006X.73.5.914

Whaley, A. L. (2001). Cultural mistrust and mental health services for African Americans: A review and meta-analysis. The Counseling Psy- chologist, 29, 513–531. http://dx.doi.org/10.1177/0011000001294003

Worthington, R. L., Mobley, M., Franks, R. P., & Tan, J. A. (2000). Multicultural counseling competencies: Verbal content, counselor attri- butions, and social desirability. Journal of Counseling Psychology, 47, 460 – 468. http://dx.doi.org/10.1037/0022-0167.47.4.460

Received September 5, 2014 Revision received May 17, 2015

Accepted May 19, 2015 �

268 HAYES, MCALEAVEY, CASTONGUAY, AND LOCKE