Week 3 Discussion

O R I G I N A L P A P E R

Case Management as a Significant Component of Usual Care Psychotherapy for Youth with Disruptive Behavior Problems

Rachel Zoffness Æ Ann Garland Æ Lauren Brookman-Frazee Æ Scott Roesch

Published online: 7 May 2009 � The Author(s) 2009. This article is published with open access at Springerlink.com

Abstract Youth with disruptive behavior problems (DBPs) represent the majority of youth served in usual care (UC) psychotherapy, and are at high risk for maladaptive

outcomes. Little is known about UC psychotherapeutic strategies utilized with this pop-

ulation. Researchers and clinicians suggest that case management (CM) is a major activity

occurring in usual care. CM includes coordinating care with service providers and indi-

viduals, including schools, psychiatrists, and community-based services. This study

assesses the prevalence and predictors of clinician use of CM in usual care. Results from

this study suggest that CM is frequently used in UC psychotherapy with youth with DBPs.

The extent of use of CM in UC may have implications for implementation of evidence-

based practices in usual care psychotherapy.

Keywords Usual care � Child and adolescent psychotherapy � Community-based services � Case management � Disruptive behavior problems

Introduction

The majority of youths served in outpatient public mental health service systems are

referred for disruptive behavior problems (DBPs; Garland et al. 2001; Weersing and Weisz

R. Zoffness � A. Garland � L. Brookman-Frazee � S. Roesch SDSU/UCSD Joint Doctoral Program in Clinical Psychology, San Diego, CA, USA

R. Zoffness � A. Garland � L. Brookman-Frazee Department of Psychiatry, University of California, San Diego, CA, USA

R. Zoffness (&) � A. Garland � L. Brookman-Frazee Child and Adolescent Services Research Center, Rady Children’s Hospital-San Diego, 3020 Children’s Way, MC 5033, San Diego, CA 92123, USA e-mail: [email protected]

S. Roesch Department of Psychology, San Diego State University, 5500 Campanile Drive, San Diego, CA 92182, USA

123

Child Youth Care Forum (2009) 38:185–200 DOI 10.1007/s10566-009-9077-7

2002). These youths are at high risk for a variety of maladaptive adolescent and adult

outcomes; DBPs are also challenging and costly for families and for society at large (Earls

1994). Improved knowledge about treatment for this patient population is therefore

important. Research indicates that specific evidenced-based practices (EBPs) can have a

significant impact on improving outcomes for youth with DBPs in randomized, controlled

trials (Kazdin 2000; Waddell and Godderis 2005). EBPs such as Parent Management

Training (PMT), anger management and social problem-solving interventions have been

shown to be effective treatments for this population (Brestan and Eyberg 1998; Chambless

et al. 1998; Hoagwood et al. 2001; Kazdin and Wassell 2000).

Despite the growing research base on efficacious interventions for youth with DBPs, little

is known about how often or intensively psychotherapeutic strategies that are common in

EBPs interventions for children with DBPs are being utilized by community-based clini-

cians (Garland et al. 2006a, b; Zima et al. 2005). There have been no large-scale studies

identifying therapeutic strategies common in usual care (UC) psychotherapy practice for this

population. A recent meta-analysis of 32 randomized trials that directly compared EBPs

with UC found that usual care was not as effective as evidence-based treatments in treating

youth with multiple diagnoses (Weisz et al. 2006). These researchers also found that

descriptions of UC psychotherapy were so meager that they did not permit adequate char-

acterization, concluding that an essential step in ultimately improving services is descriptive

research about usual care. Our research group is conducting a large scale study character-

izing UC psychotherapy for children with DBPs to meet the need for a greater understanding

and improvement of usual care. An understanding of UC practice is a critical first step

towards implementing evidence-based practices in community settings and identifying

potential barriers to implementation and dissemination of evidence-based practices.

While one of the main goals of our research characterizing UC psychotherapy for

children with DBPs is to examine the use of strategies common in EBPs for this popu-

lation, we are also interested in examining other strategies that are highly relevant to

clinicians’ work and may impact the implementation of EBPs. There are data to suggest

that one of the major components of community-based treatment for youth is more general

clinical case management (CM) (King 2006; McPherson et al. 2004; Ziguras and Stuart

2000). Research suggests that quality mental health care for children may include psy-

chosocial interventions, familial involvement and appropriate linkage to other service

sectors (Zima et al. 2005). Children receiving publicly-funded mental health services often

require intensive services for extended periods of time because they have severe symp-

toms, low socioeconomic status, and are exposed to multiple psychosocial stressors (Zima

et al. 2005). These stressors may include a history of physical or sexual abuse, living in

out-of-home placements, parental mental illness, substance abuse, poverty, and/or a history

of CPS involvement, which may all be partially addressed through case management.

Research has shown that children with emotional and behavioral problems tend to miss

school because of suspensions, fear of failure, or low self-esteem (Blanchard et al. 2006).

Children with emotional, behavioral and academic problems are more likely to improve

when supportive relationships among the family, school, and community are facilitated and

integrated into treatment (Burns et al. 2000).

Coordination of this type of ‘‘extra-therapeutic’’ care may be particularly relevant for

children with DBPs, as research suggests that these children have lower rates of partici-

pation in familial and social activities, which may be important as individual interventions

in improving overall development and quality of life (Blanchard et al. 2006). It follows that

treating clinicians will need to coordinate care with teachers and other health and social

service professionals to varying degrees depending on the child’s functioning in different

186 Child Youth Care Forum (2009) 38:185–200

123

environments. Anecdotally, our clinician partners have identified CM as a critical and

significant component of care, reporting that they provide a great deal of CM for most

families. They also suggested that the intensity of CM is associated with child/family

characteristics and therapist training and discipline. UC clinicians emphasize the impor-

tance of the child’s family, school, social and community context, suggesting that coor-

dination of care across multiple contextual layers is critical. Research further supports the

importance of integrating case management into treatment. Multisystemic Therapy (MST),

which targets not only the child, but his/her environment, including family, peers, school

and neighborhood, has a strong evidence base (Henggeler et al. 1998, 2002; Littell 2005).

In MST, care is coordinated across multiple sectors, as case management is considered an

essential aspect of treatment.

Case management is operationally defined in this study as coordinating care with

psychiatrists, school personnel, and other service providers: facilitating respite care for

caregivers; attending and reviewing individualized education programs (IEP), and making

referrals to other home and community-based services for the family (Rapp 1998). This

includes home- or community-based programs which provide parent training, social skills

training, or other psychosocial interventions. For example, clinicians may involve Child

Protective Services (CPS) when a child’s safety is at risk. They may also communicate

with school personnel to coordinate services and consistently implement behavioral plans.

Further, clinicians may refer children and families to community-based programs to foster

social skills, encourage interaction with peers, and help give structure to their everyday

lives (McGrew et al. 2003).

Given that CM may be particularly relevant in treating the complex needs of youth with

DBPs served in UC outpatient psychotherapy services, and anecdotal reports from clinicians

suggest that coordination of care is crucial, it is important to characterize how often and

intensively CM is used in community-based psychotherapy and what predicts clinician use

of this element of treatment. As such, our goal is to define and measure the prevalence of,

and assess the predictors of, case management in UC as a first step towards examining the

contribution of CM to treatment outcomes. Specifically, this study examines how often and

intensively community-based clinicians at six outpatient clinics in a large, urban area use

CM strategies to coordinate extra-therapeutic care for children with DBPs, and the youth

and clinician factors that predict the use of this strategy. These potential factors include

child symptom severity, clinician discipline, clinician theoretical orientation and demo-

graphic variables. We specifically hypothesize that youth with high symptom severity will

be associated with more intensive coordination of care, given that their functioning is likely

impaired across multiple domains. We further hypothesize that social workers will use CM

more intensively than therapists trained in non-social work disciplines, such as psychology

or marriage and family therapy, since social work training focuses on coordination of care

across multiple service sectors and knowledge about various community-based resources.

Method

This study represents a preliminary sub-study of a larger project designed to characterize

treatment processes and outcomes in UC psychotherapy for children with DBPs and their

families, and to examine the extent to which UC psychotherapy is consistent and incon-

sistent with common elements of EBPs for this patient population (Practice and Research:

Advancing Collaboration, PRAC; Garland et al. 2006a, b). To identify and operationalize

common elements of evidence-based principles for youth with DBPs, the PRAC research

Child Youth Care Forum (2009) 38:185–200 187

123

team reviewed several comprehensive reviews of efficacious treatments for children with

DBPs (Brestan and Eyberg 1998; Burns et al. 1999; Garland et al. 2008; Kazdin and Weisz

1998). Investigators consulted with a ‘‘Therapist Advisory Group’’ (TAG) to generate

additional common elements of UC psychotherapy for children with DBPs. Case man-

agement was identified as an important strategy via this collaborative process, and was

added to the observational coding system Therapy Process Observational Coding Sys-

tem—Strategies (TPOCS-S) used to characterize UC practice in this study. This investi-

gation focuses specifically on this practitioner-identified element of treatment.

Participants

Family Participants

All new patients entering the six participating clinics who granted permission to be con-

tacted by research staff through an initial screen at the clinic (according to HIPAA reg-

ulations) were recruited consecutively by the research staff. Participants were recruited into

the study if they were beginning a new episode (defined as no treatment in the previous

3 months) of outpatient psychotherapy with a participating psychotherapist and if they met

the following eligibility criteria as determined in the baseline telephone screening inter-

view with research staff: (a) disruptive behavior problem was one of the presenting

problems for treatment (including aggression, defiance, delinquency, oppositional behav-

ior), (b) child age between 4–13 years, (c) primary language for child and caregiver was

English or Spanish. Family participants were given financial incentives to participate in the

study ($40 to the caregiver and $10 to the youth at the baseline interview), but were

assured that their decision regarding participation would not impact treatment. There were

no inclusionary criteria based on diagnosis. Not limiting participants based on diagnosis

allowed us to include subjects representing a wide range of the patient population with a

variety of symptom severity scores and comorbidities. Youth were excluded if they had

documented mental retardation (IQ \ 70), significant organic brain damage or major medical problems, as they may have required specific adaptations of psychotherapy and

thus, their treatment would not be representative of UC practice with youths with DBPs.

The participants of this sub-study were 120 children ages 4–13 (M = 8.82, SD = 2.58) with DBPs receiving publicly funded outpatient mental health services in San Diego,

California. Seventy-two percent of the sample was male (n = 86). Forty-seven percent were Caucasian (n = 57), 29% Latino (n = 35), 5% African–American (n = 6), and 18% Mixed/Other (n = 22). This was a partial sample of those for whom data were available at the time of these analyzes. See Table 1 for a summary of child characteristics.

Clinician Participants

The sample included 58 clinicians practicing in six community-based clinics in San Diego.

Eighty-four percent (n = 49) of this sample was female, with a mean of 3 years of psy- chotherapy experience (SD = 1.94; range 0–15 years). Sixty percent of the clinician

sample was Caucasian (n = 35), 7% Latino (n = 4), 3% African American (n = 2) and 16% Mixed/Other (n = 10). Multiple primary therapeutic disciplines were represented; 60% were Marriage and Family Therapists (MFT; n = 35), 26% Psychologists (n = 15), and 14% Social Workers (n = 8). Multiple theoretical orientations were also represented; when therapists were asked to identify their predominant theoretical orientation, 35%

identified with Family Systems (n = 20), 26% Cognitive Behavioral (n = 15), 21%

188 Child Youth Care Forum (2009) 38:185–200

123

Eclectic (n = 12), 7% Psychodynamic (n = 4), 4% Behavioral (n = 2), 4% Humanistic (n = 2) and 5% Other (n = 3). Sixty percent of the clinicians were trainees (n = 35), while 40% were staff (n = 23). See Table 2 for a summary of clinician characteristics.

Measures

Information about child/family demographics and symptom severity were collected during

the in-person baseline interview with youth and caregivers. The baseline assessment took

approximately 60 min for the caregiver and 30 min for children over age 8. Data was not

collected directly from children younger than age 9, but was instead collected from the

caregiver. Clinicians completed a self-report questionnaire at baseline to assess age,

gender, race/ethnicity, level of experience (staff vs. trainee), primary theoretical orientation

and discipline.

The Eyberg Child Behavior Problems Checklist (ECBI; Eyberg and Ross 1978) was

used to measure symptom severity at baseline in order to relate child clinical character-

istics at baseline to clinician use of CM. The ECBI is a parent report measure consisting of

36-items designed to assess youth behavior problems (Eyberg and Ross 1978). This

measure has two components: the Intensity score, which examines the frequency of

behaviors on a 7-point Likert scale ranging from 1 (‘‘never’’) to 7 (‘‘always’’), and the

Total Problem score, which indicates whether or not a behavior is a problem on a

dichotomous ‘‘yes’’ or ‘‘no’’ scale. The psychometric characteristics of the EBCI are

strong. Test-retest reliability coefficients for the Intensity and Problem scales were .86 and

.88, respectively (Robinson et al. 1980). Internal consistency for the Intensity scale and the

Problem scale is high (Eyberg and Robinson 1983; Robinson et al. 1980). Because the two

ECBI scales were multicollinear (r = .738, p \ .001), only the Parent Intensity scale was used. The Intensity scale was selected because it allowed us to test our a-priori hypothesis

that greater symptom severity would be associated with more intensive use of CM.

Procedures

Treatment Process Measures

The TPOCS-S for child psychotherapy is a comprehensive coding system developed to

characterize psychotherapy for youth (McLeod and Weisz 2005). The TPOCS-S was

Table 1 Characteristics of par- ticipating children (n = 120)

Child characteristics n M (SD) or % Range

Child age at baseline 120 8.8 (2.5) 4–13

Child gender

Male 86 71.7%

Female 34 28.3%

Child race/ethnicity

Caucasian 57 47.5%

Latino/Hispanic 35 29.2%

Multiracial 22 15.8%

African American 5 5.0%

Native American 3 2.5%

Asian American/Pacific Islander 0 0%

Child Youth Care Forum (2009) 38:185–200 189

123

adapted specifically for the PRAC study, and is designed to assess a wide variety of

intervention strategies. The TPOCS-S includes ratings of the occurrence (frequency) and intensity (thoroughness) of clinician use of 27 different types of intervention strategies. The therapy procedures checklist (TPC; Weersing et al. 2002) was a primary source for

TPOCS-S content. The PRAC research team further collaborated with the Therapist

Advisory Group (TAG), which included one clinician from each of the six clinics par-

ticipating in this study (Garland et al. 2006a, b). The TAG reviewed the TPOCS-S and

suggested additions of therapeutic strategies that were not previously included. Specifi-

cally, practitioners reported that they spend a lot of time in session with this patient

population using CM and coordinating ‘‘extra-therapeutic’’ care. Thus, a ‘‘coordination of

external care’’ code, reflecting CM, was subsequently added to the TPOCS-S. Importantly,

the TAG reported that although a great deal of CM activities occur outside of therapy

sessions (e.g., phone consultation, attending an IEP), observed, in-session CM is an ade-

quate proxy for how much total CM is used with individual patients. This is because

clinicians typically discuss these CM activities in-session with the child and/or caregiver

(e.g., reviewing what occurred during the IEP).

Table 2 Characteristics of par- ticipating therapists (n = 58)

Therapist characteristics n M (SD) or %

Gender

Female 49 84.5%

Male 9 15.5%

Years practiced 2.9 (1.9)

Race/ethnicity

Caucasian 35 60.3%

Asian American/Pacific Islander 7 12.1%

Multiracial 6 10.3%

Hispanic/Latino 4 6.9%

African American 2 3.4%

Other 3 5.2%

Filipino/a American 1 1.7%

Mental health discipline

Marriage, family counseling 35 60.3%

Psychology 15 25.9%

Social work 8 13.8%

Primary theoretical orientation

Family systems 20 34.5%

Cognitive behavioral 15 25.9%

Eclectic/integrated 12 20.7%

Psychodynamic/psychoanalytic 4 6.9%

Other 3 5.2%

Behavioral 2 3.4%

Humanistic/client centered 2 3.4%

Status 23 39.7%

Trainee 35 60.3%

Staff 23 39.7%

190 Child Youth Care Forum (2009) 38:185–200

123

While the TPOCS-S includes 27 individual psychotherapeutic strategies, this paper

focuses exclusively on the code capturing CM (‘‘Coordinating Child’s External Care’’).

This strategy is defined as in-session discussions relating to coordinating care with school

staff, case managers, or referrals to or consultations with physicians/psychiatrists, social

workers, clinicians, probation officers, etc. CM also includes discussions about community

care and ancillary services such as Child Protective Services (CPS) and Individualized

Education Program (IEP) meetings. Coordination of care also includes disposition plan-

ning, such as arranging referrals to other treatment settings and providers, and discussions

about structured extra-curricular or community-based activities (e.g., after school pro-

grams). As with all TPOCS-S codes, both the occurrence and intensity of CM was rated.

Occurrence indicates whether it was observed during a session, regardless of how inten- sively it was employed. Intensity reflects both the time spent on CM and the thoroughness with which it was pursued. Intensity was rated on a Likert scale of 0 (not used) to 6 (high

intensity). Intensity scores of 1–2 = low, 3–4 = medium, and 5–6 = high intensity. A

high intensity score was assigned if the coder observed the clinician spending considerable

time reviewing care coordination, needs for other services, obstacles to receiving necessary

services, or identifying community-based programs. For example, if a therapist simply

mentioned to the caregiver that he/she spoke with the child’s teacher, this would be a low

intensity example of therapist use of CM, and would be scored as a 1–2. If, however, the

therapist went into great detail with the parent about this meeting, spending a significant

amount of time providing psychoeducation as well as references to other community-based

services such as after-school programs, this would be a more extensive example of ther-

apist use of CM and would be scored as a 5–6. Thus, intensity scores depend both on the

length of time the therapist spends on the topic of CM as well as the breadth of content.

The presence (occurrence) and intensity of CM use were coded separately for youth and

caregivers. That is, separate ratings were assigned to each potential ‘‘target’’ of CM.

In order to maximize the number of behavioral observations of CM, all sessions in

which this strategy was observed were included in the analyzes, irrespective of whether the

target was the caregiver or the child. In the case that CM was delivered to both the child

and to the caregiver, the highest intensity rating was used for analysis.

Reliability

Forty-two percent (n = 153) of the total 364 sessions were randomly selected to be double- coded to test inter–rater reliability. Inter–rater reliability was assessed separately for child

and for caregiver targets. The mean ICC for CM directed to caregivers = .80 and

Kappa = .69. When CM was directed to children mean ICC = .74 and Kappa = .49.

These reliability estimates reflect adequate to strong inter–rater reliability (Cicchetti 1994;

Landis and Koch 1977).

Psychotherapy Process Data Collection

To gather observational data on clinician use of CM in treatment, all sessions were vid-

eotaped up to 16 months, or whenever the child stopped attending therapy. Consent for

participation in the study included consent by the clinician, caregiver and child to vid-

eotape sessions. Videotaped sessions were randomly selected and assigned for coding

according to the following schedule: 4 sessions for the first 4 months; 3 sessions for moths

5–8; 2 sessions for months 9–12, and 1 session for months 13–16 to gather the most data

during the time frames in which most participants were still in treatment. The project

Child Youth Care Forum (2009) 38:185–200 191

123

coordinator set up video equipment prior to treatment and collected videotapes from

clinicians. Coding was conducted by 10 trained coders who were all research assistants for

this study (see PRAC Coder Training). None of the coders were involved in therapy with

any of the PRAC participants at any time during the study. Coders entered data directly

onto the computer while viewing sessions using a customized, Microsoft Access software

program developed by one of the co-investigators.

PRAC Coder Training on TPOCS-S

Coder training was a multi-step process which involved four group sessions and six

individual training sessions. Group training sessions included an overview of coding

procedures and reviewing each individual code. Coders learned to specifically identify

when clinicians were using CM. Coders learned how to use the computer coding program,

coded sample sessions together as a group, and reviewed intensity ratings. Coders were

considered ‘‘trained’’ when they reached 80% agreement (within one point of intensity

ratings) with ‘‘gold standard’’ codes on at least three consecutive practice sessions.

Analyzes

Multilevel modeling was employed to examine the associations between child and ther-

apists characteristics with average intensity of CM. Multilevel modeling was chosen over

procedures that use ordinary least squares estimations because this technique provides

better parameter estimates when data are nested (Roesch et al. 2009). MPlus software was

used to account for the nested structure of the data, wherein sessions (n = 364) are nested within children (n = 120) are nested within clinicians (n = 58) are nested within clinics (n = 6). Primary analyzes were conducted on six-two-level models: sessions (level 1) nested within children (level 2), sessions (level 1) within clinician (level 2), sessions (level

1) within clinic (level 2), child (level 1) within clinician (level 2), child (level 1) within

clinic (level 2), and clinician (level 1) within clinic (level 2). Because of the nested

structure of the data, average cluster sizes were calculated to examine the data at each

level.

Multilevel modeling was used to determine at which levels—session, child, clinician

and/or clinic—a significant amount of variation was accounted for, indicating that vari-

ables at this level could account for differences in clinician use of CM. After identifying

the levels at which differences in the use of CM were statistically significant, bivariate

correlations, t-tests and ANOVAs were used to determine which level 2 predictors were significant at the child and clinician levels. Predictors that were significant at the bivariate

level were then entered into MPlus to test multivariate models.

Results

Occurrence and Intensity of CM

Overall, CM was observed in 71.4% (n = 260) of all 364 therapy sessions directed to either caregivers or to children. In those 260 sessions in which CM was observed, the

average intensity was 2.50 on a scale of 1–6 (SD = 1.43). In 28.5% of the sessions

(n = 104), the intensity rating was moderate at 3 or higher. CM directed to the caregiver,

192 Child Youth Care Forum (2009) 38:185–200

123

specifically, was observed in 72.4% of the sessions in which a caregiver was present, with

an average intensity of 2.83 (SD = 1.21). This was one of the most frequently coded items

directed to the caregiver out of the 27 TPOCS-S codes. CM directed to the child was

observed in 44.9% of sessions, with an average intensity rating of 2.02 (SD = 1.44).

Multilevel Modeling

Average cluster sizes are presented in Table 3. There were an average of 3.03 sessions

coded per child (range 1–8), 6.27 sessions per clinician (range 1–26), and 60.77 sessions

per clinic (range 14–128). There were an average of 2.06 children per clinician (range

1–7), and an average of 20 children per clinic (range 4–48). There were an average of 9.67

clinicians per clinic (range 2–22).

The intraclass correlation coefficient (ICC) was generated using the null or the inter-

cept-only model. The ICC provides an indication of variance for the outcome variable,

intensity of use of CM, at both levels of the data structure. An ICC greater than .05 (or 5%

of the variance) was used as a cutoff to warrant inclusion of the higher level (level 2)

variable in each separate analysis. Design effects (DE) were also calculated, which take

into account both the ICC and the cluster size. DEs indicate whether the variance for

clustered or nested data is comparable to that of a simple random sample. Intraclass

correlation coefficients are summarized in Table 4.

The ICC calculated for the nested data structure where sessions (level 1) are nested

within children (level 2) was .162 (DE = 1.33), indicating that that 16.2% of the variance

in intensity of therapist use of CM was due to differences among children. The ICC

calculated for sessions (level 1) nested within clinicians (level 2) was .103 (DE = 1.54),

indicating that 10.3% of the variance in intensity of use of CM across therapy sessions was

accounted for by differences among clinicians. The ICC calculated for children (level 1)

nested within clinicians (level 2) was .280 (DE = 1.30), indicating that 28% of the vari-

ance in intensity of CM across children was accounted for by differences among clinicians.

The ICC calculated for clinicians (level 1) nested within clinics (level 2) was not

significant (ICC = .001, DE = 1.01), suggesting that when CM intensity scores were

Table 3 Average cluster sizes

Level 2 Level 1

Session (n = 364) Child (n = 120) Clinician (n = 58)

Child 3.03 (range 1–8)

Clinician 6.27 (range 1–26) 2.06 (range 1–7)

Clinic 60.77 (range 14–128) 20 (range 4–48) 9.67 (range 2–22)

Table 4 Intraclass correlations for average therapist use of CM at session, child and clinician levels

Level 2 Level 1

Session (n = 364)

Child (n = 120)

Clinician (n = 58)

Child .162

Clinician .103 .280

Clinic .006 .011 .001

Child Youth Care Forum (2009) 38:185–200 193

123

averaged across clinicians, clinics did not account for a significant amount of variance. The

ICCs for sessions (level 1) nested within clinic (level 2) (ICC = .006, DE = 1.36) and

children (level 1) nested within clinic (level 2) (ICC = .011, DE = 1.21) were also not

significant, suggesting that there was no difference in the use of CM due to variation

among clinics. As such, clinic predictors were not considered for further analyzes. The fact

that all design effects were close to one further strengthens the reliability of this sample, as

it suggests that the variance for the clustered or nested data is roughly the same as if the

sample was drawn as a simple random sample.

Identifying Potential Predictors

We used bivariate correlations, t-tests and ANOVAs to identify potential predictors of clinician use of CM. Potential predictors categorized as ‘‘child characteristics’’ were child

gender, age at baseline, symptom severity (ECBI intensity score) and child race/ethnicity.

Predictors categorized as ‘‘clinician characteristics’’ were clinician gender, race/ethnicity,

level of experience (staff vs. trainee), discipline (Social Worker, Marriage and Family

Therapist (MFT), Psychologist), and self-reported theoretical orientation (Family Systems,

Cognitive Behavioral, Eclectic, Psychodynamic, Behavioral Humanistic and Other). Based

on our a-priori hypothesis that social workers would use CM more intensively than cli-

nicians from any other discipline, we created the additional variable ‘‘social workers (SW)

versus others.’’ These analyzes are summarized in Table 5.

Child Characteristics

Results indicated that when sessions (level 1) were nested within children (level 2), child

gender was significantly associated with use of CM. Specifically, males were observed

receiving more intensive CM (M = 1.96, SD = 1.67) than females (M = 1.45,

Table 5 Associations between child- and clinician-level predictors and average therapist use of CM

Variable Level 1 = Session Level 2 = Child

Level 1 = Child Level 2 = Clinician

Level 1 = Session Level 2 = Clinician

Test statistic

p-value Test statistic

p-value Test statistic

p-value

Child

Gender (male = 0) t = 2.56 p = .011 t = 3.45 p = .001 NA NA

Sx severity r = .163 p = .010 r = .201 p = .026 NA NA

Race/ethnicity – – – – NA NA

Clinician

Gender (male = 0) NA NA – – – –

Race/ethnicity NA NA – – – –

Discipline (SW vs. Others) NA NA – – t = 2.38 p = .018

Theoretical orientation NA NA – – – –

Level of training: staff versus trainee

NA NA – – – –

Note: Therapist use of CM was aggregated at each level of the data structure. Associations that were not significant are indicated by a dash. Analyzes that were not conducted due to the nested structure of the data are indicated by the notation NA (not applicable)

194 Child Youth Care Forum (2009) 38:185–200

123

SD = 1.56; t(362) = 2.56, p = .011). Greater child symptom severity, as measured by the baseline ECBI intensity score, was positively and significantly correlated with CM use

(r = .163, p \ .01). The more severe a child’s symptoms, the more intensively clinicians used CM. When children (level 1) were nested within clinicians (level 2), being male was

associated with more intensive use of CM than being female (M = 2.18, SD = 1.46 vs. M = 1.22, SD = 1.15; t(118) = 3.45, p \ .001), as was greater symptom severity (r = .20, p = .026).

Clinician Characteristics

When sessions (level 1) were nested within clinicians (level 2), there was a significant

association between clinician discipline and how intensively clinicians used CM. Specif-

ically, social workers used CM more intensively than psychologists and MFTs combined

[t(362) = -2.38, p = .018].

Multivariate Analyzes

Variables that were significantly associated with therapist use of CM in the previous step

were entered into two-level models to test their relationship to each other and to the

dependent variable, intensity of use of CM. MPlus was used for the multilevel model

testing to adjust for the influence of factors at both levels of the model. Predictors with

z-scores [1.96 or \-1.96 were considered significant at p \ .05 and were included in the model. When sessions were nested within children, child gender and symptom severity

were significantly and uniquely associated with how intensively therapists used CM. More

intensive CM was observed with males compared to females (z = 2.76, p \ .05). Greater symptom severity also predicted more intensive use of CM (z = 3.42, p \ .05). When sessions were nested within clinicians, clinician discipline (social work vs. others) was also

significantly and uniquely associated with intensity of CM; being a social worker predicted

more intensive use of CM than being a psychologist or MFT (z = 2.42, p \ .05). Thus, both clinician and child characteristics predicted differences in how intensively clinicians

used CM. Significant results of these multivariate analyzes are summarized in Table 6.

MPlus could not estimate the standard errors of the children nested within clinician

model because of low sample size at both levels, so multiple regression was used to

analyze the model. The model summary suggested that the model was a good fit

(R2 = .137, F(2, 117) = 9.28, p \ .001). The standardized regression coefficients revealed that gender and symptom severity both significantly and uniquely contributed to model fit

(b = .309, p \ .001 and b = .214, p \ .01, respectively). According to Cohen (1988), the

Table 6 Multilevel models significantly associated with average therapist use of CM

Variable Level 1 = Session Level 2 = Child

Level 1 = Child Level 2 = Clinician

Level 1 = Session Level 2 = Clinician

Test statistic p-value Test statistic p-value Test statistic p-value

Child gender (males = 0) z = 2.76 p \ .05 z = 2.76 p \ .05 NA NA Child sx severity z = 3.42 p \ .05 – – NA NA Clinician discipline

(SW vs. Others) NA NA – – z = 2.42 p \ .05

Child Youth Care Forum (2009) 38:185–200 195

123

effect size was medium for gender and small to medium for symptom severity. Being male

and having higher caregiver-reported symptom severity intensity scores independently

predicted higher use of CM when both predictors were included in this model.

Discussion

Our findings suggest that case management strategies are not only used in usual care

psychotherapy with children, but are highly prevalent, observed in 71% of sessions. In fact,

CM was one of the most frequently observed of the 27 therapeutic strategies measured by

the TPOCS-S system. CM includes coordinating care with psychiatrists and teachers,

making referrals to community-based resources, including after-school programs and

respite care. These CM strategies, on average, were observed at low to moderate intensity.

Research suggests that CM is particularly important for children with serious behavioral

and emotional problems (Neill 2006). Specifically, our clinician partners indicate that a

significant part of usual care is CM, which involves communicating and planning with

people and agencies to attain positive outcomes for both child and family. Case man-

agement is an element of care that is not necessarily emphasized in many evidence-based

practices. Although there are overlapping elements of psychotherapeutic interventions for

children with DBPs, including youth skill-building and parenting skills, it appears that the

more highly structured, empirically supported treatments may not explicitly emphasize the

CM aspects of treatment. Concurrently, it is possible that the amount of time clinicians

spend coordinating care for, and with, these families makes it challenging to implement

structured EBP protocols. It should be noted, however, that this study is not designed to

address the extent to which the emphasis on CM is essential to effective care; that is, we

cannot determine if the high proportion of therapy time devoted to CM is absolutely

necessary or if it is a ‘‘default’’ approach when therapists perceive a lack of alternative

approaches.

We predicted a-priori that practitioners trained in the social work discipline would use

CM more intensively than clinicians trained as either psychologists or MFTs. This

hypothesis was supported by our results, which suggest that clinician discipline is indeed a

predictor of how intensively clinicians use CM in outpatient psychotherapy with children

with DBPs. This hypothesis is supported by existing literature, which suggests that CM has

deep roots in social work, relying on interagency collaboration to marshal a range of

resources within the community to benefit the child and the family, and is considered a

major component of mainstream social work practice (Neill 2006). The role of social

workers is often that of case manager, in that they integrate formal systems of care into the

activities of families and other primary and community groups (Moore 1990). Accessing

multiple services is emphasized in MSW training programs, such as utilizing interdisci-

plinary teams which include psychiatrists, nurses, social workers and counselors (Rapp

1998).

We also predicted a-priori that the severity of a child’s emotional and behavioral

problems (as indicated by elevated ECBI scores) would lead to greater use of CM, due to

the increased likelihood that the clinician will need to interact with other sectors of care.

This was supported by our results, which suggest that children with greater symptom

severity at baseline received more intensive coordination of care than those with lower

baseline ECBI scores. Because severe symptoms may affect multiple aspects of a child’s

world, including social and academic functioning, it is likely that treatment will not focus

196 Child Youth Care Forum (2009) 38:185–200

123

solely on the child, but also on the school and the larger community in order to maximize

treatment effectiveness.

The final significant predictor of clinician use of CM was child gender. Males were

observed receiving significantly more intensive CM than females. This may be because

males in general tend to manifest their symptoms externally, while females show a ten-

dency towards internalization (Fergusson et al. 1996; Hartung et al. 2002). This pattern is

consistent with current developmental models of DBPs (Lahey et al. 2000). Males may be

more aggressive in session and may engage in more aggressive acts out of session than

females, which may compel clinicians to draw on other systems of care to supplement

outpatient psychotherapy. An alternative or additional explanation is that we may see less

intensive use of CM with females because they are more likely than males to openly

discuss relationships and emotions, which are the focus of more ‘‘traditional’’ psycho-

therapeutic interventions. If females are indeed more engaged in ‘‘typical’’ psychotherapy

than their male counterparts, clinicians may not spend as much time on CM. Further

research is needed to determine whether these hypotheses hold merit. Notably, neither

clinician orientation nor level of experience (staff vs. trainee) was significantly related to

how intensively clinicians used CM. At the child level, demographic variables such as

race/ethnicity and child’s age were also not significant.

Strengths and Limitations

There have been numerous calls to open the ‘‘black box’’ of UC services in order to better

understand psychotherapeutic processes (Bickman 2000; Weisz et al. 2006). This is the

first known study to rigorously examine psychotherapy treatment processes—specifically,

the use of case management—with a broad, representative sample of youth receiving UC

psychotherapy. The representativeness of this sample is supported by a recent national

survey of 1,200 clinicians from 100 clinics across the United States (Glisson et al. 2008). In

the national sample, 76% of clinicians were female and 71% were Caucasian, compared to

84% and 68%, respectively, in this study sample. Slightly more than half of the therapists

in this current study were trainees, a finding supported by another sample of 42 therapists

in four community-based clinics in Los Angeles, where 49% of therapists were trainees

(Hawley and Weisz 2005). Our clinician sample is therefore relatively comparable to

national and local samples on these basic demographic and experience characteristics.

Additionally, our methods were developed in collaboration with community-based

clinician partners, which enabled us to examine elements of EBPs used in UC psycho-

therapy as well as other elements of care. The assessment of CM was specifically driven by

clinician feedback and interest. In addition, our sample of clinicians and patients is gen-

erally representative of the publicly-funded, mental health sector in our county, as opposed

to a highly selective convenience sample. Inter–rater reliability is adequate, lending sup-

port to the rigor of our methods. Furthermore, this study indicates that, with good part-

nership, both clinicians and families are willing to participate in research that is potentially

intrusive in order to learn more about psychotherapy. Documentation of the prevalent use

of CM in UC psychotherapy for children with DBPs has important implications, and

informs our understanding of potential barriers to the implementation of EBPs in com-

munity settings.

One limitation of this study is the fact that there is no standard, operational definition of

case management. This is not a well-researched topic. As researchers in different fields and

disciplines define CM in various ways, it is a difficult construct to measure. However,

because we defined CM based on input from clinicians who work in the field, we feel that

Child Youth Care Forum (2009) 38:185–200 197

123

this is an appropriately defined construct in the context of this study. The community-based

clinicians who participated in this study reported that observed occurrences of CM in the

therapy session are a satisfactory proxy for how much CM they use both in and out of

session; we do recommend, however, that CM be clarified operationally and studied in

greater depth.

We must consider also the possibility that other variables that predict therapist use of

CM were not represented in this model. Findings at the child level might have been

significant if other variables—such as caregivers’ physical and emotional health, socio-

economic status, and history of mental illness—were included. Furthermore, we are unable

to state how well our results generalize to other types of care, such as private insurance-

funded care or care in other geographic locations. For example, there was a high repre-

sentation of trainees and MFTs in our clinician sample, which may not be representative of

privately-funded care. However, this may also be perceived as strength of the study, as it is

representative of UC. In addition, we do not know how much the videotaping process

affected practice itself, although we tried to minimize the impact of this by using small,

unobtrusive cameras, and encouraged clinicians to record all sessions to establish a routine.

Clinicians did report that both they themselves and their patients quickly acclimated to

being videotaped, and did not feel that this was an obstacle to providing treatment as usual

(TAG, personal communication). Additionally, although clinic differences did not account

for a significant amount of variance, it is possible that the climate and culture of the clinics

did impact clinicians’ use of CM, and that this was not captured due to the small sample

size at the clinic level (n = 6). Importantly, this study serves as a base from which other studies can be conducted

regarding the role and importance of CM in the treatment of youths with DBPs. The next

step in this process is to examine how treatment outcomes may be related to the use of CM,

and whether or not more intensive use of this treatment strategy leads to clinical and

functional improvement. Isolating elements of treatment that are commonly used in UC

psychotherapy will help us understand what aspects of treatment are most effective, and

can ultimately serve to facilitate more effective dissemination and implementation of EBPs

by balancing the push for evidence-based practice with practice-based evidence. This study

serves as an example of how researchers and practitioners can work together to facilitate

increased knowledge about treatment as usual. It also suggests that, with good partnership,

both youth patients and their caregivers are willing to be active participants in studies of

psychotherapy to help characterize and ultimately improve usual care. As such, it is our

contention that the gap between research and practice can be bridged via shared knowledge

through communication and collaboration.

Acknowledgements This work was supported by NIH R01MH66070 (A.G.), and K23MH077584 (L.B.F.). The authors thank Bill Ganger for his invaluable assistance, Deb Dupuis, M.P.H. and Robin Taylor for project management, and the Therapist Advisory Group for input.

Open Access This article is distributed under the terms of the Creative Commons Attribution Noncom- mercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

References

Bickman, L. (2000). The most dangerous and difficult question in mental health research. Mental Health Services Research, 2, 71–73. doi:10.1023/A:1010100119789.

198 Child Youth Care Forum (2009) 38:185–200

123

Blanchard, L., Gurka, M., & Blackman, J. (2006). Emotional, developmental, and behavioral health of American children and their families: A report from the 2003 national survey of children’s health. Pediatrics, 117, 1202–1212. doi:10.1542/peds.2005-2606.

Brestan, E., & Eyberg, S. (1998). Effective psychosocial treatments of conduct-disordered children and adolescents: 29 years, 82 sstupies, 5, 272 kids. Journal of Clinical Child Psychology, 27, 180–189. doi: 10.1207/s15374424jccp2702_5.

Burns, B. J., Hoagwood, K., & Mrazek, P. J. (1999). Effective treatment for mental disorders in children and adolescents. Clinical Child and Family Psychology Review, 2, 199–254. doi:10.1023/A:102 1826216025.

Burns, B., Schoenwald, S., Burchard, J., Faw, L., & Santos, A. (2000). Comprehensive community-based interventions for youth with severe emotional disorders: Multisystemic therapy and the wraparound process. Journal of Child and Family Studies, 9, 283–314. doi:10.1023/A:1026440406435.

Chambless, D., Baker, M., Baucom, D., Beutler, L., Calhoun, K., Crits-Cristoph, P., et al. (1998). Update on empirically validated therapies, II. Clinical Psychologist, 51, 3–16.

Cicchetti, D. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290. doi:10.1037/1040- 3590.6.4.284.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.

Earls, F. (1994). Oppositional defiant and conduct disorders. In M. Rutter, E. Taylor, & L. Hersov (Eds.), Child and adolescent psychiatry, modern approaches (pp. 308–329). Oxford, UK: Blackwell.

Eyberg, S., & Robinson, E. (1983). Conduct problem behavior: Standardization of a behavioral rating scale with adolescents. Journal of Clinical Child Psychology, 12, 347–354. doi:10.1207/s15374424 jccp1203_19.

Eyberg, S., & Ross, A. W. (1978). Assessment of child behavior problems: The validation of a new inventory. Journal of Clinical Child Psychology, 7, 113–116.

Fergusson, D., Lynskey, M., & Horwood, L. (1996). Factors associated with continuity and changes in disruptive behavior patterns between childhood and adolescence. Journal of Abnormal Child Psy- chology, 24, 533–553. doi:10.1007/BF01670099.

Garland, A. F., Hawley, K. M., Brookman-Frazee, L. I., & Hurlburt, M. (2008). Identifying common elements of evidence-based psychosocial treatments for children’s disruptive behavior problems. Journal of the American Academy of Child and Adolescent Psychiatry, 47, 505–514.

Garland, A., Hough, R., McCabe, K., Yeh, M., Wood, P., & Aarons, G. (2001). Prevalence of psychiatric disorders in youths across five sectors of care. Journal of the American Academy of Child and Adolescent Psychiatry, 40, 409–418. doi:10.1097/00004583-200104000-00009.

Garland, A., Hurlburt, M., & Hawley, K. (2006a). Examining psychotherapy processes in a services research context. Clinical Psychology: Science and Practice, 13, 30–46. doi:10.1111/j.1468-2850.2006.00004.x.

Garland, A., Plemmons, D., & Koontz, L. (2006b). Research-practice partnership in mental health: Lessons from participants. Administration and Policy in Mental Health and Mental Health Services Research, 33, 517–528. doi:10.1007/s10488-006-0062-2.

Glisson, C., Landsverk, J., Schoenwald, S., Kelleher, K., Hoagwood, K., Mayberg, S., et al. (2008). Assessing the organizational social context (OSC) of mental health services: Implications for research and practice. Administration and Policy in Mental Health and Mental Health Services Research, 35, 98–113. doi:10.1007/s10488-007-0148-5.

Hartung, C. M., Milich, R., Lynam, D. R., & Martin, C. A. (2002). Understanding the relations among gender, disinhibition, and disruptive behavior in adolescents. Journal of Abnormal Psychology, 111, 659–664. doi:10.1037/0021-843X.111.4.659.

Hawley, K., & Weisz, J. (2005). Youth versus parent working alliance in usual clinical care: Distinctive associations with retention, satisfaction, and treatment outcome. Journal of Clinical Child and Ado- lescent Psychology, 34, 117–128. doi:10.1207/s15374424jccp3401_11.

Henggeler, S., Schoenwald, S., Borduin, C., Rowland, M., & Cunningham, P. (1998). Mutisystemic treat- ment of antisocial behavior in children and adolescents. New York: Guilford Press.

Henggeler, S., Schoenwald, S., Rowland, M., & Cunningham, P. (2002). Serious emotional disturbances in children and adolescents: Multisystemic therapy. New York: Guilford Press.

Hoagwood, K., Burns, B., Kiser, L., Ringeisen, H., & Schoenwald, S. (2001). Evidence-based practice in child and adolescent mental health services. Psychiatric Services (Washington, DC), 52, 1179–1189. doi:10.1176/appi.ps.52.9.1179.

Kazdin, A. (2000). Psychotherapy for children and adolescents: Directions for research and practice. New York: Oxford University Press.

Child Youth Care Forum (2009) 38:185–200 199

123

Kazdin, A., & Wassell, G. (2000). Therapeutic changes in children, parents, and families resulting from treatment of children with conduct problems. Journal of the American Academy of Child and Ado- lescent Psychiatry, 39, 414–420. doi:10.1097/00004583-200004000-00009.

Kazdin, A., & Weisz, J. (1998). Identifying and developing empirically supported child and adolescent treatments. Journal of Consulting and Clinical Psychology, 66, 19–36. doi:10.1037/0022-006X.66.1.19.

King, R. (2006). Intensive case management: A critical re-appraisal of the scientific evidence for effec- tiveness. Administration and Policy in Mental Health and Mental Health Services Research, 33, 529– 535. doi:10.1007/s10488-006-0051-5.

Lahey, B., Schwab-Stone, M., Goodman, S., Waldman, I., Canino, G., Rathouz, P., et al. (2000). Age and gender differences in oppositional behavior and conduct problems: A cross-sectional household study of middle childhood and adolescence. Journal of Abnormal Psychology, 109, 488–503. doi:10.1037/ 0021-843X.109.3.488.

Landis, J., & Koch, G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174. doi:10.2307/2529310.

Littell, J. (2005). Lessons from a systematic review of effects of multisystemic therapy. Children and Youth Services Review, 27, 445–463. doi:10.1016/j.childyouth.2004.11.009.

McGrew, J., Pescosolido, B., & Wright, E. (2003). Case managers’ perspectives on critical ingredients of assertive community treatment and on its implementation. Psychiatric Services (Washington, DC), 54, 370–376. doi:10.1176/appi.ps.54.3.370.

McLeod, B., & Weisz, J. (2005). The therapy process observational coding system-alliance scale: Measure characteristics and prediction of outcome in usual clinical practice. Journal of Consulting and Clinical Psychology, 73, 323–333. doi:10.1037/0022-006X.73.2.323.

McPherson, M., Weissman, G., Strickland, B., van Dyck, P., Blumberg, S., & Newacheck, P. (2004). Implementing community-based systems of services for children and youths with special health care needs: How well are we doing? Pediatrics, 113, 1538–1544.

Moore, S. (1990). A social work practice model of case management: The case management grid. Social Work, 35, 444–448.

Neill, T. K. (2006). Helping others help children: Clinical supervision of child psychotherapy. Washington, DC: American Psychological Association.

Rapp, C. (1998). The active ingredients of effective case management: A research synthesis. Community Mental Health Journal, 34, 363–381. doi:10.1023/A:1018783906741.

Robinson, E., Eyberg, S., & Ross, A. (1980). The standardization of an inventory of child problematic conduct behaviors. Journal of Clinical Child Psychology, 9, 22–28.

Roesch, S. C., Aldridge, A. A., Vickers, R. R., & Helving, L. K. (2009). Testing personality-coping diatheses for negative and positive affect: A longitudinal evaluation. Anxiety, Stress, and Coping, 22, 263–282.

Waddell, C., & Godderis, R. (2005). Rethinking evidence-based practice for children’s mental health. Evidence-Based Mental Health, 8, 60–62. doi:10.1136/ebmh.8.3.60.

Weersing, R. V., & Weisz, J. R. (2002). Mechanisms of action in youth psychotherapy. Journal of Child Psychology and Psychiatry and Allied Disciplines, 43, 3–29. doi:10.1111/1469-7610.00002.

Weersing, R. V., Weisz, J. R., & Donenberg, G. (2002). Development of the therapy procedures checklist: A therapist-report measure of technique use in child and adolescent treatment. Journal of Clinical Child and Adolescent Psychology, 31(16), 8–180.

Weisz, J., Jensen, A., & Hawley, K. (2006). Evidence-based youth psychotherapies versus usual clinical care: A meta-analysis of direct comparisons. The American Psychologist, 61, 671–689. doi:10.1037/ 0003-066X.61.7.671.

Ziguras, S., & Stuart, G. (2000). A meta-analysis of the effectiveness of mental health case management over 20 years. Psychiatric Services (Washington, DC), 51, 1410–1421. doi:10.1176/appi.ps.51.11.1410.

Zima, B., Hurlburt, M., Knapp, P., Ladd, H., Tang, L., Duan, N., et al. (2005). Quality of publicly-funded outpatient specialty mental health care for common childhood psychiatric disorders in California. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 130–144. doi:10.1097/ 00004583-200502000-00005.

200 Child Youth Care Forum (2009) 38:185–200

123