Single Study on Collaborative and Proactive Solutions

Parent Management Training and Collaborative & Proactive Solutions: A Randomized Control Trial

for Oppositional Youth

Thomas H. Ollendick, Ross W. Greene, and Kristin E. Austin

Department of Psychology, Virginia Tech

Maria G Fraire

Department of Psychiatry, McLean Hospital=Harvard Medical School

Thorhildur Halldorsdottir

Department of Psychiatry, Max Planck Institute for Psychiatry

Kristy Benoit Allen

Western Psychiatric Institute and Clinic, University of Pittsburgh School of Medicine

Matthew A. Jarrett

Department of Psychology, University of Alabama

Krystal M. Lewis

Department of Psychiatry, University of Illinois at Chicago

Maria Whitmore Smith

Department of Psychology, Virginia Tech

Natoshia R. Cunningham

Division of Behavioral Medicine and Clinical Psychology, Cincinnati Children’s Hospital and Medical Center

Ryoichi J. P. Noguchi

George Mason University

Kristin Canavera

Department of Psychology, St. Jude’s Children’s Research Hospital

Jennifer C. Wolff

Bradley=Hasbro Research Center, Brown University School of Medicine

Correspondence should be addressed to Thomas H. Ollendick, Child Study Center, Department of Psychology, Virginia Tech, 460 Turner St.

NW, Ste. 207, Blacksburg, VA 24061-0293. E-mail: [email protected]

Journal of Clinical Child & Adolescent Psychology, ( ), – , 201

Copyright # Taylor & Francis Group, LLC

ISSN: 1537-4416 print=1537-4424 online

DOI: 10.1080/15374416.2015.1004681

645 5 59 601 4

This study examined the efficacy of Collaborative & Proactive Solutions (CPS) in treating oppositional defiant disorder (ODD) in youth by comparing this novel treatment to Parent Management Training (PMT), a well-established treatment, and a waitlist control (WLC) group. One hundred thirty-four youth (ages 7–14, 61.9% male, 83.6% White) who fulfilled Diagnostic and Statistical Manual of Mental Disorders (4th ed.) criteria for ODD were randomized to CPS, PMT, or WLC groups. ODD was assessed with semistructured diagnostic interviews, clinical global severity and improve- ment ratings, and parent report measures. Assessments were completed pretreatment, posttreatment, and at 6 months following treatment. Responder and remitter analyses were undertaken using intent-to-treat mixed-models analyses. Chronological age, gender, and socioeconomic status as well as the presence of comorbid attention deficit=hyperactivity and anxiety disorders were examined as predictors of treatment outcome. Both treatment conditions were superior to the WLC condition but did not differ from one another in either our responder or remitter analyses. Approximately 50% of youth in both active treatments were diagnosis free and were judged to be much or very much improved at posttreatment, compared to 0% in the waitlist condition. Younger age and presence of an anxiety disorder predicted better treatment outcomes for both PMT and CPS. Treatment gains were maintained at 6-month follow-up. CPS proved to be equivalent to PMT and can be considered an evidence-based, alterna- tive treatment for youth with ODD and their families.

Oppositional defiant disorder (ODD) is a childhood disorder characterized by a recurrent pattern of develop- mentally inappropriate levels of negativistic, defiant, dis- obedient, and hostile behaviors toward authority figures (American Psychiatric Association [APA], 1994, 2013). The prevalence rates for ODD in community samples range from 2.6% to 15.6%, and in clinical samples from 28% to 65% (Boylan, Vaillancourt, Boyle, & Szatmari, 2007; Wolff & Ollendick, 2010). ODD has also been shown to be highly comorbid with other childhood psy- chiatric disorders including attention deficit=hyperactiv- ity disorder (ADHD), the depressive disorders, and the anxiety disorders (ADs; Greene et al., 2002; see Cun- ningham & Ollendick, 2010, for review). There is also a strong association between ODD and conduct dis- order (CD), with a significant proportion of youth who develop CD meeting criteria for ODD prior to the onset of CD (Biederman et al., 1996; Hinshaw, Lahey, & Hart, 1993).

To date, the predominant approach to the treatment of youth with ODD has been Parent Management Training (PMT; Barkley, 1997; Brestan & Eyberg, 1998; Eyberg, Nelson, & Boggs, 2008; Kazdin, 2005; McMahon, Long, & Forehand, 2011). In general, PMT emphasizes ineffective parenting practices in the origins and course of oppositional behavior in youth, and improving compliance is the primary focus of inter- vention. As such, PMT typically includes interventions aimed at helping parents be more consistent and contin- gent in their behavior management practices, including use of clear and direct commands, differential attention, contingent reinforcement, response cost, and time-out from reinforcement. An impressive body of research has documented the efficacy of PMT and the evidence

is sufficiently compelling to qualify PMT as an empiri- cally supported, well-established treatment (see Brestan & Eyberg, 1998; Eyberg et al., 2008; and Murrihy, Kidman, & Ollendick, 2010, for reviews). Moreover, PMT interventions have been shown to produce compa- rable results in both efficacy and effectiveness trials in ‘‘real-world’’ clinical settings (Michelson, Davenport, Dretzke, Barlow, & Day, 2013).

However, limitations in PMT outcomes have been noted: Treatment effects are not always replicated, treatment often leaves children with deviant behavior still above the range of normative levels, treatment gains often dissipate once the intervention is removed, attrition rates are as high as 50%, and there is some evidence to suggest that older, more aggressive youth may not benefit from such treatment (Frick, 2001; Kazdin, 2005; Ollendick & Cerny, 1981). In addition, Greene and colleagues (Greene, 1998, 2010; Greene & Doyle, 1999) have argued that PMT does not address the reciprocal, adult-child processes giving rise to oppositional behavior in many children. In response, Greene (1998) proposed an alternative model for the treatment of ODD, now called Collaborative & Proactive Solutions (CPS; previously referred to as Collaborative Problem Solving). The CPS intervention model, based on Greene’s (1998) book The Explosive Child, emphasizes lagging skills—especially in the domains of flexibility, adaptability, and problem solving—as a major factor contributing to the development of oppositional behavior in youth. In contrast to PMT, CPS focuses primarily on helping parents and children learn to collaboratively and proactively solve the problems that contribute to these challenging behaviors.

OLLENDICK ET AL.592

Preliminary research has suggested the effectiveness of CPS in a small clinic sample of 47 urban youth from the Boston area with ODD (Greene et al., 2004). This preliminary trial showed that the effects of CPS were commensurate with PMT (based on Barkley’s 1997 program for defiant children) on most measures of treat- ment outcome both at posttreatment and at 4-month follow-up. However, this study was limited by sample size (28 children in CPS, 19 in PMT) and the lack of a waitlist control group; moreover, the follow-up period was only 4 months in duration. Although these initial results are promising, they require replication in a larger sample of children with ODD and in a randomized control trial with a longer follow-up period.

The present study builds upon this early study and uses a randomized control design to test the compara- tive efficacy of PMT, CPS, and a waitlist control con- dition (WLC) in a larger sample of youth with ODD. First, we predicted that both treatment conditions would be superior to the WLC condition. Second, given the clear support for PMT as an efficacious treatment for ODD and the emerging support for CPS as an effi- cacious treatment for ODD, we predicted that the two treatments would be equivalent to one another. Third, we explored predictors of change associated with these treatments. Specifically, we examined chronological age, gender, socioeconomic status, and presence of diag- nostic comorbidity as potential predictors of treatment outcome. Although firm support for these predictions is lacking, we tentatively hypothesized that increasing age, male gender, and low socioeconomic status would be associated with fewer positive treatment outcomes. We also predicted that presence of comorbid ADHD would produce fewer positive treatment outcomes but the presence of a comorbid anxiety disorder (AD) would produce more positive treatment outcomes. Limited research has shown that ADHD exacerbates the effects of ODD, whereas ADs tend to mitigate the effects of ODD (see Drabick, Ollendick, & Bubier, 2010, for review).

METHOD

Participants

The clinical trial took place in the United States, in rural southwest Virginia. Parents of youth with oppositional problems were both referred by mental health profes- sionals, family practice physicians, and school personnel and recruited through advertisements in local newspa- pers and television programs announcing the clinical trial. Parents of 275 youth completed a brief telephone screen for ODD, as well as for conditions that would preclude the family’s participation in the trial (see exclusion criteria next). Parents whose children appeared to meet eligibility criteria (n¼164) were informed of the procedures of the study including the randomization process. Children and their parents pro- vided written informed assent and consent, as approved by our Institutional Review Board. Subsequently, these parents and their children underwent a comprehensive assessment to confirm the ODD diagnosis and deter- mine associated comorbid disorders. Youth were included in the study if they were between 7 and 14 years of age and met full diagnostic criteria for ODD. As can be seen in Table 1, 64% of the youth had ODD as a pri- mary diagnosis, 30% as a secondary diagnosis, and 6% as a tertiary diagnosis. However, ODD was the principal reason for referral in all instances. Inclusion of youth with primary, secondary, and tertiary diagnoses of ODD was intentional so we could examine the efficacy of our interventions with children who present with varying levels of ODD and comorbid disorders (99% had at least one comorbid disorder and 83% had a second comorbid disorder). The most common comor- bid diagnoses were ADHD and an AD (defined as gen- eralized anxiety disorder, social anxiety disorder, or separation anxiety disorder; see Table 1). Of the 134 participants, 33 (25%) were on stable doses of ADHD stimulant medication, 11 (8%) on ADHD nonstimulant medication, five (4%) on antipsychotic=bipolar

TABLE 1

Primary, Secondary, and Tertiary Diagnoses

Diagnosis

Primary Secondary Tertiary

N (%) N (%) N (%)

Oppositional Defiant Disorder 86 (64) 40 (30) 8 (6)

Attention-Deficit Hyperactivity Disorder 34 (25) 42 (31) 16 (12)

Anxiety Disorder 10 (8) 30 (22) 34 (25)

Specific Phobia 0 (0) 10 (8) 31 (23)

Major Depressive Disorder=Dysthymia 0 (0) 4 (3) 10 (8)

Other Disorders 4 (3) 7 (5) 12 (9)

No Disorder 0 (0) 1 (1) 23 (17)

Note: Anxiety disorder includes diagnosis of generalized anxiety disorder, social phobia, and separation anxiety disorder;

other disorders include diagnoses of obsessive compulsive disorder, post traumatic stress disorder, panic disorder, reactive

attachment disorder, enuresis, and encopresis.

PMT AND CPS 593

medication, four (3%) on antidepressant medication, four (3%) on antianxiety medication, and one (<1%) on antiseizure medication (used to treat anxiety). Youth were excluded if they met diagnostic criteria for CD, autism spectrum disorder, a psychotic disorder, intellec- tual impairment, or current suicidal or homicidal idea- tion. Overall, 134 youth met inclusion criteria and participated in the trial (see Figure 1).

Procedure

Youth were randomly assigned to one of the two active treatment groups or a 6-week WLC. Following the wait- ing period, those youth and families in the WLC group who continued to meet criteria for ODD and still desired

treatment were randomly reassigned to one of the two treatment groups. Youth and their families were also assessed at posttreatment and at 6 months following treatment. Each family received a stipend of $50 for each assessment completed at pretreatment, posttreat- ment, and 6-month follow-up for a total of $150.

At each assessment session, two clinicians were assigned to each family. All clinicians were supervised research assistants, graduate students in clinical psy- chology in our APA-approved clinical scientist doctoral training program, or postdoctoral fellows associated with our Center, and were trained to requisite levels of competence to help ensure reliability and validity of data procurement. None of the assessment clinicians served as therapists for the families they assessed.

FIGURE 1 Flow chart of participants through the study. Note: ODD¼oppositional defiance disorder; PDD¼pervasive developmental disorder; MDD¼major depressive disorder; PMT¼Parent Management Training; CPS¼Collaborative & Proactive Solutions.

OLLENDICK ET AL.594

Eligible families, randomly assigned to treatment, participated in PMT or CPS with a highly trained and closely supervised clinician. All treatment clinicians were post-master’s, doctoral students in our APA-approved clinical scientist doctoral psychology training program or postdoctoral fellows associated with our Center.

Participant Characteristic Measures

In addition to chronological age, gender, race=ethnicity, socioeconomic status (defined by parent education level and family income), and family structure (single= divorced parent, married or cohabiting parent) derived from a demographic form, measures of receptive and expressive language ability were obtained. These latter measures were used to more fully describe our sample and because deficits in these abilities are frequently asso- ciated with ODD (see Kimonis & Frick, 2010).

Peabody Picture Vocabulary Test, fourth edition (Dunn & Dunn, 2007) and the Expressive Vocabulary Test, second edition (Williams, 2007). The Peabody Picture Vocabulary Test, Fourth Edition and the Expressive Vocabulary Test, Second Edition are reliable and valid measures of receptive and expressive language abilities, respectively. They were administered only at pretreatment.

Treatment Response Outcome Measures

Anxiety Disorders Interview Schedule for DSM–IV, child and parent versions (ADIS-C=P; Silverman & Albano, 1996). The ADIS-C=P is a semistructured diagnostic interview designed for the diagnosis of most psychiatric disorders of childhood and adolescence. The clinician assesses symptoms and obtains frequency, intensity, and interference ratings (0–8 scale), which are then used by the clinician to identify diagnostic criteria and to develop a clinician severity rating (CSR). A CSR of 4 or above on a 0–8 scale indicates a diagnosis. Recently, the ADIS-C=P has been found to be reliable and valid for the diagnosis of both ODD and ADHD, in addition to the anxiety and affective disorders (Anderson & Ollendick, 2012; Jarrett, Wolff, & Ollendick, 2007).

The ADIS-C=P has yielded acceptable to excellent 7- to 14-day test–retest reliability (Silverman, Saavedra, & Pina, 2001) and acceptable interrater agreement (Grills & Ollendick, 2003). Trained-to-criterion clinicians con- ducted the diagnostic interviews. Training consisted of a 3-hr workshop on the ADIS-C=P, two practice inter- views with the trainer, two live observations of adminis- tration of the ADIS-C=P with a trained clinician, and two interviews conducted with the trainer in the session

with the trainee. All interviews were videotaped, and 20% of the pretreatment diagnostic interviews were reviewed by a second clinician to compute Kappa coeffi- cients. Using Cohen’s kappa, agreements on diagnoses were .77, .85, and .86 on primary, secondary and tertiary diagnoses. At each time point, consensus diagnoses were determined based on the independent findings of the ADIS-C and ADIS-P. This process occurred in weekly staff meetings with the two ADIS clinicians and the doctoral-level clinical psychologist who supervised diag- nostic assessments. Prior to treatment and for the 6-month follow-up assessment, the full ADIS-C=P was administered. At the posttreatment assessment, only the ADIS-C=P modules of disorders that were endorsed at pretreatment were administered.

Clinical Global Impression–Severity (CGI-S; Guy, 1976). The CGI-S was completed by the same clini- cians who completed the ADIS-IV at the designated assessment intervals. The CGI-S includes a rating, on a 7-point Likert scale, of the child’s current overall impairment, ranging from 1 (normal, not impaired) to 7 (very seriously impaired).

Disruptive Behavior Disorders Rating Scale (DBDRS; Barkley, 1997; Pelham, Gnagy, Greenslade, & Milich, 1992). The DBDRS is composed of the DSM-IV symptom lists for ADHD, ODD, and CD and uses a 4-point response scale ranging from 0 (not at all) to 3 (very much). There are eight symptoms for ODD; individual symptoms coded as a 2 (much) or 3 (very much) are viewed as meeting criteria for the symp- tom. Thus, scores can range from 0 to 8, with a score of 4 or above indicating the necessary symptom count for possible ODD. The DBDRS has been shown to have excellent psychometric properties (Cronbach’s a¼ .90 in the current study; see Pelham et al., 1992, for norma- tive data). This measure was completed by the parents at each assessment point.

Behavior Assessment System for Children–Second Edition (BASC; Reynolds & Kamphaus, 1992). The BASC evaluates the behaviors, thoughts, and emotions of children and adolescents. For the present study, the Aggression scale of the Parent Rating Scales was of pri- mary interest, with T scores greater than or equal to 70 falling in the clinically significant range and T scores from 60 to 69 being considered ‘‘at risk.’’ The Aggression Scale of the Parent Rating Scales possesses acceptable internal consistency (Kamphaus & Frick, 2005; Cronbach’s a¼ .90 in the current study) and test–retest reliability over a 2- to 8-week period (.74– .94; Reynolds & Kamphaus, 1992). This measure was administered at pretreatment, 1-week posttreatment, and 6 months posttreatment.

PMT AND CPS 595

Parent Consumer Satisfaction Questionnaire. The Parent Consumer Satisfaction Questionnaire is a parent-report questionnaire rating satisfaction with and efficacy of the PMT and CPS interventions. Designed for this study, it consisted of seven items each rated on a 7-point scale (rated 0–6) assessing the degree to which the parent was satisfied (ranging from very dis- satisfied to very satisfied) with the PMT or CPS treat- ment and the extent to which they perceived the treatments as helpful for them and their child (ranging from considerably worse to greatly improved). This mea- sure was completed by parents at 1-week posttreatment and at 6-month follow-up.

Treatment Remission Outcome Measures

Clinical Global Impression–Improvement (CGI-I; Guy, 1976). The CGI-I was completed by assessment clinicians. The degree to which the child’s symptoms improved since the beginning of treatment was rated on a 7-point Likert scale (very much improved to mark- edly worse). Symptom improvement for remission was defined as a rating of 1 (very much improved) or 2 (much improved) on the CGI-I. This measure was administered at 1 week and 6 months posttreatment.

Diagnostic status. Diagnostic status was defined as presence or absence of a clinical diagnosis of ODD as determined by consensus diagnosis on the ADIS at the posttreatment assessment and at 6-month follow-up. A CSR less than 4 was required for remission of clinical diagnosis.

Interventions

PMT. In the PMT condition, six therapists (two male, four female) provided treatment based on Barkley’s (1997) training program. This widely used, structured program provides nine consecutive weekly sessions for parents with one additional session 4 weeks after the last session to review and consolidate treatment gains. Based on our pilot work with families in our com- munity, we extended the program to twelve 75-min ses- sions and implemented the follow-up session 2 weeks after the last regularly scheduled session, rather than 4 weeks later. In addition, we modified the program to include the children in each session so that the parents could practice the skills learned prior to implementing them in the home setting. The program includes an explicit description of the goals and content for each ses- sion, along with standard handouts. The treatment is manualized and consists of the following components: (a) educating parents about the causes of defiant, non- compliant behavior; (b) instructing parents on positive

attending through use of ‘‘special time’’; (c) training par- ents to use attending skills to increase compliant beha- vior; (d) increasing the effectiveness of parental commands; (e) implementing a contingency manage- ment program; (f) using the time-out procedure; (g) managing children’s behavior in public places; and (h) using a daily school-home ‘‘report-card.’’ Therapists received a 4-hr training workshop in PMT prior to the beginning of the project and live supervision for 75 min each week from Dr. Ollendick.

CPS. In the CPS condition, eight therapists (four male, four female) provided treatment based on Greene’s CPS model (Greene, 1998, 2010). CPS is orga- nized into four treatment modules: (a) identification of lagging skills and unsolved problems (typical problems include expectations such as completing homework, doing chores, etc.) that contribute to oppositional epi- sodes, and a discussion of how existing parental responses may be counterproductive; (b) prioritiza- tion—helping parents prioritize which unsolved prob- lems will be the focal point of initial problem-solving discussions; (c) introduction of the Plans framework— helping parents understand the three potential responses to solving problems: Plan A (solving a problem unilater- ally, through imposition of adult will and often accompanied by adult-imposed consequences), Plan B (solving a problem collaboratively and proactively), and Plan C (setting aside the problem for now); and (d) implementing Plan B—helping parents and children become proficient in the use of Plan B and largely dis- continuing the use of Plan A. Although the clinician actively guides the problem-solving process initially, the goal of treatment is to help the child and parents become increasingly independent in solving problems together. CPS, implemented in a flexible and individua- lized manner, was also provided in twelve 75-min ses- sions with one follow-up session 2 weeks following the last regularly scheduled session. As with PMT, the child and parent were present in each session so that the skills learned could be practiced in the session prior to imple- menting them in the home setting. The therapists received a 4-hr training workshop in CPS prior to the beginning of the project and supervision via teleconfer- encing for 75 min each week from Dr. Greene.

Treatment Adherence

Treatment adherence for both treatments was assessed with a six-item checklist completed by the supervisors and based on the verbalizations and behaviors of the therapists as observed in the session videotapes and reviewed in supervision. The checklist, completed fol- lowing each session, included three prescriptive and

OLLENDICK ET AL.596

three proscriptive items for each treatment. Representa- tive items included ‘‘Therapists and the parents dis- cussed implementation of a contingency contracting system to monitor specific behaviors and to reinforce and consequate behaviors according to the contracting system’’ for PMT, and ‘‘Therapists instructed parents on three potential response options for dealing with their child’s behaviors and helped them implement Plan B strategies (e.g., how to solve problems collaboratively taking into consideration identifying lagging skills in the child)’’ for CPS.

RESULTS

Statistical Analyses

Differences between treatment groups (i.e., PMT, CPS, WLC) were compared on key demographic variables at pretreatment with ANOVAs and chi-square statistics. For our main treatment outcomes, intent-to-treat analy- ses were conducted with all participants who were assessed and randomized to the treatment protocols (n¼134). For the 6-month follow-up analyses, the WLC condition was omitted because participants in this condition were reassigned randomly to one of the two active treatment conditions. The 6-month analyses included these reassigned participants in the two-group analyses (n¼134). Due to the nested nature of our data (i.e., time points within participants), longitudinal analyses were conducted using mixed-models analyses with full maximum likelihood estimation to deal with missing data. Time (pretreatment¼0, post- treatment¼1, and 6-month follow up¼2) and treat- ment condition (PMT¼0, CPS¼1) were dummy coded. In all analyses, fixed effects were time, treatment condition, and Time�Treatment Condition. The inter- cept and time were random effects. In the predictor analyses models, indicators for time, treatment con- dition, candidate predictor, and all two-way interaction terms were included. Any candidate pretreatment vari- able that had a significant (p < .05) main effect was con- sidered a predictor.

To examine group differences (i.e., PMT, CPS, WLC), traditional null hypothesis significance testing was used. However, equivalence testing was used to explore group comparability of the PMT and CPS con- ditions (Rogers, Howard, & Vessey, 1993). The equival- ence interval was defined as�10% (90% confidence interval) of the PMT group mean (d) and considered necessary to demonstrate a meaningful difference. For treatment remission analyses, we applied multiple impu- tation procedures to account for missing data, as these measures were obtained only at posttreatment and 6-month follow-up. We used pooled estimates across

20 imputations to reduce bias in estimation for these analyses (see Salim, Mackinnon, Christensen, & Grif- fiths, 2008).

Attrition

Initially, 63 participants were randomly assigned to the PMT condition, 60 to the CPS condition, and 11 to the WLC condition, for a total of 134 participants. It should be noted that participants were assigned to the WLC condition only during the first year of the 5-year clinical trial. The WLC was discontinued because none of the 11 participants improved during the 6-week wait period and their clinical state was deteriorating. All WLC participants requested treatment and were subse- quently reassigned randomly but disproportionately to the two active treatment conditions (2:1, CPS: PMT) so that an equal number of participants would be in each treatment condition (n¼67). Although the WLC included a very small number of participants, we retained them in our initial analyses to illustrate the rela- tive effects of PMT and CPS to a no-treatment control condition.

Thirteen participants dropped out of PMT (19.4%) and 15 participants dropped out of CPS (22.4%), defined as completing six or fewer treatment sessions. The cri- terion for completer status (seven or more treatment ses- sions) was used, as all treatment strategies were introduced by that time and the remaining sessions were used for refinement of skills learned in the previous ses- sions. The 106 ‘‘completer’’ families were seen for an average of approximately 12 sessions (M¼11.80, SD¼1.60, range¼7–14; seven families, five PMT and two CPS, were seen for 14 sessions). The number of treatment sessions was determined by clinical need, which led to some variation in number of sessions. The 28 youth who dropped out from treatment did not differ from the 106 who completed treatment on pre- treatment measures of parent-rated aggression, disrup- tive behavior, or clinician rating of clinical severity of ODD. Furthermore, the 28 youth who dropped out from treatment did not differ from those who completed treatment on gender, race=ethnicity, family structure, family income, or age. However, treatment dropouts did have fewer mothers, v2(1, N¼134)¼9.87, p < .01, and fathers, v2(1, N¼134)¼10.31, p < .01, who gradu- ated from college.

Of the 106 participants who completed treatment, 89 were available for posttreatment assessment and 57 for the 6-month follow-up assessment. Reasons for not par- ticipating in posttreatment and follow-up assessment included insufficient time on the part of the family to complete assessments, a stated disinterest in the ‘‘need’’ for assessment, our inability to maintain contact with the family despite at least three attempts to do so, and

PMT AND CPS 597

relocation. Pretreatment difference analyses were con- ducted for posttreatment assessment completers and 6-month follow-up assessment completers. There were significant differences between posttreatment assessment completers and noncompleters on income, t(112)¼ �2.18, p < .05; mother education, v2(1, N¼133)¼8.43, p < .01; and father education, v2(1, N¼123)¼13.01, p < .001. Those who did not complete the posttreatment assessment had fewer mothers and fathers who graduated from college and lower family income. Regarding 6-month follow-up, there were significant differences between assessment completers and noncompleters on family structure, v2(1, N¼133)¼4.12, p < .05; mother education, v2(1, N¼133)¼7.23, p < .01; father education, v2(1, N¼123)¼6.74, p < .01; and income, t(112)¼ �3.11, p < .01. Specifically, those who did not complete the 6-month follow-up assessment consisted of more single-parent families, more mothers and fathers who did not complete college, and lower family income com- pared with those who did complete the 6-month follow-up assessment. As noted, all primary analyses were conduc- ted with the full intent-to-treat sample of 134 participants (67 in PMT and 67 in CPS after waitlist randomization). Attrition did not differ significantly by treatment

condition, v2(1, N¼134)¼0.18, p¼ .67. See Figure 1 for a flow chart of participants.

Sociodemographic and Participant Characteristics

No differences in gender, family structure, maternal education, paternal education, or family income were found among the three treatment conditions. However, significant differences were found for race=ethnicity by treatment condition, with fewer non-Caucasian parti- cipants in the CPS condition than in the PMT or WLC conditions, v2(2, N¼134)¼7.88, p¼ .02. In addition, there were more older children in the WLC condition than in the PMT or CPS conditions, F(2, 131)¼3.19, p¼ .04. Frequencies and percentages of all demographic variables are presented in Table 2. After participants in the WLC condition were rerandomized to PMT or CPS, differences in age were no longer present; however, significant differences remained for race=ethnicity, with fewer non-Caucasian participants in CPS than PMT, v2(1, N¼134)¼7.83, p¼ .01. Finally, no differences in receptive, F(1, 128)¼1.06, p¼ .31, or expressive language, F(1, 123)¼0.22, p¼ .64, existed between the two treatment groups.

TABLE 2

Participant Characteristics for PMT, CPS, and WLC Groups

Categorical Variables

PMTa CPSb WLCc

v2 pN (%) N (%) N (%)

Gender

Male 36 (57) 40 (67) 7 (64) 1.20 .55

Female 27 (43) 20 (33) 4 (36)

Race

Non-Caucasian 16 (25) 4 (7) 2 (18) 7.88� .02 Caucasian 47 (75) 56 (93) 9 (82)

Family Structure

Single Parent 13 (21) 9 (15) 4 (36) 2.85 .24

Two Parents 49 (79) 51 (85) 7 (64)

Mother Education

Not College Graduate 25 (41) 27 (45) 6 (55) 0.86 .65

College Graduate 37 (59) 33 (55) 5 (45)

Father Education

Not College Graduate 33 (59) 36 (63) 3 (30) 3.86 .15

College Graduate 23 (41) 21 (37) 7 (70)

Continuous Variables M (SD) M (SD) M (SD) F p

Age 9.52 (1.80) 9.28 (1.78) 10.73 (1.10) 3.19� .04 Income 66724.89 (37781.72) 64436.00 (34887.94) 78820.00 (55785.98) 0.59 .56

PPVT 109.08 (14.98) 110.10 (12.45) 112.73 (21.57) 0.31 .74

EVT 105.60 (12.46) 107.18 (10.91) 114.50 (14.07) 2.39 .10

Note: Missing data in family structure for PMT (n¼1), in mother education for PMT (n¼1), in father education for PMT (n¼7), CPS (n¼3), WLC (n¼1). PMT¼Parent Management Training; CPS¼Collaborative & Proactive Solutions; WLC¼waitlist control; PPVT¼Peabody Picture Vocabulary Test; EVT¼Expressive Vocabulary Test.

aN¼63. bN¼60. c N¼11. �p < .05.

OLLENDICK ET AL.598

Treatment Adherence

Treatment adherence was determined by aggregate rat- ings obtained from the PMT and CPS supervisors on the three prescriptive and three proscriptive items from the six-item adherence rating checklist. Collapsed across the three prescriptive items, the mean number of items checked was 2.94 for PMT and 2.92 for CPS (maximum score¼3), indicating that the thera- pists focused on the prescribed treatment elements for their respective treatments much or most of the time. For the three proscriptive items, the means were 0.3 and 0.2 (maximum score¼3), respectively, indicating that PMT therapists were not using CPS treatment ele- ments and CPS therapists were not using PMT treat- ment elements. Thus, both treatments were delivered as specified, and there was limited crossover in the therapeutic strategies used.

Treatment Response

Table 3 presents the means and standard errors for the treatment response outcome measures at pretreatment, posttreatment, and 6-month follow-up.

1 One-way analy- ses of covariance were used to assess treatment group

differences at pretreatment, controlling for age and race=ethnicity. No differences were found on any of the outcome measures across treatment conditions at pretreatment.

CSRs. At posttreatment, with the three groups, mixed-models analyses revealed significant time, F(1, 114)¼50.47, p < .001; treatment, F(2, 130)¼7.94, p¼ .001; and Treatment�Time interaction, F(2, 119)¼5.65, p < .01, effects while controlling for age and race=ethnicity. Participants in the PMT (p < .001, Cohen’s d¼1.39) and CPS (p < .001, Cohen’s d¼1.35) conditions experienced greater reduction in ODD CSRs compared to the WLC condition. However, differences between the two active treatment conditions were not significant (p¼ .89). The within-group effect size (Cohen’s d) for PMT from pre- to posttreatment was 1.06 and the effect size for CPS was 1.13. At the 6-month follow-up, with the two-group analyses, there was a significant time effect, F(2, 201)¼88.48, p < .001, whereas the treatment, F(1, 176)¼0.17, p¼ .68, and Treatment�Time, F(2, 202)¼0.06, p¼ .95, effects were nonsignificant. The findings indicate comparable reductions in ODD CSRs between the two treatment conditions at the 6-month follow-up (p¼ .96). Means are presented in Table 3. Equivalence testing indicated that the PMT and CPS groups were comparable at each time point (see Table 4).

TABLE 3

Estimated Marginal Means and Standard Errors of Measures of Treatment Response Controlling for Age and Race=Ethnicity Broken Down by

Treatment Condition at Each Time Point

Measure

PMT a

CPS b

WLC c

M (SE) M (SE) M (SE)

ADIS-CSR

Pretreatment 5.89 (.18) 5.91 (.19) 6.34 (.44)

Posttreatment 3.69 (.22) 3.64 (.24) 6.07 (.44)

6-Month Follow-Up 3.78 (.28) 3.76 (.29) —

CGI-S

Pretreatment 4.47 (.12) 4.39 (.12) 4.49 (.28)

Posttreatment 3.35 (.14) 3.40 (.16) 4.67 (.28)

6-Month Follow-Up 3.72 (.18) 3.37 (.19) —

DBDRS ODD Symptoms

Pretreatment 5.33 (.27) 5.97 (.27) 5.81 (.62)

Posttreatment 2.43 (.34) 2.82 (.42) 5.81 (.62)

6-Month Follow-Up 2.63 (.51) 3.26 (.51) —

BASC Aggression

Pretreatment 70.86 (1.50) 70.70 (1.50) 71.66 (3.43)

Posttreatment 57.68 (1.60) 59.57 (1.87) 72.40 (3.27)

6-Month Follow-Up 57.98 (2.02) 60.51 (2.03) —

Note: PMT¼Parent Management Training; CPS¼Collaborative & Proactive Solutions; WLC¼waitlist control; ADIS-CSR¼Anxiety Disor- ders Interview Schedule clinician severity rating; DBDRS ODD¼Disruptive Behavior Disorders Rating Scale, oppositional defiant disorder; CGI-S¼Clinical Global Impression–Severity (higher scores indicate more severe impairment); BASC¼Behavior Assessment Schedule for Children.

aN¼63. bN¼60. c N¼11.

1 ‘‘Completer’’ analyses (n¼106) were also conducted, and results

were similar to intent-to-treat analyses; therefore, ‘‘completer’’ results

are not reported but are available upon request.

PMT AND CPS 599

CGI-S. Similarly, results of the CGI-S mixed-mod- els mixed models analyses revealed significant time, F(1, 85)¼23.59, p < .001; treatment, F(2, 111)¼3.89, p¼ .02; and Treatment�Time interaction, F(2, 89)¼6.81, p < .01, effects for the three groups when controlling for age and race=ethnicity. The CPS (p < .001, Cohen’s d¼1.06) and PMT (p < .001, Cohen’s d¼1.21) groups displayed a greater reduction in clinical severity than the WLC group, whereas there was no dif- ference between the two active treatment conditions at posttreatment (p¼ .83; see Table 3 for means). The within group effect size (Cohen’s d) for PMT from pre- to posttreatment was .74, and the effect size for CPS was .63. At the 6-month follow-up for the two-group analyses, the main effect for time remained significant, F(2, 160)¼46.83, p < .001, whereas treat- ment, F(1, 133)¼0.93, p¼ .34, and Treatment�Time interaction, F(2, 160)¼0.66, p¼ .52, effects were once again not significant. That is, treatment gains were maintained at the 6-month follow-up for both PMT and CPS (p¼ .18). Equivalence testing indicated that the PMT and CPS groups were comparable (see Table 4).

DBDRS. When controlling for age and race=eth- nicity, results for the mixed-models DBDRS analyses demonstrated significant time, F(1, 80)¼37.47, p < .001; treatment, F(2, 104)¼6.69, p < .01; and Treat- ment�Time interaction, F(2, 85)¼6.62, p < .01, effects for the three groups. Compared to the WLC condition,

the CPS (p < .001, Cohen’s d¼ .96) and PMT (p < .001, Cohen’s d¼1.29) conditions displayed significantly greater reduction in ODD symptoms as measured by the DBDRS, yet no differences were found between the two active conditions (p¼ .47). The within-group effect size (Cohen’s d) for PMT from pre- to posttreat- ment was .87, and the effect size for CPS was .72. Like- wise, at the 6-month follow-up with the two-group analyses revealed a significant time effect, F(2, 119)¼54.68, p < .001, whereas the treatment, F(1, 117)¼1.96, p¼ .16, and Treatment�Time interaction, F(2, 119)¼ .053, p¼ .95, effects were nonsignificant. The findings suggest that the reductions in ODD symp- toms were maintained in PMT and CPS at the 6-month follow-up; however, no difference was found between the two treatment conditions (p¼ .38). Means of the DBDRS broken down by treatment condition and time are displayed in Table 3. Equivalence testing findings for the DBDRS indicated that the PMT and CPS groups were equivalent across time points (see Table 4).

Aggression. When controlling for age and race= ethnicity, results for the mixed-models analyses for the aggression measure revealed a significant main effect for time, F(1, 73)¼40.89, p < .001, and the Treat- ment�Time interaction, F(2, 75)¼9.35, p < .001; how- ever, the main effect for treatment was only marginally significant, F(2, 127)¼2.76, p¼ .07. The significant Treatment�Time interaction demonstrated that aggression (as measured by the BASC at posttreatment)

TABLE 4

Equivalence Testing Between the PMT and CPS Groups

Measure EI (�10%)

PMT CPS

pLower Upper Lower Upper

ADIS-CSR

Pretreatment 0.59 5.57 6.16 5.71 6.12 < .001

Posttreatment 0.37 3.30 4.28 3.12 4.14 < .001

6-Month Follow-Up 0.38 3.23 4.38 3.18 4.37 < .001

CGI-S

Pretreatment 0.45 4.62 4.68 4.44 4.64 < .001

Posttreatment 0.34 3.06 3.55 3.04 3.55 < .001

6-Month Follow-Up 0.37 3.43 4.03 3.06 3.67 .27

DBDRS ODD Symptoms

Pretreatment 0.53 5.06 5.90 5.50 6.32 .01

Posttreatment 0.24 1.71 3.04 2.24 3.75 .01

6-Month Follow-Up 0.26 1.81 3.39 2.35 3.95 < .001

BASC Aggression

Pretreatment 7.09 68.51 73.30 68.57 73.26 < .001

Posttreatment 5.77 55.07 60.16 57.05 62.60 < .001

6-Month Follow-Up 5.80 54.64 61.33 57.24 63.95 < .001

Note: PMT¼Parent Management Training; CPS¼Collaborative & Proactive Solutions; EI¼equivalence intervals; ADIS-CSR¼Anxiety Dis- orders Interview Schedule clinician severity rating; DBDRS ODD¼Disruptive Behavior Disorders Rating Scale, oppositional defiant disorder; CGI-S¼Clinical Global Impression–Severity (higher scores indicate more severe impairment); BASC¼Behavior Assessment System for Children.

OLLENDICK ET AL.600

decreased significantly in the two active treatment con- ditions, whereas aggression did not decrease signifi- cantly in the WLC condition (PMT vs. WLC p < .001, Cohen’s d¼1.81 and CPS vs. WLC p¼ .001, Cohen’s d¼ .92). Means at each time point are displayed in Table 3. The within-group effect size (Cohen’s d) for PMT from pre- to posttreatment was 1.11, and the effect size for CPS was .75. At the 6-month follow-up for the two-group analyses, the main effect for time remained significant, F(2, 113)¼74.09, p < .001, whereas treat- ment, F(1, 145)¼0.62, p¼ .43, and Time�Treatment, F(2, 113)¼0.74, p¼ .48, effects were nonsignificant. The findings suggest that CPS and PMT maintained comparable treatment gains at the 6-month follow-up (p¼ .38; see Table 3 for means). Equivalence testing indicated that the PMT and CPS groups yielded compa- rable aggression scores at each time point (see Table 4).

Consumer satisfaction. At posttreatment, families in the PMT and CPS conditions did not differ on their satisfaction with the program, F(1, 69)¼0.45, p¼ .51. Overall, families in both PMT (M¼33.20, SD¼5.51) and CPS (M¼33.08, SD¼7.06) reported being satisfied (ranging between slightly satisfied to highly satisfied) with the program. Similarly, parents in the two treat- ment conditions (PMT: M¼32.47, SD¼7.09; CPS: M¼34.11, SD¼5.49) did not differ on their satisfaction with the program at 6-month follow-up, F(1, 36)¼2.50, p¼ .12.

Treatment Remission

Treatment remission was determined at both posttreat- ment and at 6-month follow-up on clinician-rated mea- sures: diagnostic status (CSR < 4) and a rating of 1 or 2 on the CGI-I (much or very much improved). At posttreat- ment, 48.8% of youth in the PMT condition were diagnosis free compared to 48.0% of youth in the CPS condition. Similarly, 46.3% of youth in PMT were viewed as much or very much improved following treatment compared to

46.7% of youth in CPS. These differences between treat- ments were not significant; the effects were largely main- tained at 6-month follow-up although some deterioration was noted in both groups (see Table 5).

Predictor Analyses

Presence of an AD and chronological age predicted out- comes; gender, race=ethnicity, socioeconomic status, and presence of ADHD did not.

For the presence of an AD, there was a significant effect for the AD�Time interaction for both CSRs, F(2, 200)¼3.38, p¼ .04, and CGI-S, F(2, 153)¼7.63, p¼ .001, revealing that the presence of an AD predicted better treatment outcomes for both PMT and CPS across our two clinician-rated measures. However, such interaction effects were not observed for our parent-rated measures, although trends in the same direction were observed: DBDRS, F(2, 122)¼2.44, p¼ .09, and BASC-Aggression, F(2, 110)¼2.38, p¼ .10.

For chronological age, there was a significant effect for Age�Time interaction for both clinician CSRs, F(14, 181)¼2.19, p¼ .01, and parent DBDRS ratings, F(13, 126)¼2.14, p¼ .02, revealing that older children did not improve as much as younger children across the two treatments over time on these measures.

DISCUSSION

The present study compared an evidence-based, well-established treatment (PMT; Barkley, 1997; Eyberg et al., 2008; Weisz et al., 2012) to a less researched but promising treatment (CPS; Greene, 1998, 2010) and a WLC group in a sample of children and adolescents diagnosed with ODD.

Our primary hypotheses were confirmed. First, PMT and CPS produced better outcomes than the small WLC group on all four of our treatment response outcome measures: lower CSRs on the ADIS C=P, lower severity ratings on the CGI-S, lower scores on the DBDRS, and

TABLE 5

Treatment Remission Measures by Treatment Condition at Post- and 6-Month Follow-Up

Remission Measures

PMT CPS WLC

v2 pN (%) N (%) N (%)

Diagnosis Free

Posttreatment 24 (48.8) 22 (48.0) 0 (0) 0.04 .85

6-Month Follow-Up 22 (43.9) 21 (46.0) — 0.98 .32

CGI-I–Improved

Posttreatment 23 (46.3) 21 (46.7) — 0.15 .70

6-Month Follow-Up 20 (39.4) 20 (45.1) — 3.29 .07

Note: PMT¼Parent Management Training; CPS¼Collaborative & Proactive Solutions; WLC¼waitlist control; CGI-I¼Clinical Global Impression–Improvement (rated as a 1 or 2); percentages are based on pooled estimates from imputed data at each time point.

PMT AND CPS 601

lower scores on the Aggression subscale of the BASC. None of the youth in the WLC group were rated as bet- ter following the waitlist period: All continued to meet criteria for ODD and all requested reassignment to an active treatment.

Second, consistent with our primary hypotheses, both treatments were shown to be equivalent to one another, affirming preliminary findings reported by Greene et al. (2004) and lending support to CPS as an evidence-based intervention. In addition, equivalent results were obtained for both treatments in our remission analyses: Nearly 50% of youth in both treatment conditions were diagnosis free at posttreatment and were viewed as much or very much improved by raters masked to treat- ment conditions. Treatment gains were largely main- tained at follow-up. The rates for both groups compare favorably to those obtained in other studies in youth with ODD (Fabiano et al., 2009; Kazdin & Whitley, 2006; Weisz et al., 2012).

Third, regarding predictors of treatment outcomes, we found that our treatments were more effective for younger children than older children, a finding consist- ent with prior research (cf. Fabiano et al., 2009). We also found that presence of an AD was associated with better outcomes across treatments for our ODD youth. This finding is similar to earlier findings by Walker et al. (1991) and Ollendick, Seligman, Butcher, and Div (1999) and more recent findings by Jarrett, Siddiqui, Lochman, and Qu (2014), who showed that internalizing problems like anxiety and depression can serve to miti- gate the behavioral expression of conduct problems in youth. If these findings are replicated in additional stu- dies, it will be important to study the exact mechanisms through which their beneficial effects occur (see Drabick et al., 2010).

In as much as our two treatments were equivalent and we were unable to explore moderators of change due to our small sample sizes (Kraemer, Wilson, Fair- burn, & Agras, 2002), we are not able to comment on ‘‘for whom’’ these two treatments work best. As described earlier, PMT focuses primarily on improving children’s compliance with adult directives by modifying faulty parental disciplinary practices (Kazdin, 2005; McMahon et al., 2011), whereas CPS focuses primarily on helping parents and children collaboratively and proactively solve the problems that are contributing to challenging behaviors (Greene, 1998, 2010). Given these differences in approach, it is likely that moderators exist. Although speculative, they may well include factors such as family preferences, therapist preferences, the thera- peutic alliance, and other family-therapist characteris- tics that signal the ‘‘goodness-of-fit’’ between therapists, treatments, and families. These variables will be the focus of further, more refined analyses with this data set and in subsequent studies. In terms of other

future directions, exploring mediators of treatment out- comes may prove fruitful as the mechanisms through which gains are conferred will likely differ between the treatment conditions.

The current study possesses both strengths and weak- nesses. The strengths are related to randomization of the sample to the two treatment conditions, use of psycho- metrically sound assessment measures, thorough opera- tionalization of the treatments via treatment manuals, and carefully trained assessors and therapists, as well as supervision of these assessors and therapists by experts in the two treatments and analysis of the intent-to-treat data via mixed models analyses and equivalence testing.

Weaknesses are also present. The number of parti- cipants in our WLC condition was very small (n¼11), and participants were randomized to this condition only during the first year of this 5-year clinical trial. As noted, the decision was made to drop the WLC condition because none of the 11 families improved during the wait period. For clinical and ethical reasons, we discon- tinued randomization to this condition. Still, it is impor- tant to note that these families did not differ on our main outcome variables at pretreatment and that all WLC families sought and accepted randomization to one of the active treatments. We are cognizant of the shortcomings of this decision but believe it was clinically responsive and ethically defensible.

A second major weakness is related to the number of families who dropped out of treatment and=or failed to return for assessment at posttreatment and 6-month follow-up. As noted, these families also differed from families who completed treatment (less education) and who were available for posttreatment assessment and follow-up assessment (less education, lower income, and single-parent family status). Although the failure of families with these characteristics to return for assess- ment is a significant shortcoming, we did employ mixed-models analyses, which also use maximum likeli- hood to address missing data. As noted by Salim et al. (2008) and others (Young, Weckman, & Holland, 2011), this approach is generally acceptable even when dropout rates are ‘‘substantial’’ as they were in this study. Furthermore, the current guidelines of the Conso- lidated Standards of Reporting Trials (CONSORT) rec- ommend use of data imputation and mixed models analyses when outcome data are missing (Moher et al., 2010).

A third major weakness is a lack of established treat- ment adherence and competency measures. Here, we reported only on treatment adherence as determined by the supervisors who provided weekly 75-min super- visory sessions to our clinicians. Such supervisory ses- sions included review of ongoing videotapes and careful monitoring to ensure that PMT and CPS were

OLLENDICK ET AL.602

implemented as specified in their respective conditions and that, conversely, elements of these treatments were not used in the alternative treatment. Ratings by the supervisors indicated that treatment was delivered as intended and that little to no crossover in the thera- peutic strategies was evident. Still, we did not obtain measures of how competently the clinicians implemen- ted our treatments. We are presently obtaining such competency ratings as well as ratings of the therapeutic alliance for our two treatment conditions.

Other weaknesses include our sample of largely middle-class Caucasian families and conducting the study in a university setting with carefully trained and supervised clinicians (which may not be reflective of other community samples and treatment conditions). Given the characteris- tics of the sample, particularly in the CPS group, there are limitations to generalizability. The efficacy of CPS and its equivalence to PMT may be only in educated Caucasian samples, and further research is needed to implement and evaluate CPS in more racial=ethnic and socioeconomi- cally diverse samples. We also lack longer term follow-up data on the effects of our intervention. In addition, not all of our youth had a primary diagnosis of ODD. Still, ODD was the principal reason for referral for all youth, and all youth did have a diagnosis of ODD as one of their top three diagnoses in this highly comorbid sample. However, we suggest that not limiting the sample to a primary diag- nosis of ODD may better reflect ‘‘real-world’’ applicability of these treatments.

Clinical Significance of Findings

In this study, CPS was shown to be equivalent to PMT, and both treatments evidenced large effect sizes in com- parison to the WLC and over time within each treatment. This was shown to be the case with youth varying in chronological age, gender, receptive and expressive verbal ability, and presence of co-occurring ADHD and AD. As such, CPS may be a useful, evidence-based option for fam- ilies seeking alternative and=or additional interventions. Given some of the limitations of PMT described earlier in this article, the existence of a comparably efficacious but different psychosocial treatment is a positive develop- ment in the treatment of youth with ODD and their fam- ilies. Both patient and therapist preferences might be realized with such equivalent treatments; however, such possibilities await further study and evaluation.

FUNDING

Funding was provided by R01 MH59308 from NIMH and by the Institute for Society, Culture, and Environ- ment at Virginia Tech. We express appreciation to

graduate student colleagues and research scientists who assisted us with various aspects of this project, including assistance with data reduction, assessment, and treatment of the youth: Kaushal Amatya, Scott Anderson, Jordan Booker, Lisa Buonomono, Natalie Costa, and Marshaun Glover. We also extend thanks to the many undergraduate students at Virginia Tech who assisted with data coding, entry, and verification. Finally, we are grateful to the youth and families who participated in this clinical research trial.

REFERENCES

American Psychiatric Association. (1994). Diagnostic and statistical

manual of mental disorders (4th ed.). Washington, DC: Author.

American Psychiatric Association. (2013). Diagnostic and statistical

manual of mental disorders (5th ed.). Washington, DC: Author.

Anderson, S. R., & Ollendick, T. H. (2012). Diagnosing oppositional

defiant disorder using the Anxiety Disorders Interview Schedule

for DSM-IV: Parent version and the diagnostic interview schedule

for children. Journal of Psychopathology and Behavioral Assessment,

34, 467–475.

Barkley, R. A. (1997). Defiant children: A clinician’s manual for parent

training (2nd ed.). New York, NY: Guilford Press.

Biederman, J., Faraone, S. V., Milberger, S., Jetton, J. G., Chen, L.,

Mick, E., . . . Russell, R. L. (1996). Is childhood oppositional defiant

disorder a precursor to adolescent conduct disorder? Findings from

a four-year follow-up study of children with ADHD. Journal of the

American Academy of Child & Adolescent Psychiatry, 35, 1193–1204.

Boylan, K., Vaillancourt, T., Boyle, M., & Szatmari, P. (2007). Comor-

bidity of internalizing disorders in children with oppositional defiant

disorder. European Child & Adolescent Psychiatry, 16, 484–494.

Brestan, E. V., & Eyberg, S. M. (1998). Effective psychosocial treat-

ments of conduct-disordered children and adolescents: 29 years, 82

studies, and 5,272 kids. Journal of Clinical Child Psychology, 27,

180–189.

Cunningham, N. R., & Ollendick, T. H. (2010). Comorbidity of anxi-

ety and conduct problems in children: Implications for clinical

research and practice. Clinical Child and Family Psychology Review,

13, 333–347.

Drabick, D. A. G., Ollendick, T. H., & Bubier, J. L. (2010).

Co-occurrence of ODD and anxiety: Shared risk processes and evi-

dence for a dual-pathway model. Clinical Psychology: Science and

Practice, 17, 307–318.

Dunn, L. M., & Dunn, D. M. (2007). Peabody Picture Vocabulary

Test: Manual (4th ed.). Minneapolis, MN: Pearson Assessments.

Eyberg, S. M., Nelson, M. M., & Boggs, S. R. (2008). Evidence-based

psychosocial treatments for children and adolescents with disruptive

behavior. Journal of Clinical Child & Adolescent Psychology, 37,

215–237.

Fabiano, G. A., Pelham, W. E., Jr., Coles, E. K., Gnagy, E. M.,

Chronis-Tuscano, A., & O’Connor, B. C. (2009). A meta-analysis

of behavioral treatments for attention-deficit=hyperactivity disorder.

Clinical Psychology Review, 29, 129–140.

Frick, P. J. (2001). A comprehensive and individualized treatment

approach for children and adolescents with conduct disorders.

Cognitive and Behavioral Practice, 7, 30–37.

Greene, R. W. (1998). The explosive child: A new approach for

understanding and parenting easily frustrated, ‘‘chronically inflexible’’

children. New York, NY: HarperCollins.

PMT AND CPS 603

Greene, R. W. (2010). The aggressive, explosive child. In B.

Zuckerman, E. Caronna, & M. Augustyn (Eds.), Zuckerman and

parker’s handbook of developmental and behavioral pediatrics (2nd

ed.). Baltimore, MD: Lippincott, Williams, & Wilkins.

Greene, R. W., Ablon, J. S., Goring, J. C., Raezer-Blakely, L.,

Markey, J., Monuteaux, M. C., . . . Rabbitt, S. (2004). Effectiveness

of collaborative problem solving in affectively dysregulated children

with oppositional-defiant disorder: Initial findings. Journal of

Consulting and Clinical Psychology, 72, 1157–1164.

Greene, R. W., Biederman, J., Zerwas, S., Monuteaux, M. C., Goring,

J. C., & Faraone, S. V. (2002). Psychiatric comorbidity, family dys-

function, and social impairment in referred youth with oppositional

defiant disorder. American Journal of Psychiatry, 159, 1214–1224.

Greene, R. W., & Doyle, A. E. (1999). Toward a transactional conceptua-

lization of oppositional defiant disorder: Implications for assessment and

treatment. Clinical Child and Family Psychology Review, 2, 129–148.

Grills, A. E., & Ollendick, T. H. (2003). Multiple informant agreement

and the Anxiety Disorders Interview Schedule for parents and

children. Journal of the American Academy of Child & Adolescent

Psychiatry, 42, 30–40.

Guy, W. (1976). ECDEU assessment manual for psycho-pharmacology.

Rockville, MD: U.S. National Institute of Health, PsychoPharma-

cology Research Branch.

Hinshaw, S. P., Lahey, B. B., & Hart, E. L. (1993). Issues of taxonomy

and comorbidity in the development of conduct disorder. Develop-

ment and Psychopathology, 5, 31–49.

Jarrett, M., Siddiqui, S., Lochman, J., & Qu, L. (2014). Internalizing

problems as a predictor of change in externalizing problems in

at-risk youth. Journal of Clinical Child & Adolescent Psychology,

43, 27–35.

Jarrett, M. A., Wolff, J. C., & Ollendick, T. H. (2007). Concurrent

validity and informant agreement of the ADHD module of the

Anxiety Disorders Interview Schedule for DSM-IV. Journal of Psy-

chopathology and Behavioral Assessment, 29, 159–168.

Kamphaus, R. W., & Frick, P. J. (2005). Clinical assessment of child

and adolescent personality and behavior (2nd ed.). New York, NY:

Springer.

Kazdin, A. E. (2005). Parent management training: Treatment for

oppositional, aggressive, and antisocial behavior in children and

adolescents. New York, NY: Oxford University Press.

Kazdin, A. E., & Whitley, M. K. (2006). Comorbidity, case com-

plexity, and effects of evidence-based treatment for children referred

for disruptive behavior. Journal of Consulting and Clinical

Psychology, 74, 455–467.

Kimonis, E. R., & Frick, P. J. (2010). Oppositional defiant disorder

and conduct disorder grown-up. Journal of Developmental &

Behavioral Pediatrics, 31, 244–254.

Kraemer, H. C., Wilson, G. T., Fairburn, C. G., & Agras, W. S.

(2002). Mediators and moderators of treatment effects in

randomized clinical trials. Archives of General Psychiatry, 59,

877–883.

McMahon, R. J., Long, N., & Forehand, R. L. (2011). Parent train-

ing for the treatment of oppositional behavior in young children:

Helping the noncompliant child. In R. C. Murrihy, A. D. Kidman,

& T. H. Ollendick (Eds.), Clinical handbook of assessing and treat-

ing conduct problems in youth (pp. 163–191). New York, NY:

Springer.

Michelson, D., Davenport, C., Dretzke, J., Barlow, J., & Day, C.

(2013). Do evidence-based interventions work when tested in the

‘‘Real World?’’ A systematic review and meta-analysis of parent

management training for the treatment of child disruptive behavior.

Clinical Child and Family Psychology Review, 16, 18–34.

Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C.,

Devereaux, P. J., . . . Altman, D. G. (2010). CONSORT 2010 expla-

nation and elaboration: Updated guidelines for reporting parallel

group randomised trials. Journal of Clinical Epidemiology, 63, e1–e37.

Murrihy, R. C., Kidman, A. D., & Ollendick, T. H. (Eds.). (2010).

Clinical handbook of assessing and treating conduct problems in

youth. New York, NY: Springer.

Ollendick, T. H., & Cerny, J. A. (1981). Clinical behavior therapy with

children. New York, NY: Plenum Press.

Ollendick, T. H., Seligman, L. D., Butcher, A. T., & Div, M. (1999).

Does anxiety mitigate the behavioral expression of severe conduct

disorder in delinquent youths? Journal of Anxiety Disorders, 13,

565–574.

Pelham, W. E., Gnagy, E. M., Greenslade, K. E., & Milich, R. (1992).

Teacher ratings of DSM-III-R symptoms for the disruptive behavior

disorders. Journal of the American Academy of Child & Adolescent

Psychiatry, 31, 210–218.

Reynolds, C. R., & Kamphaus, R. W. (1992). BASC: Behavior assess-

ment system for children: Manual. Circle Pines, MN: American

Guidance Service.

Rogers, J. L., Howard, K. I., & Vessey, J. T. (1993). Using significance

tests to evaluate equivalence between two experimental groups.

Psychological Bulletin, 113, 553–565.

Salim, A., Mackinnon, A., Christensen, H., & Griffiths, K. (2008).

Comparison of data analysis strategies for intent-to-treat analysis

in pre-test–post-test designs with substantial dropout rates.

Psychiatry Research, 160, 335–345.

Silverman, W. K., & Albano, A. M. (1996). The Anxiety Disorders

Interview Schedule for children for DSM-IV: Child and parent ver-

sions. San Antonio, TX: Psychological Corporation.

Silverman, W. K., Saavedra, L. M., & Pina, A. A. (2001). Test-retest

reliability of anxiety symptoms and diagnoses with the Anxiety Dis-

orders Interview Schedule for DSM-IV: Child and parent versions.

Journal of the American Academy of Child & Adolescent Psychiatry,

40, 937–944.

Walker, J. L., Lahey, B. B., Russo, M. F., Frick, P. J., Christ, M. A.

G., McBurnett, K., . . . Green, S. M. (1991). Anxiety, inhibition, and conduct disorder in children: I. Relations to social impairment.

Journal of the American Academy of Child & Adolescent Psychiatry,

30, 187–191.

Weisz, J. R., Chorpita, B. F., Palinkas, L. A., Schoenwald, S. K.,

Miranda, J., Bearman, S. K., . . . Gibbons, R. D. (2012). Testing stan-

dard and modular designs for psychotherapy treating depression,

anxiety, and conduct problems in youth: A randomized effectiveness

trial. Archives of General Psychiatry, 69, 274–282.

Williams, K. T. (2007). The Expressive Vocabulary Test (2nd ed.). Cir-

cle Pines, MN: AGS Publishing.

Wolff, J. C., & Ollendick, T. H. (2010). Conduct problems in youth:

Phenomenology, classification, and epidemiology. In R. C.

Murrihy, A. D. Kidman, & T. H. Ollendick (Eds.), Clinical

handbook of assessing and treating conduct problems in youth

(pp. 3–20). New York, NY: Springer.

Young, W., Weckman, G., & Holland, W. (2011). A survey of meth-

odologies for the treatment of missing values within datasets: Lim-

itations and benefits. Theoretical Issues in Ergonomics Science, 12,

15–43.

OLLENDICK ET AL.604

Copyright of Journal of Clinical Child & Adolescent Psychology is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.