Single Study on Collaborative and Proactive Solutions

Age 4 Predictors of Oppositional Defiant Disorder in Early Grammar School

John V. Lavigne Department of Child and Adolescent Psychiatry, Ann & Robert H. Lurie Children’s Hospital of Chicago, Feinberg School of Medicine, Northwestern University and Mary Ann and J. Milburn

Smith Child Health Research Program, Children’s Hospital of Chicago Research Center

Fred B. Bryant Department of Psychology, Loyola University of Chicago

Joyce Hopkins College of Psychology, Illinois Institute of Technology

Karen R. Gouze Department of Child and Adolescent Psychiatry, Ann & Robert H. Lurie Children’s Hospital of

Chicago and Feinberg School of Medicine, Northwestern University

Our ability to predict which children will exhibit oppositional defiant disorder (ODD) at the time of entry into grammar school at age 6 lags behind our understanding of the risk factors for ODD. This study examined how well a set of multidomain risk factors for ODD assessed in 4-year-old children predicted age 6 ODD diagnostic status. Participants were a diverse sample of 796 4-year-old children (391 boys).The sample was 54% White, non-Hispanic; 16.8% African American, 20.4% Hispanic; 2.4% Asian; and 4.4% Other or mixed race. The classification accuracy of two models of multidomain risk factors, using either a measure of overall ODD symptoms or dimensions of ODD obtained at age 4, were compared to one another, to chance, and to a parsimonious model based solely on parent-reported ODD using Automated Classification Tree Analysis. Effect Strength for Sensitivity (ESS), a measure of classification accuracy, indicated a multidomain model including a general measure of ODD symptoms at age 4 yielded a large effect (56.29%), a 13.7% increase over the ESS for the parsimonious model (ESS = 42.9%). The ESS (51.23%) for a model including two ODD dimensions (behavior and negative affect) was smaller than that for the model including a measure of overall ODD symptoms. The Classification Tree Analysis approach showed a small but distinct advantage that would be useful in screening for which children would most likely meet criteria for age 6 ODD.

Symptoms of difficult-to-manage, oppositional behavior are common in preschoolers (Egger & Angold, 2006). Studies

suggest that these symptoms are reasonably stable from preschool to the early school years (Campbell, 1990; Lavigne et al., 1998). Nonetheless, a significant number of oppositional preschoolers may not exhibit these symptoms by early grammar school (Lavigne et al., 1998). Campbell (1995) estimated that approximately half of difficult-to-man- age children will continue to have high levels of symptoms of disruptive behavior 3–7 years later. Lavigne et al. (1998) report that children with early indications of a disruptive

Correspondence should be addressed to John V. Lavigne, Department of Child and Adolescent Psychiatry (#10), Ann & Robert H. Lurie Children’s Hospital of Chicago, 225 East Chicago Avenue, Chicago, IL 60611. E-mail: [email protected]

Supplemental data for this article can be accessed on the publisher’s website.

Journal of Clinical Child & Adolescent Psychology, 48(1), 93–107, 2019 Copyright © Society of Clinical Child & Adolescent Psychology ISSN: 1537-4416 print/1537-4424 online DOI: https://doi.org/10.1080/15374416.2017.1280806

disorder are 47%–50% more likely to have such a disorder in early grammar school. Overall, this pattern of results suggests that, although about 50% of children with the disorder in preschool continue to exhibit high levels of symptoms of oppositional defiant disorder (ODD) in early grammar school, for others ODD symptoms decline over time.

Oppositional behavior in early grammar school can be highly problematic. Studies have identified (a) a life-course pattern of homotypic continuity in which a subset of young oppositional children later develop conduct disorder (Moffitt, 1993), (b) a pattern of heterotypic continuity in which oppositional preschoolers later develop anxiety or depression (Burke & Loeber, 2010; Drabick, Ollendick, & Bubier, 2010; Lavigne et al., 2001), and (c) preschool children with ODD exhibit problems with emotional regula- tion (Lavigne, Gouze, Hopkins, Bryant, & LeBailly, 2012) linked to problems with academic achievement (McClelland et al., 2007; Ponitz, McClelland, Matthews, & Morrison, 2009).

Because ODD in early childhood may have deleterious long-term effects, early intervention could be critical, parti- cularly because empirically supported interventions for ODD are available (Barlow, Bergman, Konner, Wei, & Bennett, 2016). Because ODD prevalence is high in young children—approximately 8%–10% when impairment is included in the diagnosis (Lavigne, LeBailly, Hopkins, Gouze, & Binns, 2009), mustering the treatment resources to address the need is potentially problematic. As a result, from a population health perspective, finding ways to iden- tify young children most in need of treatment may be important. Such concerns have prompted pediatricians to recommend screening for behavioral problems in primary care (Foy, Kelleher, & Laraque, 2010). Because symptoms of ODD remit in some children and not others before grammar school entry, determining which children are most likely to have persistent problems with ODD, or those whose subthreshold ODD symptoms might become more problematic, is an important issue.

Our understanding of the factors associated with the onset, stability, and course of ODD is increasing, and the knowledge gained might help identify children needing a targeted intervention to reduce the likelihood of an ODD diagnosis in early grammar school. The large number of risk factors associated with ODD symptoms have been described elsewhere (Lavigne et al., 2012). In general, the association of these risk factors with ODD has been examined in small- scale studies of individual or small sets of risk factors at a time. Few studies have integrated those risk factors into an overall model including risk factors from multiple domains while specifying the pathways through which such risk factors are associated with ODD symptom levels. One such model has been reported, linking multidomain factors associated with the stability of ODD syptoms in both cross- sectional (Lavigne et al., 2012) and longitudinal (Lavigne,

Gouze, Hopkins, & Bryant, 2015) models. Risk factors in the multidomain model include (a) at the contextual level, lower socioeconomic status (SES; Evans, 2004), parental stress (McMahon, Grant, Compas, Thurm, & Ey, 2003), and family conflict (Grant et al., 2006); (b) at the parent level, parental depression (Goodman et al., 2011); (c) at the parenting level, poor parenting (Hipwell, Keenan, Kasza, Stouthamer-Loeber, & Bean, 2008); and (d) at the child level, less secure attachment (DeVito & Hopkins, 2001) and temperament variables of negative affect, poor effortful control (Eisenberg et al., 2009), and poor sensory regulation (Gouze, Hopkins, LeBailly, & Lavigne, 2009). Finally, the best predictor of meeting criteria for a diagnosis at a future date may be indications that symptoms of that disorder were present previously (Lavigne, Gouze, et al., 2015). In addi- tion, because symptoms of psychopathology often are cor- related and comorbidity among disorders is often high (Angold, Costello, & Erkanli, 1999), symptoms of ODD, or ODD symptoms combined with other types of symptoms, may be strong predictors of who meets criteria for an ODD diagnosis in the future.

As studies of risk factors associated with ODD symp- toms have accrued, our understanding of the structure of ODD has changed. It is now recognized that ODD is not a unitary construct; rather, there are multiple dimensions to ODD. Presently, the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5; American Psychiatric Association, 2013) recognizes a three-dimensional structure of ODD, with dimensions of angry/irritable mood, argumen- tative/defiant behavior, and vindictiveness. A recent study comparing various proposed structures suggested that a two- factor structure (ODD behavior, similar to the DSM-5 argu- mentative/defiant factor, and negative affect, similar to the DSM-5 dimension of angry/irritable mood) may more appropriately characterize the structure of ODD symptoms in young children (Lavigne, Bryant, Hopkins, & Gouze, 2015). Studies have examined whether specific dimensions are better predictors of subsequent disorders (Lavigne, Gouze, Bryant, & Hopkins, 2014; Stringaris & Goodman, 2009), and specific dimensions may be better predictors of subsequent ODD diagnosis than overall disorder alone.

A major limitation to developing practical applications for the growing information about the risk factors and structure for ODD is that the results of inferential statistical approaches used to establish associations between predictors and outcomes (e.g., correlations, effect sizes, regression coefficients) do not readily translate into a form useful for identifying individual children at risk for disorder. Clinicians often find research results to be of limited clinical utility (Garland, Hurlburt, & Hawley, 2006), but they may find that results identifying a specific percentage of children with a particular set of attributes (i.e., likely to benefit from treatment, develop a disorder, etc.) are more useful than traditional presentations of statistical analyses. An alterna- tive approach is needed that can (a) identify cutoff scores

94 LAVIGNE, BRYANT, HOPKINS, GOUZE

useful for predicting who will later have a disorder and (b) examine how multiple risk factors might best be combined to predict future disorder. The purpose of the present study is to determine how well multidomain risk factors for ODD assessed in a preschool or primary care pediatric setting can be used to identify which children are either likely or unlikely to meet criteria for an ODD diagnosis at age 6, when most children are enrolled in, or about to enter, Grade 1. The classification accuracy of two models for predicting which children would meet criteria for an ODD disorder at age 6 were compared to one another, to chance, and to classification accuracy based solely on the presence of ODD symptoms at age 4. These two classification models included multidomain risk factors and either (a) a measure of overall ODD symptoms or (b) measures of dimensions of ODD.

METHOD

Participants

This report is part of a longitudinal study of factors related to symptoms of ODD in young children (Lavigne, Gouze, et al., 2015). Children and their parents were followed from 4 to 6 years of age. For this report, predictor measures were completed when the child was 4, and the diagnosis of ODD was based on assessment at age 6, when most children were in Grade 1. Children (N = 796) and their parents were recruited from 13 Chicago Public School preschool pro- grams and 23 primary care pediatric clinics in Cook County, Illinois. All children included in the study were (a) age 4 at enrolment, (b) English speaking or Spanish speaking, (c) living with the parent(s) for the previous 6 months or longer, (d) obtained a standard score on the Peabody Picture Vocabulary Test of 70 or higher (Dunn & Dunn, 1997), and (e) did not meet criteria for autism spec- trum disorder.

During recruitment, 1,738 families expressed interest in the study and 827 (47.5%) agreed to participate. Of those families, 31 were ineligible, resulting in a final sample of 796 children (183 were recruited through the schools and 613 from pediatric practices). The mean age was 4.44 years, and 391 (49.1%) were boys. Members of all social classes (Hollingshead, 1975) participated, with 303 (38.1%) chil- dren in the highest class (Class I), 290 (36.4%) in Class II, 79 (9.9%) in Class III, 63 (7.9%) in Class IV, and 61 (7.7%) in Class V. We sought a racially/ethnically diverse sample similar to the diversity of Cook County, Illinois, where the sample was recruited. Parent-reported racial/ethnicity for the sample was 433 (54.4%) White, non-Hispanic; 162 (20.4%) Hispanic; 133 (16.7%) African American; 19 (2.4%) Asian; and 35 (4.4%) self-described as multiracial or other. Race/ ethnicity was not reported by 14 (1.8%) parents. For 31

families (3.9%), a primary caretaker father was the partici- pating parent.

A total of 626 children and families (78.6%) participated in all waves of data collection. Families completing the first and third waves differed from those who did not with respect to (a) being lower SES, χ2(4, N = 796) = 69.61, p = .001; (b) race, with a greater proportion of dropouts among minority partici- pants, χ2(5, N = 796) = 77.7, p = .001; and (c) younger children, 25 days older at study entry, t(773) = 2.41, p = .02. Because listwise deletion is more likely to introduce biases than multiple imputation procedures (Graham, 2009), missing data were multiply imputed (see the upcoming Data Analysis section). The final total sample size was 796.

Measures

A multi-informant approach to measurement included obser- ver ratings for risk factors of child attachment and parent scaffolding, parent reports and interviews to assess outcomes and specific risk factors, and performance-based measures of inhibitory control and child verbal skills. A parent-reported diagnosis based on a structured interview at age 6 was used because (a) parent report typically forms the basis of clinical evaluations, so a structured parent interview was clinically relevant; (b) there are no structured child-reported interviews for 6-year-olds; (c) significant evidence has emerged indicat- ing that teacher reports are substantially different from parent reports and that parent and teacher symptom reports cannot serve as proxies for one another (Drabick, Gadow, & Loney, 2007; Lavigne, Dahl, Gouze, LeBailly, & Hopkins, 2015); and (d) there are no suitable teacher interview or other observer- based approaches for assigning a DSM diagnosis for young children.

Because the primary study (Lavigne, Gouze, et al., 2015) was designed to examine mediating processes, multiple measures were obtained to estimate latent factors. In the present report, individual scales useful in clinical settings were used to measure study variables. There was one excep- tion: A composite measure of family conflict based on multiple scales was used because the internal consistency for that composite was better than that for any individual scale. Internal reliability coefficients reported were obtained at age 4. Additional information on reliability and validity are available in an online supplement.

Contextual Measures

SES and Demographics

Parent-reported demographic information was obtained on the child’s race, age, sex, parent’s education, and parent’s employment status. SES was coded using the Hollingshead Four-Factor Index of Social Status (Hollingshead, 1975).

ODD PREDICTORS 95

Family Conflict

The composite measure of parent-reported family conflict (coefficient α = .71) included the following: (a) Family Environment Scale Conflict scale (Moos & Moos, 1986): Sample items include “Family members criticize each other often” and “Fight a lot in family”; (b) McCubbin Family Distress Index (H. I. McCubbin, Thompson, & Elver, 1996): Sample items include “Increase in conflict between adults in the house” and “Increased disagreement about a member’s friends or activities”; and (c) Family Problem Solving/ Communication scale (M. A. McCubbin, McCubbin, & Thompson, 1996). Sample items include “We talk things through till we reach a solution” and “We make matters more difficult by fighting and bringing up old matters.”

Life Stress

The Perceived Stress Scale (Cohen, Kamarck, & Mermelstein, 1983) is a 14-item measure in which respondents describe the degree to which their lives appear be uncontrol- lable and overwhelming. Sample items include “In the last month, how often have you been upset because of something that happened unexpectedly?” and “In the last month, how often have you been upset because of something that happened unexpectedly?” Coefficient alpha was .87.

Parental Depression

The Beck Depression Inventory (Beck, Ward, Mendelson, Mock, & Erbaugh, 1961) is a widely used 21-item self-report measure of symptoms of adult depression. Sample items include “Suicidal thoughts or wishes,” “Self-dislike,” and “Guilty feelings.” In this study, coefficient alpha was .86.

Parenting Measures

Parent Support and Hostility

The Parent Behavior Inventory (Lovejoy, Weis, O’Hare, & Rubin, 1999) is a parent-report measure of parenting behavior with preschool and school-age children. The Parent Behavior Inventory has two factor-analytically- derived subscales—Support/Engagement and Hostility/ Coercion. Sample items for Support/Engagement include “Pleasant conversations with child” and “Listen and try to understand child’s feelings.” Sample items for Hostility/ Coercion are “Lose temper when child doesn’t do some- thing requested” and “Demand child does something right away.” Coefficient alpha was .85 for Support and .73 for Coercion.

Scaffolding

A 15-min videotaped, semistructured parent–child inter- action procedure, the National Institute of Child Health and Human Development (NICHD) Three Boxes Paradigm

(NICHD Early Childhood Research Network, 1999), was used to assess scaffolding. Scaffolding refers to the skill with which parents assist their child in managing frustration and developing skills needed to succeed on a given task (Baker, Fenning, Crnic, Baker, & Blacher, 2007). The task includes (a) two activities designed to be too difficult for the child to accomplish without assistance and (b) a free-play activity. Parental scaffolding was a composite score of rat- ings by trained research assistants on 7-point qualitative scales for supportive presence, respect for autonomy, quality of assistance, cognitive stimulation, confidence, and hosti- lity. Interrater reliability for these scales was good to excel- lent (NICHD Early Childhood Research Network, 1999), and composite ratings of the scales significantly predict children’s attachment security and social competencies (NICHD Early Childhood Research Network, 1999). Study coders were trained to a criterion of 80% reliability com- pared with two master coders trained by a doctoral-level NICHD study member. A random sample of 20% of the tapes was double-coded to assess interrater reliability. Reliabilities (intraclass correlation coefficients) in the pre- sent study ranged from .80 for quality of assistance to .69 (likely deflated by a low base rate) for maternal hostility, with a mean reliability of .74. To create a composite mea- sure of parenting behavior, the six parent ratings were factor-analyzed using a maximum likelihood method. A one-factor solution provided the best fit to the data. As a result, a composite measure of scaffolding was calculated by summing the ratings for supportive presence, respect for autonomy, cognitive stimulation, quality of assistance, and confidence, then subtracting the hostility rating. The mean interrater reliability score calculated using intraclass correla- tion was .74.

Child Factors

Child Negative Affect and Effortful Control

Rothbart, Ahadi, Hershy, and Fisher’s (2001) Children’s Behavior Questionnaire (CBQ) is a widely used parent- report measure of temperament yielding a measure of nega- tive affect (NA; α = .62) and effortful control (EC; α = .72). Sample items for NA are “Is very frightened by nightmares” and “Tends to feel down at the end of the day.” Sample items for EC include “Can lower his voice when asked to do so” and “Has an easy time leaving play to come to dinner.”

Child Sensory Regulation Ability

A 38-item parent-report questionnaire, the Short Sensory Profile (study α = .81) yielded a single total score of sensory regulation (SR) ability (McIntosh, Miller, Shyu, & Hagerman, 1999). Higher scores reflect better SR ability. Sample items include “Avoids going barefoot, especially in sand or grass” and “Becomes anxious or distressed when feel leave the ground.”

96 LAVIGNE, BRYANT, HOPKINS, GOUZE

Attachment

The Attachment Q-Sort (Waters, 1987) is a continuous measure of attachment security exhibiting good convergent validity with the Strange Situation Paradigm (van Ijzendoorn, Vereijken, Bakermans-Kranenburg, & Riksen- Walraven, 2004) . A research assistant observed the mother and child during the home visit for 120 min and then completed the Q-Sort. A second observer rated a 20% ran- dom sample of visits. Interrater reliability was .77.

Receptive Language

The Peabody Picture Vocabulary Test (PPVT; α = .94; Dunn & Dunn, 1997), a measure of single-word receptive language, was used as a measure of receptive language skill. Supporting the validity of the PPVT as a measure of lan- guage ability, the PPVT correlates .85 with the Stanford- Binet Intelligence test (Ollendick, Finch, & Ginn, 1974).

Child Inhibitory Control

The Statue subtest from the Developmental Neuropsy- chological Assessment (Korkman, Kirk, & Kemp, 1997; study α = .91) assesses the child’s ability to inhibit prepotent responses to a stimulus. The child is asked to maintain a specific body position, with closed eyes, for 75 s while an assistant makes sounds to elicit the child’s visual attention. Both the Statue task and the EC measure, in part, assess the child’s ability to inhibit certain responses. The tasks are considered different because the Statue task assesses inhibitory control on “cool” tasks, whereas the CBQ EC subscale assesses inhibition of negative emotions (sample CBQ items are able to resist temptations, approaches dangerous places cautiously; Zelazo & Carlson, 2012). Statue provides a single indicator of inhibitory control.

Child Psychopathology

The Child Symptom Inventory (CSI; Gadow & Sprafkin, 1997) is a parent-reported, DSM-IV-keyed (American Psychiatric Association, 1994) problem behavior checklist with symptom occurrence rated from 0 (never) to 3 (very often). Parent-reported scale scores at age 4 from the follow- ing scales were used: ODD (study α = .86), Depression (study α = .68), Generalized Anxiety (GAD; study α = .70), Separation Anxiety (study α = .77), ADHD Inattention (study α = .88), and ADHD Hyperactivity- Impulsivity (study α = .88). Typically, scores for each item on a scale are summed together and converted to T scores. DSM-5, however, considers a symptom to be present if it occurs “often” or “very often.” To increase the generaliz- ability of results, for each item, scores of 0 (never) or 1 (sometimes) were scored as zero (i.e., symptom not present) and scores of 2 (often) or 3 (very often) were scored as 1 (i.e., symptom present). Scores were summed to create the scale total score.

The DSM-5 (American Psychiatric Association, 2013) recognizes that there are different dimensions of the ODD diagnostic construct. Prior studies have shown that a two- factor structure (Burke, Loeber, Lahey, & Rathouz, 2005) is preferable to the three-factor structure of ODD incorporated into DSM-5 for young children. In the present study, a two- factor structure that was invariant for age and gender in young children (Lavigne, Bryant, et al., 2015) was used in analyses involving ODD dimensions. The two factors are ODD–Behavior (ODD-B, loading on items for argues, defies, and temper tantrums) and ODD–Negative Affect (ODD-NA, loading on items touchy, angry, spiteful/vindic- tive). For the CSI ODD–B dimension, alpha was .82 at age 4; for NA, alpha was .76 at age 4. Sample CSI items include “Loses temper” for ODD-B and “touchy” for ODD-NA, “Is sad for most of the day (depression),” “Worries more than other children (generalized anxiety),” “Worries about being left at home alone or with a sitter (separation anxiety),” “Has difficulty organizing tasks or activities (ADHD-inat- tentive),” and “Is on the go or acts as if driven by a motor (ADHD-hyperactivity).”

The Diagnostic Interview Schedule for Children– Parent Scale—Young Child Version (DISC-YC)

The DISC-YC (Fisher & Lucas, 2006) is a developmentally appropriate adaptation of the DISC-P, a DSM-IV-based (American Psychiatric Association, 1994) structured parent interview that yields information about the presence or absence of a psychiatric disorder, and the type of diagnosis. The DISC- YC is a “fully structured” interview (Costello, Egger, & Angold, 2005) in which parents answer “yes” or “no” to questions about symptoms, contingent questions assess details about the symptom, and the computer program skips to the next question contingent upon parental response. Unlike semi- structured, interviewer-based procedures, the role of clinical judgment in the DISC-YC is essentially eliminated. As a result, the interview can be conducted by trained research assistants without extensive clinical experience. The research assistants administering the DISC-YC were either clinical psychology graduate students or, in one instance, a B.A.- level graduate applying to graduate school. Trained by DISC-trained staff, the interviewers practiced conducting interviews until they were able to reproduce the skip patterns used by the DISC-trained interviewers. This follows proce- dures in Shaffer, Fisher, Lucas, Dulcan, and Schwab-Stone (2000). Children meeting criteria for an ODD diagnosis at age 6 exhibited Impairment Level A, defined by the test developers as showing an intermediate or severe level of impairment on at least one symptom.

Procedure

Parents of 4-year-olds were approached by research assis- tants in pediatric offices and at drop-off and pickup times at

ODD PREDICTORS 97

preschools. Subsequently, a home visit was arranged with interested families, who were contacted by telephone, and study measures were completed at the home visit. Parents were recontacted for another visit when their children were 6 years of age, and study measures were readministered. This study was approved by the authors’ Institutional Review Boards; written consent was obtained. Families were paid for participation to compensate for time spent in data collection.

Statistical Analysis

Univariate optimal data analysis (uniODA; Yarnold & Soltysik, 2005) was used to identify optimal cutoff scores for age 4 predictor variables, using an exact permutation probability requiring no distributional assumptions. Each predictor variable is analyzed individually (hence, “uniODA” analyses) to assess accuracy in predicting the target variable (i.e., presence or absence of ODD diagnosis at age 6). After determining the value, statistical signifi- cance, and effect strength of the optimal cut-score on the predictor that maximizes its predictive accuracy, the expected cross-sample stability of the ODA model is assessed. In this process, “jack-knife” leave-one-out (LOO) analyses are conducted to determine the likelihood of repli- cating the analysis of each predictor in an independent sample. In the LOO analysis, each observation is removed one at a time from the sample, the uniODA analysis is repeated using the remaining observations, the resulting ODA model is used to classify the single held-out observa- tion, and overall LOO classification accuracy is computed across all of the held-out observations. Results are consid- ered LOO stable for the given predictor variable if the overall classification accuracy of the LOO analysis does not fall below the classification accuracy of the initial uniODA model. To maximize the expected cross-sample generalizability of results of the present study, only statisti- cally significant uniODA analyses that demonstrated stabi- lity in the LOO analyses were used.

Subsequently, automated classification tree analysis (CTA) software (Soltysik & Yarnold, 2010) was used to (a) determine the combination of variables maximizing classification accuracy in predicting age 6 diagnostic status and (b) apply a sequentially rejective Bonferroni-type adjustment (Yarnold & Soltysik, 2005) to the p value asso- ciated with each predictor in the CTA model. In the present study, the minimum sample size at each endpoint in the CTA model was set at 20; predictors not significant at the alpha-adjusted .05 level were eliminated. Using this proce- dure, all possible combinations of attributes are tested across the first three levels of analyses. After determining which variables warranted inclusion, an automatic “pruning” pro- cess deconstructs the initial “Bonferroni-pruned” model into all possible nested sub-branches to identify the sub-branch maximizing mean sensitivity, eliminating sub-branches not

improving effect strength for sensitivity of the classification tree model to avoid overfitting (Yarnold & Soltysik, 2010).

To enable comparisons of the effect strength of each predictor in the uniODA and CTA analyses, effect strength for sensitivity (ESS) is reported. ESS summarizes the over- all accuracy of the prediction in terms of a chance-corrected estimate of classification accuracy for which 0% represents the expected chance effect and 100% represents perfect classification accuracy. Thus, an ESS of 50% indicates that the model’s classification accuracy is halfway toward max- imum attainable classification accuracy. Yarnold and Soltysik (2005) suggested that the following guidelines for interpreting ESS values: < 25%, weak; 25–50%, moderate; 50–75%, relatively strong; 75–90%, strong; > 90%, very strong. ESS is also useful for comparing alternative models. In this study, the ESS for each classification tree also was compared to chance, and to the ESS for a parsimonious model employing only the best single-variable predictor of age 6 ODD diagnosis. Findings regarding specific indices of classification accuracy (sensitivity, etc.) for each predictor and classification tree are included in the supplemental online material.

In the initial uniODA analyses conducted for each spe- cific predictor, a large number of p values were computed. Corrections to adjust for Type I experiment-wise error rate, however, were not reported for each uniODA model, because these individual models were considered to be preliminary analyses designed to determine the relative strength of each variable as an individual predictor. Each predictor that was statistically significant in the uniODA analyses was not necessarily expected to be useful when combined with the other predictors as they would be in the optimal multivariate CTA model. In contrast to the uniODA analyses, the CTA model, which was designed to determine the combination of predictors that maximized predictive accuracy, was pruned so that only variables adding to over- all prediction at least at .05 were retained.

RESULTS

Percent of Children Meeting Age 6 ODD Diagnostic Criteria

At age 6, 11.7% (n = 93) of the 796 children met criteria for an ODD diagnosis with some impairment.

Single Variable (Unioda) Predictors

Table 1 lists the individual age 4 predictors of an ODD diagnosis based on the DISC-YC at age 6, along with ESS values, specific cutoff scores, and directions of effect. Of the 27 individual predictors, 16 (59.2%) were both statistically significant and stable in the LOO analyses. Of these sig- nificant, stable individual predictors, (a) seven (43.8%),

98 LAVIGNE, BRYANT, HOPKINS, GOUZE

specifically marital status, race, caretaker depression, NA, separation anxiety, conflict, and the ODD NA dimension, had weak predictive effects (i.e., ESS < 25%), and (b) nine (56.2%), specifically parental stress, child attachment secur- ity, SR ability, child EC, and symptoms of overall opposi- tional behavior, generalized anxiety, ADHD-inattentive type, ADHD-hyperactive impulsive type, and the ODD behavior dimension, had moderate predictive effects (i.e., ESS = 25%–50%).

Demographic Variables

Marital status, specifically having parents who were married, had a weak effect predicting ODD diagnosis at age 6. Race had a significant, but weak, predictive effect, with being White or self-identified as “other” for race predicting an age 6 ODD diagnosis. Sex and SES were not significant single variable predictors of age 6 ODD diagnosis.

TABLE 1 Individual Age 4 Predictors of Age 6 Diagnosis

Domain (Age 4) Age 4 Predictors M (SD) Range p Value (Stability)a ESS Direction of Significant Effects and Cutoff Scores

(Range for Continuous Measures)

Demographics Sex — — 1 Age (Wave 1) 3.87 (0.32) 3.87–5.14 .18 Marital — — .033 (stable) 10.14% Married predicts ODD present SES group — — .1 Race — — .049 (stable) 12.93% White and Other predicts ODD diagnosis

Context Conflict (FES) 2.45 (1.86) 0–9 .0002 (stable) 22.92% Conflict > 2.66 predicts ODD present Stress (Cohen) 21.64 (7.4) 3–52 .0001 (stable) 25.99% Stress > 20.86 predicts ODD present

Parent Depression: BDI 5.62 (6.66) 0–46.97 .005 (stable) 17.47% BDI > 5.35 predicts ODD present Parenting Support-engagement 44.53 (6.17) 0–50 .076

Hostile–overall 14.05 (6.0) 0–45 .0004 (not stable) Scaffold Overall 0 (2.65) −12.21 –

5.67 .13

Child-Personality Attachment security .40 (.19) −.44 – .77 .00000 (stable) 28.69% Less secure attachment (< .38) predicts ODD diagnosis Sensory regulation total

161.63 (16.2) 73.2–190.0 .0000000 (stable) 32.12% Poorer sensory regulation ability (< 159.82) predicts ODD diagnosis

NA 0 (3.05) −11.9 – 11.0

.004 (stable) 18.64% More negative temperament (> 4.67) predicts ODD diagnosis

EC 5.16 (.60) 2.96–7.0 .000000 (stable) 33.85% EC ≤ 4.95 predicts ODD present Inhibitory control (Statue)

21.02 (7.49) 0–21.05 .001 (not stable)

Child-Cognitive Verbal (PPVT) 105.03 (15.69) 60–148 .58 Child Psycho-

pathology Oppositional-CSI 0.54 (1.37) 0–8 .000000 (stable) 42.92% ODD ≥ 1.0 predicts ODD present

Anxiety-CSI general 0.65 (1.06) 0–10 .000000 (stable) 35.12% GAD ≥ 1.0 predicts ODD present Anxiety-Separation anxiety

0.33 (0.89) 0–8 .0001 (stable) 20.99% Separation anxiety ≥ 1.0 predicts ODD present

Major depression CSI

0.04 (0.29) 0–4 .20

ADHDi 0.75 (1.62) 0–9 .0001 (stable) 30.09% ADHDi ≥ 1.0 predicts ODD present ADHDH 1.30 (2.08) 0–9 .000000 (stable) 33.29% ADHDH ≥ 1.0 predicts ODD present ODD-B 0.25 (0.65) 0–3 .000000 (stable) 35.00% ODD-B ≥ 1.0 predicts ODD present ODD-NA 0.19 (0.45) 0–3 .000000 (stable) 23.63% ODD-NA ≥ 1.0 predicts ODD present

Note: For scales with mean scores of zero, standard scores were used. ESS = effect strength for sensitivity (ESS summarizes the overall prediction accuracy; 0% = chance effect, 100% = perfect classification accuracy); ODD = oppositional defiant disorder; SES = socioeconomic status; FES = Family Environment Scale; BDI = Beck Depression Inventory; NA = negative affect; EC = effortful control; PPVT = Peabody Picture Vocabulary Scale; CSI = Child Symptom Inventory; GAD = generalized anxiety; ADHD = attention deficit/hyperactivity disorder; ADHDi = ADHD Inattentive CSI; ADHDH = ADHD Hyperactive-Impulsive CSI; ODD-B = ODD–behavior dimension.

aThe p value refers to the finding of a significant cutoff score. Stability refers to presence or absence of stable findings in leave-one-out analyses concerning cross-sample stability of classification results.

ODD PREDICTORS 99

Contextual Variables

Age 4 conflict and stress were both significant predictors of age 6 ODD diagnosis. The overall strength of the pre- dictive relationship was weak for conflict but moderate for stress. Higher conflict and stress levels were associated with age 6 ODD diagnoses.

Parent and Parenting Variables

Parental depression was a weak but significant predictor, with higher levels of age 4 parental depression predicting ODD diagnosis at age 6. The three parenting variables— support, engagement, and scaffolding—were either not sig- nificant or significant but not LOO stable predictors of age 6 ODD diagnosis.

Child Factors

Several age 4 child factors were associated with age 6 ODD diagnosis. Lower levels of EC or SR ability at age 4 were moderately strong predictors of an ODD diagnosis at age 6. Secure attachment was also a moderately strong predictor, with less securely attached children at age 4 more likely to have an ODD diagnosis at age 6. NA was a weak but significant predictor, with higher levels of NA at age 4 associated with an ODD diagnosis at age 6. Measures of behavioral inhibition (Statue) and receptive vocabulary were not significant predictors.

Child Psychopathology

Higher levels of ODD symptoms, GAD, ADHD-H, ADHD-I, and ODD-B at age 4 were moderately strong predictors of an ODD diagnosis at age 6. Separation anxiety and ODD-NA were weak predictors, whereas symptoms of child depression were not a significant predictor of an age 6 ODD diagnosis.

Combining Potential Predictors: CTAs

The next set of analyses examined the classification accu- racy of combinations of predictors of age 6 ODD diagnostic status. The first model (Figure 1) included predictors from all four risk domains and demographic characteristics, as well as total score on the ODD scale and each measure of psychopathology. The second model (Figure 2) examined the two ODD dimensions rather than overall ODD, along with the other predictors.

The enumerated CTA analysis examines all possible combinations of predictors, including predictors not sig- nificant in uniODA, to identify the multivariable model with the best global predictive ability. In interpreting the results, ESS value provides useful information about the predictive ability of the overall CTA model compared to chance. Following customary practice in diagramming CTA models, right branches predict case status, and left

branches predict noncase status. Terminal endpoints for right branches are presented as percentage values meeting criteria for case status, whereas left branches are pre- sented as percentage values meeting criteria for noncase status. Significant cutoff scores are presented for each branch, along with sequentially rejective Bonferroni- adjusted, two-tailed p values.

Model 1: Overall ODD Scale as a Predictor

In the model with overall ODD scores entered as a predictor (see Figure 1), the variable appearing at the first node is age 4 total CSI ODD score. The presence of any ODD symptom rated “often” or “very often” (i.e., the right- hand arrow) predicts the presence of an age 6 ODD diag- nosis. Individuals with 1 or greater on the symptom count scale were more likely to be a case, with 32.54% of those with one or more symptom being cases at age 6. Among those with no age 4 symptoms, 93.94% were not ODD cases at age 6.

There were three specific paths to case status: (a) for chil- dren with no CSI ODD symptoms, who were self-classified as White or “other” race, and who had poorer SR ability (SR score ≤ 171.32), 21.35% met criteria for an ODD diagnosis at age 6; (b) for children with more than one ODD symptom, 44.00% met criteria for an ODD diagnoses at age 6; (c) for those with one CSI ODD symptom who were 4.18 years (approximately 4 years 2 months) or younger when tested, 34.78% were diagnosable with ODD at age 6.

None of the branches in the total classification tree were very strong predictors of the presence of ODD at age 6, that is, no branch correctly classified children with ODD at age 6 at a rate higher than 44%. In contrast, four branches were strong predictors of the absence of ODD diagnoses at age 6: (a) the branch including children with no age 4 ODD symptoms who were Black, Latino, or Asian correctly predicted the absence of ODD diagnoses for 98.77% of children; (b) for children with no age 4 ODD symptoms who were White or of “other” race and who showed no age 4 symptoms of generalized anxiety, 93.89% did not meet age 6 ODD criteria; (c) for children with no age 4 ODD symptoms, of White or “other” race, one or more GAD symptoms at age 4, and good SR ability, 100% did not meet age 6 criteria for ODD; (d) for children with one age 4 CSI ODD symptom who were older than 4.18 months of age when evaluated, 93.48% did not meet criteria for ODD at age 6 years.

The ESS for Model 1 overall was 56.29%, which is a relatively large effect. Clearly, the Model 1 classification tree is an improvement over chance prediction of an age 6 ODD diagnosis. When compared to a parsimonious model including only the best single-variable predictor, the age 4 ODD symptom scale for which the ESS was 42.92%, there was a notable 13.37% increase in ESS for the Model 1 classification tree. When other measures of classification accuracy are considered, improvements in prediction for

100 LAVIGNE, BRYANT, HOPKINS, GOUZE

the classification tree approach are also apparent. First, the sensitivity (SE) of the CSI ODD scale alone was 59.14%; for the CTA, SE increased by 17.2%, to 76.34% (see Table 2 for classification accuracy statistics for the CTAs). Generally, when SE improves, specificity (SP) of a measure declines. For these two measures, the relatively large increase in SE for the CTA model was accompanied by a small decrease in specificity (from 83.78% for the CSI ODD scale alone, to 79.94% for the CTA model). This result is an increased ability to identify cases; the CTA approach leads to better identification of true cases at age 6, with a small decline in the ability to detect true noncases, compared to prediction using the overall ODD scale score alone.

There would also be a potential advantage to using the CTA approach over the more parsimonious use of the CSI ODD scale alone if the CTA approach improved the ability

to predict specific groups of individuals who are either likely or unlikely to develop ODD. The overall positive predictive value (the number of true positives correctly predicted by a high score on the ODD scale) of the ODD scale is 32.54%. The ability of the classification tree to predict true cases of ODD is better than 33.01% for two branches, for which the 44.0% and 34.78% were correctly identified, and worse (21.35%) for one other. The negative predictive value (NPV; the number of true negatives cor- rectly identified by a low score on the test) for the ODD scale alone is 93.94%. Because the NPV for the ODD scale is so high, the CTA approach is unlikely to produce a markedly stronger NPV. Indeed, the NPV for the four branches of the CTA that predicted no disorder were only slightly better (98.77%, 100.0%) for two branches and slightly worse (93.89%, 93.48%) for the others. Thus, the

N = 33/33

> 1.5

= 93.89%

No ODD

34.78%

ODD

98.77%

No ODD

100%

No ODD

< 171.32

44.00%

ODD

N = 44/100

N = 43/46

55/169 = 32.54% 589/627 = 93.94%

Sensory-

regulation

93.48%

No ODD

21.35%

ODD

N = 240/243

p = .045

p = .018

p = .001 p = .001

Model 1 Overall ESS = 56.29% Legend: CSI, Child Symptom Inventory; ODD, Oppositional Defiant Disorder. Cutoff scores for each measure are indicated on the branches.

CSI ODD

p = .001

Race

< 1 > 1

Black, Latino, Asian

White, Other

CSI GAD Age

CSI ODD

< 1.5

p = .002 < 1

> 1

< 4.18 mos. > 4.18 mos.

N = 8/23

N = 19/89

> 171.32

N = 246/262

FIGURE 1 Pruned tree for age 6 oppositional defiant disorder (ODD) diagnosis with overall ODD scale as an age 4 predictor (Model 1). Note: Cutoff scores for each measure are indicated on the branches. Model 1 Overall effect size for sensitivity = 56.29%. CSI = Child Symptom Inventory; GAD = generalized anxiety.

ODD PREDICTORS 101

CTA approach provides an overall advantage in classifica- tion accuracy compared to the use of the CSI ODD scale alone and is better for classifying the presence of age 6 ODD for specific subgroups. It shows no significiant advan- tage, however, in predicting the absence of an ODD diag- nosis compared to the CSI ODD scale at age 4 alone.

Model 2: ODD Dimensions as a Predictor

In Model 2, the two dimensions of ODD were used as predictors rather than the overall CSI ODD scale. The CTA approach resulted in a classification scheme that included the child’s CSI GAD level, SES, ODD-B symptom level, effortful control, and temperamental negative affect. The ODD-NA dimension did not improve classification accu- racy in the CTA model.

In the second model, the presence of any (≥ 1.0) CSI-GAD symptoms at age 4 predicted an age 6 ODD diagnosis correctly for 20.75% of children. For children with at least one GAD symptom at age 4, two branches predicted age 6 ODD: (a) for

those with at least one GAD symptom and one ODD-B symp- tom at age 4, 38.64% met criteria for an ODD diagnosis at age 6, and (b) for those with at least one GAD symptom at age 4, no ODD-B symptoms at age 4, but high levels of NA, 20.37% met criteria for an ODD diagnosis at age 6. Among children not exhibiting any GAD symptoms at age 4, an age 6 ODD diagnosis was predicted correctly by being in the highest (Class 1) SES group and exhibiting lower (≤ 4.95) levels of EC for 32.56% of such children. The absence of an ODD diagnosis was predicted correctly by (a) an absence of age 4 GAD symptoms and SES below Class 1 for 97.50% of chil- dren in that branch; (b) no GAD symptoms, SES below Class 1, and high levels of EC for 96.13% of such children; and (c) at least one age 4 GAD symptom, no age 4 ODDB symptoms, and low NA for 91.80% of such children.

The overall ESS for this second CTA model was 51.23% (SE = 75.27%, SP = 75.96%). Although this is an improve- ment compared to either the ESS for ODD-B (35.00%) or ODD-NA (23.63%), and to the ESS of 42.92% for the CSI

ODD

Behavior

< 1 > 1

CSI

GAD

= 97.50%

No ODD

SES group

32.56%

ODD

N = 22/108

Effortful

control

38.64%

ODD

N = 34/88N = 273/280

66/318 = 20.75% 451/478 = 94.35%

91.80%

No ODD

96.13%

No ODD

20.37%

ODD

N = 112/122

p = .001

p = .001

p = .001 p = .044

> 1 < 1

Model 2 Overall ESS = 51.23% Legend: CSI, Child Symptom Inventory; ODD, Oppositional Defiant Disorder. Cutoff scores for each measure are indicated on the branches.

N = 149/155 N = 14/43

Negative

affect

< Class 1 Class 1

> 4.95 < 4.95 < 4.44 > 4.44

p = .001

FIGURE 2 Pruned tree for age 6 oppositional defiant disorder (ODD) diagnosis with ODD dimensions as predictors (Model 2). Note: Cutoff scores for each measure are indicated on the branches. CSI = Child Symptom Inventory; GAD = generalized anxiety; SES = socioeconomic status.

102 LAVIGNE, BRYANT, HOPKINS, GOUZE

ODD scale alone, it is smaller than the ESS for Model 1 (56.29%). For these reasons, Model 1 is a more effective predictive model.

Sex- and Race/Ethnicity-Specific Ctas

Readers accustomed to seeing demographic differences in standardized scores for tests might prefer to use classification trees that differed across demographic groups. Because the enumeration process used in the automated CTA program tested every possible combination of predictors at each node in the first three levels of the classification trees, it is clear that no tree in which a demographic variable was tested in the first node was superior to the classification trees in Figures 1 and 2. Completely separate trees, however, could yield results in which classification accuracy was superior for one particular demographic group compared to a classification tree that was not specific to a single demographic group. As a result, auto- mated CTAs were conducted for each of the two sexes sepa- rately, and for Whites and for combined minority groups separately (sample sizes were too small to allow for analyses of each race/ethnic group). The overall classification accuracy values (ESSs) were then compared across groups.

The classification tree in Figures 1 included the variable of overall ODD symptoms; for that model the ESS was 56.29%. The ESS for the girls-only model that included the overall ODD symptom variable was slightly lower (53.20%) and lower (48.26%) for boys. When separate classification trees were examined for Whites and minority groups, the ESS for Whites of 39.35% was considerably lower than the ESS of 56.29% for the total sample. The only demographic group for which the ESS was better than that for the total sample was that for minority group members, for which the ESS was 63.97%. For that group, the total CSI ODD score was the only significant predictor. Overall, it appears that there is little advantage for separate classifica- tion trees for three of the four demographic groups.

In the second model, the two ODD dimensions were included as predictors rather than the overall ODD score. For the total sample, the ESS for this second model was 51.23%. For separate demographic groups, there was an improvement in ESS for the minority groups (ESS = 64.71%), a slight improve- ment for girls (53.20%), but the ESS was lower for boys (47.10%) and considerably lower for Whites (39.55%). When compared to the overall ESS for the total sample of 56.29%, there was improvement when ODD dimensions were predictors only for minority groups. Overall, there appears to be little advantage to using classification trees that differ based on demographic subgroups. See online supplementary material for detailed results of the CTA models for the subgroups.

DISCUSSION

There have been considerable advances in our understand- ing of the structure of ODD symptoms (Ezpeleta, Granero, De La Osa, Penelo, & Domenech, 2012; Lavigne, Bryant, et al., 2015) and the multi-domain risk factors for this disorder (Lavigne, Gouze, et al., 2015; Lavigne et al., 2012). Nevertheless, the statistical procedures used in such studies do not lend themselves to determining how well we can predict which preschool-age children will meet criteria for this disorder in the future. Because early onset ODD carries risk for future academic and mental health problems, and there are effective interventions for this disorder, it is important to optimize prediction of which young children are most likely to meet criteria for ODD in order to deter- mine whom to target for early intervention.

The aim of this study, therefore, was twofold: (a) to determine if, at age 4, optimal cut-scores on variables in multiple domains could be identified that predict ODD diagnoses at age 6 in a highly diverse sample and (b) to determine if classification tree analyses incorporating

TABLE 2 Classification Accuracy Statistics: Classification Trees

Domain (Age 4) Age 4 Predictors ESS SE SP PPV NPV

Model 1 (Overall ODD) 56.29% 76.34% 79.94% 33.49% 96.23% Model 2 (ODD dimensions) 51.23% 75.27% 75.96% 29.29% 95.87%

Subgroups: Overall ODD Boys 48.26% 67.39% 80.87% 31.96% 94.90% Girls 53.20% 80.85% 72.35% 27.74% 96.94% White 39.35% 58.33% 81.22% 33.02% 92.47% Minorities 63.97% 81.82% 82.15% 31.76% 97.80%

Subroups: ODD Dimensions Boys 47.10% 80.43% 66.67% 24.34% 96.23% Girls 53.20% 80.85% 72.35% 27.74% 96.94% White 39.55% 58.33% 81.22% 33.02% 92.47% Minorities 64.71% 93.94% 70.77% 24.60% 99.14%

Note: ESS = effect size for sensitivity; SE = sensitivity; SP = specificity; PPV = positive predictive value; NPV = negative predictive value; ODD = oppositional defiant disorder.

ODD PREDICTORS 103

multiple predictors could improve on the identification of children with and without an ODD diagnosis at age 6. First, we examined single risk factors to determine if cut-scores on specific variables predicted ODD diagnosis at age 6. We then used automated CTA to determine which combination of variables provided the greatest classification accuracy in predicting ODD status at age 6.

In the single predictor analyses, age 4 risk factors from each domain, except for the parenting domain, yielded statistically significant predictions of an ODD diagnosis at age 6 that had strong, expected cross-sample generalizabil- ity. Specifically, being White and having married parents were significant but weak predictors of an ODD diagnosis at age 6. The contextual factor of conflict (score > 2.66) was a weak predictor, whereas stress (score > 20.86) was a mod- erately strong predictor of an ODD diagnosis at age 6. Even relatively low levels of caregiver depression (Beck Depression Inventory score > 5.35) had a small effect size in predicting the likelihood of an ODD diagnosis at age 6. Child characteristics of attachment security, effortful con- trol, and SR ability had moderate effect sizes in predicting an ODD diagnosis at age 6, and NA had a small effect size.

Given that ODD symptoms are fairly stable from pre- school through the early school years (Campbell, 1990; Lavigne et al., 1998), it is not surprising that even relatively low levels (score ≥ 1 on a scale of 0–8 symptoms) of ODD symptoms at age 4 had the strongest effect on the accuracy of predicting ODD diagnosis at age 6. In addition to homo- typic symptoms predicting the likelihood of an age 6 ODD diagnosis, both generalized and separation anxiety were small, but statistically significant, predictors of an ODD diagnosis at age 6.

After examining the individual risk factors, we compared two classification tree analyses to determine which combi- nation of variables provided the greatest classification accu- racy in predicting age 6 diagnostic status. The two models differed in one important respect: Model 1 used the original age 4 ODD scale score as a predictor, and Model 2 used the two age 4 ODD dimension scores as predictors. The first model yielded an overall classification accuracy of 56.29%, with a specificity of 79.94% and a sensitivity of 76.34%. Model 2 yielded an overall classification rate of 51.23%, with a specificity of 75.96% and a sensitivity of 75.27%. Whereas both models improved predictive accuracy com- pared to chance, the ESS for Model 1 (total ODD scale) was higher than that for Model 2 (ODD dimensions); for that reason Model 1 is preferable.

When the CTA Model 1 is compared to the use of the age 4 ODD symptom scale alone, there is an improvement of more than 13% in overall classification accuracy and an improvement in SE of more than 17% at the expense of a slight, approximately 4%, decrease in SP. This is a notable improvement in classification accuracy for the CTA.

Although the CTA approach improved overall classifica- tion accuracy, our ability to predict who will meet criteria

for an ODD diagnosis at age 6 is still limited, and it is easier to predict who will not meet criteria for that disorder. So few efforts have been made to report on predictive classification accuracy for other disorders in young children that it is difficult to know how the degree of accuracy in predicting ODD compares with that of other disorders. Given the interest in trying to determine if dimensions of ODD differ in predictive value for various disorders, it is of note that analyzing separate dimensions of ODD did not materially improve results compared to using a global measure of ODD. ODD-NA was a statistically significant but weak solo predictor but was not a good predictor for any of the subgroups included in the classification tree when entered into the multivariable CTA analyses. ODD-B did contribute to the overall predictive accuracy of Model 2, but that model was less effective overall than the model that included the global CSI ODD scale.

The clinical utility of the use of classification tree approaches, and of this study in particular, will be depen- dent in large part on two factors: the availability of com- munity resources and whether the clinician or clinical administrator is concerned with population health or mental health. In the typical community or tertiary care mental health setting where treatment is provided for children and adolescents, it is unlikely that the considerations addressed in this report would influence decisions about who receives treatment. In such settings, a parent is seeking treatment for a perceived problem, increasing the likelihood that some impairment is present. If the child’s behavior meets criteria for ODD, treatment would be implemented, and this would occur for all children meeting the diagnostic criteria. Concerns about setting priorities for treatment come into play when mental health problems are being addressed at a population level, in schools, or at pediatric practices. Because screening is recommended in primary care and each pediatric practitioner will see between 1,000 and 1,700 children younger than age 18 in a primary care practice (Committee on Careers and Opportunities, 1996), and visit frequency is highest among preschoolers, between dozens and hundreds of young children would be screened annually. Given the prevalence of ODD in that age group at 8%–10% (with impairment), many children would be eligi- ble for referral or in-setting treatment, potentially more than can readily be treated in busy practices or communities with limited resources.

In such settings, there are two significant methodological problems that enter into deciding whom to treat. The first has to do with limitations in identifying children with beha- vior problems via screening. The prevalence of ODD at 8%–10% is high enough to warrant attention as a matter of concern for population health. However, given the clas- sification accuracy (Lavigne, Meyers, & Feldman, 2016) of even the best available screening instruments, the false positive rate will be very high for disorders with that pre- valence (Lavigne, Feldman, & Meyers, 2016). To reduce the

104 LAVIGNE, BRYANT, HOPKINS, GOUZE

impact of this problem, sequential screening has been recommended (Lavigne, Feldman, et al., 2016). Once the preschool children with ODD have been identified, the issues concerning whom to prioritize for treatment discussed in the present report become relevant. If resources allow, all children with high levels of ODD symptoms could be trea- ted; there would be no need to be concerned about how likely the ODD symptoms are to be present in 2 years. Other factors (e.g., GAD and ADHD symptoms) might need to be considered, but some groups of children with a lower like- lihood of exhibiting ODD at age 6 (e.g., children with only one ODD symptom who were screened later at age 4) might be assigned to a watchful waiting group to be monitored at future visits. For a test or classification tree with high negative predictive value and high specificity, confidence that neither intervention nor watchful waiting is needed increases, and parents can be informed with greater confi- dence that their child is likely to exhibit ODD at a later date.

This approach could also play a role in evidence-based assessment approaches such as that recommended by Youngstrom, Choukas-Bradley, and Calhoun (2015) that include considerations of base rates, psychopathology risk factors, and rating scales to supplement and potentially streamline evaluations. Information derived from classifica- tion trees can also be important in informing parent con- sumers who are deciding whether to pursue treatment in view of other family, child, and work considerations in concrete terms about the likelihood of remission so that they can estimate what might be gained from initiating treatment.

Limitations

When considering the CTA results, it is important to keep in mind that this statistical approach is designed to optimize the predictive value of combinations of variables and not to explicate causal relationships between variables. Although the pruning process for reducing the branches of the tree that do not add significant value to prediction reduces the likelihood of chance findings, this analytic approach cannot articulate why the combinations of factors lead to the results they do, for example, why there are very different like- lihoods of meeting diagnostic criteria for ODD at age 6 for boys and girls who are low in ODD symptoms, high on EC, and of self-described White or Other race. Finally, this study is based on parental reports, and results may differ if early childhood predictors included both parent and tea- cher reports.

Despite these limitations, the present findings provide useful information for determining which children to target for treatment. Future studies are needed to try to improve on the classification accuracy of these variables and to deter- mine whether these results can be replicated. There is gen- eral concern about the limitations of the utility of research findings for clinicians, and an approach such as the one we

adopted here holds some promise for increasing the clinical utility of findings on risk factors and processes by which disorders may develop. In addition, little is known about the ability to predict other disorders among young children, and studies of disorders that might be present in early grammar school are needed.

FUNDING

This research was supported by National Institute of Mental Health grant MH 063665.

REFERENCES

American Psychiatric Association. (1994). Diagnostic and statistical man- ual of mental disorders (4th ed.). Washington, DC: American Psychiatric Association.

American Psychiatric Association. (2013). Diagnostic and statistical man- ual of mental disorders (5th ed.). Washington, DC: American Psychiatric Association.

Angold, A., Costello, E. J., & Erkanli, A. (1999). Comorbidity. Journal of Child Psychology and Psychiatry, 40, 57–87. doi:10.1111/jcpp.1999.40. issue-1

Baker, J. K., Fenning, R. M., Crnic, K. A., Baker, B. L., & Blacher, J. (2007). Prediction of social skills in 6-year-old children with and without developmental delays: Contributions of early regulation and maternal scaffoldng. American Journal of Mental Retardation, 112, 375–391. doi:10.1352/0895-8017(2007)112[0375:POSSIY]2.0.CO;2

Barlow, J., Bergman, H., Konner, H., Wei, Y., & Bennett, C. (2016). Group- based parent training programmes for improving emotional and beha- vioral adjustment in young children (review). Cochrane Database of Systematic Reviews. Art. No. CD0003680. doi:10.1002/14561858. CD0003680

Beck, A. T., Ward, C. H., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for measuring depression. Archives of General Psychiatry, 4, 561–571. doi:10.1001/archpsyc.1961.01710120031004

Burke, J. D., & Loeber, R. (2010). Oppositional defiant disorder and the explanation of the comorbidity between behavioral disorders and depres- sion. Clinical Psychology: Science and Practice, 17, 319–326.

Burke, J. D., Loeber, R., Lahey, B. B., & Rathouz, P. J. (2005). Developmental transitions among affective and behavioral disorders in adolescent boys. Journal of Child Psychology and Psychiatry, 46, 1200– 1210. doi:10.1111/jcpp.2005.46.issue-11

Campbell, S. B. (1990). Behavior problems in preschool children: Clinical and developmental issues. New York, NY: Guilford.

Campbell, S. B. (1995). Behavior problems in preschool children: A review of recent research. Journal of Child Psychology and Psychiatry, 36(9), 113–114. doi:10.1111/j.1469-7610.1995.tb01657.x

Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behavior, 24, 385–396. doi:10.2307/2136404

Committee on Careers and Opportunities. (1996). Committee report: Population-to-pediatrician ratio estimates: A subject review. Pediatrics, 97, 597–600.

Costello, E. J., Egger, H. E., & Angold, A. (2005). 10-year research update review: The epidemiology of child and adolescent psychiatric disorders: I. Methods and public health burden. Journal of the American Academy of Child and Adolescent Psychiatry, 44(10), 972–986. doi:10.1097/01. chi.0000172552.41596.6f

DeVito, C., & Hopkins, J. (2001). Attachment, parenting, and marital dissatisfaction as predictors of disruptive behavior in preschoolers.

ODD PREDICTORS 105

Developmental Psychopathology, 13, 215–231. doi:10.1017/ S0954579401002024

Drabick, D. A. G., Gadow, K. D., & Loney, J. (2007). Source-specific oppositional defiant disorder: Comorbidity and risk factors in referred elementary schoolboys. Journal of the American Academy of Child and Adolescent Psychiatry, 46(1), 92–101. doi:10.1097/01. chi.0000242245.00174.90

Drabick, D. A. G., Ollendick, T. H., & Bubier, J. L. (2010). Co-occurrence of ODD and anxiety: Shared risk processes and evidence for a dual- pathway model. Clinical Psychology: Science and Practice, 17, 307– 318. doi:10.1111/j.1468-2850.2010.01222x

Dunn, L., & Dunn, L. (1997). The peabody picture vocabulary test (3rd ed.). Circle Pines, MN: American Guidance Service.

Egger, H. E., & Angold, A. (2006). Common emotional and behavioral disorders in preschool children: Presentation, nosology, and epidemiol- ogy. Journal of Child Psychology and Psychiatry, 47, 313–337. doi:10.1111/j.1469-7610.2006.01618.x

Eisenberg, N., Valiente, C., Spinrad, T. L., Cumberland, A., Liew, J., Reiser, M., … Losoya, S. H. (2009). Longitudinal relations of children’s effortful control, impulsivity, and negative emotionality to their externa- lizing, internalizing, and co-occurring behavior problems. Developmental Psychology, 45, 988–1008. doi:10.1037/a0016213

Evans, G. W. (2004). The environment of childhood poverty. American Psychologist, 59, 77–92. doi:10.1037/0003-066X.59.2.77

Ezpeleta, L., Granero, R., De La Osa, N., Penelo, E., & Domenech, J. M. (2012). Dimensions of oppositional defiant disorder in 3-year-old pre- schoolers. Journal of Child Psychology and Psychiatry, 53, 1128–1138. doi:10.1111/j.1469-7610.2012.02545.x

Fisher, P., & Lucas, C. (2006). Diagnostic Interview Schedule For Children (DISC-IV)-Young Child. New York, NY: Columbia University.

Foy, J. M., Kelleher, K. J., & Laraque, D. (2010). Appendix S4: The case for routine mental health screening. Pediatrics, 125, S133–S139. doi:10.1542/peds.2010-0788J

Gadow, K. D., & Sprafkin, J. (1997). Child symptom inventory 4, norms manual. Stonybrook, NY: Checkmate Plus.

Garland, A., Hurlburt, M., & Hawley, K. (2006). Examining psychotherapy processes in a services research context. Clinical Psychology: Science and Practice, 13, 30–46.

Goodman, S. H., Rouse, M. H., Connell, A. M., Broth, M. R., Hall, C. M., & Heyward, D. (2011). Maternal depression and child psychopathology: A meta-analytic review. Clinical Child and Family Psychology Review, 14, 1–27. doi:10.1003/sl10567-010-0080-1

Gouze, K. R., Hopkins, J., LeBailly, S. A., & Lavigne, J. V. (2009). Re- examining the epidemiology of sensory regulation dysfunction and comorbid psychopathology. Journal of Abnormal Child Psychology, 37, 1077–1087. doi:10.1007/s10802-009-9333-1

Graham, J. W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60, 549–576. doi:10.1146/annurev. psych.58.110405.085530

Grant, K. E., Compas, B. E., Thurm, A. E., McMahon, S. D., Gipson, P. Y., Campbell, A. J., … Weserholm, R. I. (2006). Stressors and child and adolescent psychopathology: Evidence of moderating and mediating effects. Clinical Psychology Review, 26, 257–283. doi:10.1016/j. cpr.2005.06.011

Hipwell, A., Keenan, K., Kasza, K. R. L., Stouthamer-Loeber, M., & Bean, T. (2008). Reciprocal influences between girls’ conduct problems and depression, and parental punishment and warmth: A six year prospective analysis. Journal of Abnormal Child Psychology, 36, 663–677. doi:10.1007/s10802-007-9206-4

Hollingshead, A. B. (1975). Four-factor index of social position (Unpublished manuscript). Department of Psychology, Yale University, New Haven, CT.

Korkman, M., Kirk, U., & Kemp, S. (1997). NEPSY: A developmental neuropsychological assessment manual. San Antonio, TX: Psychological Corporation.

Lavigne, J. V., Arend, R., Rosenbaum, D., Binns, H. J., Christoffel, K. K., & Gibbons, R. D. (1998). Psychiatric disorders with onset in the pre- school years: I. Stability of diagnosis. Journal of the American Academy of Child and Adolescent Psychiatry, 37(12), 1246–1254. doi:10.1097/ 00004583-199812000-00007

Lavigne, J. V., Bryant, F. B., Hopkins, J., & Gouze, K. R. (2015). Dimensions of oppositional defiant disorder in young children: Model comparisons, gender and longitudinal invariance. Journal of Abnormal Child Psychology, 43, 423–439. doi:10.1007/s10802-014-9919-0

Lavigne, J. V., Cicchetti, C., Gibbons, R. D., Binns, H. J., Larsen, L., & DeVito, C. (2001). Oppositional defiant disorder with onset in preschool years: Longitudinal stability and pathways to other disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 40, 1393– 1400. doi:10.1097/00004583-200112000-00009

Lavigne, J. V., Dahl, K. P., Gouze, K. R., LeBailly, S. A., & Hopkins, J. (2015). Multi-domain predictors of oppositional defiant disorder symp- toms in preschool children: Cross-informant differences. Child Psychiatry and Human Development, 46, 308–319. doi:10.1007/ s10578-014-0472-4

Lavigne, J. V., Feldman, M., & Meyers, K. M. (2016). Screening for mental health problems: Addressing the base rate fallacy for a sustainable screening program in integrated primary care. Journal of Pediatric Psychology. doi:10.1093/jpepsy/jsw048

Lavigne, J. V., Gouze, K. R., Bryant, F. B., & Hopkins, J. (2014). Dimensions of oppositional defiant disorder in young children: Heterotypic continuity with anxiety and depression. Journal of Abnormal Child Psychology, 42, 937–951. doi:10.1007/s10802-014- 9853-1

Lavigne, J. V., Gouze, K. R., Hopkins, J., & Bryant, F. (2015). A multi- domain cascade model of early childhood risk factors associated with ODD symptoms in a community sample of 6-year-olds. Development and Psychopathology, 28, 1547–1562. doi:10/1017/ S9054579415001194

Lavigne, J. V., Gouze, K. R., Hopkins, J., Bryant, F. B., & LeBailly, S. A. (2012). A multi-domain model of risk factors for ODD symptoms in a community sample of 4-year-olds. Journal of Abnormal Child Psychology, 40, 741–757. doi:10.1007/s10802-011-9603-6

Lavigne, J. V., LeBailly, S. A., Hopkins, J., Gouze, K. R., & Binns, H. J. (2009). The prevalence of ADHD, ODD, depression and anxiety in a community sample of 4-year-olds. Journal of Clinical Child and Adolescent Psychology, 38, 315–328. doi:10.1080/15374410902851382

Lavigne, J. V., Meyers, K. M., & Feldman, M. (2016). Systematic review: Classification accuracy of behavioral screening measures for use in integrated primary care settings. Journal of Pediatric Psychology, 41, 1091–1109. doi:10.1093/jpepsy/jsw049

Lovejoy, M. C., Weis, R., O’Hare, E., & Rubin, E. C. (1999). Development and initial validation of the parent behavior inventory. Psychological Assessment, 11(4), 1–12. doi:10.1037/1040-3590.11.4.534

McClelland, M. M., Cameron, C. E., Connor, C. M., Farris, C. L., Jewkes, A. M., & Morrison, F. J. (2007). Links between behavioral regulation and preschoolers’ literacy, vocabulary, and math skills. Developmental Psychology, 43(4), 947–959. doi:10.1037/0012-1649.43.4.947

McCubbin, H. I., Thompson, A., & Elver, K. M. (1996). Family distress index. In H. I. McCubbin, A. Thompson, & M. A. McCubbin (Eds.), Family assessment: Resiliency, coping, and adaptation-Inventories for research and practice (pp. 783–788). Madison: University of Wisconsin.

McCubbin, M. A., McCubbin, H. I., & Thompson, A. (1996). Family problem solving and communication. In H. I. McCubbin, A. Thompson, & M. A. McCubbin (Eds.), Family assessment: Resiliency, coping,and adaptation-Inventories for research and practice (pp. 639–686). Madison: University of Wisconsin.

McIntosh, D. M., Miller, L. J., Shyu, V., & Hagerman, R. J. (1999). Sensory-modulation disruption, electrodermal responses, and functional behaviors. Developmental Medicine and Child Neurology, 41, 608–615. doi:10.1111/j.1469-8749.1999.tb00664.x

106 LAVIGNE, BRYANT, HOPKINS, GOUZE

McMahon, S. D., Grant, K. E., Compas, B. E., Thurm, A. E., & Ey, S. (2003). Stress and psychopathology in children and adolescents: Is there evidence of specificity? Journal of Child Psychology and Psychiatry, 44 (1), 107–133. doi:10.1111/1469-7610.00105

Moffitt, T. E. (1993). Adolescent-limited and life-course-persistent antiso- cial behavior: A developmental taxonomy. Psychological Review, 100, 674–701. doi:10.1037/0033-295X.100.4.674

Moos, R. H., & Moos, B. S. (1986). Family Environment Scale manual. Palo Alto, CA: Consulting Psychologists Press.

NICHD Early Childhood Research Network. (1999). Child care and mother-child interaction in the first three years of life. Developmental Psychology, 35, 1399–1413. doi:10.1037/0012-1649.35.6.1399

Ollendick, T. H., Finch, A. J., & Ginn, F. W. (1974). Comparison of Peabody, Leiter, WISC, and academic achievement scores among emo- tionally disturbed children. Journal of Abnormal Child Psychology, 2(1), 47–51. doi:10.1007/BF00919353

Ponitz, C. C., McClelland, M. M., Matthews, J. S., & Morrison, F. J. (2009). A structured observation of behavioral self-regulation and its contribution to kindergarten outcomes. Developmental Psychology, 45, 606–619. doi:10.1037/a0015365

Rothbart, M. K., Ahadi, S. A., Hershy, K. L., & Fisher, P. (2001). Investigations of temperament at three to seven years: The children’s behavior questionnaire. Child Development, 72(5), 1394–1408. doi:10.1111/1467-8624.00355

Shaffer, D., Fisher, P., Lucas, C. P., Dulcan, M. K., & Schwab-Stone, M. E. (2000). NIMH Diagnostic Interview Schedule for Children Version IV (NIMH DISC-IV): Description, differences from previous versions, and reliability of some common diagnoses. Journal of the American Academy

of Child and Adolescent Psychiatry, 39, 28–38. doi:10.1097/00004583- 200001000-00014

Soltysik, R. C., & Yarnold, P. R. (2010). Automated CTA software: Fundmental concepts and control commands. Optimal Data Analysis, 1, 144–160.

Stringaris, A., & Goodman, R. (2009). Three dimensions of oppositionality in youth. Journal of Child Psychology and Psychiatry, 50, 216–233. doi:10.1111/j.1469-7610.2008.01989.x

van Ijzendoorn, M., Vereijken, C., Bakermans-Kranenburg, M., & Riksen- Walraven, J. (2004). Assessing attachment security with the Attachment Q-sort: Meta-Analytic evidence for the validity of the observer AQS. Child Development, 75, 1188–1213. doi:10.1111/j.1467- 8624.2004.00733.x

Waters, E. (1987). Attachment Q-set (version 3.0). Stony Brook: State University of New York at Stony Brook.

Yarnold, P. R., & Soltysik, R. C. (2005). Optimal data analysis: A guide- book with software for Windows. Washington, DC: American Psychological Association.

Yarnold, P. R., & Soltysik, R. C. (2010). Maximizing accuracy of classifi- cation trees by optimal pruning. Optimal Data Analysis, 1, 23–29.

Youngstrom, E. A., Choukas-Bradley, S., & Calhoun, C. D. (2015). Clinical guide to the evidence-based assessment approach to diagnosis and treat- ment. Cognitive and Behavioral Practice, 22, 20–35. doi:10.1016/j. cbpra.2013.12.005

Zelazo, P. D., & Carlson, S. M. (2012). Hot and cool executive function in childhood and adolescence: Development and plasticity. Child Development Perspectives, 6, 354–360. doi:10.1111/j.1750- 8606.2012.00246x

ODD PREDICTORS 107

Copyright of Journal of Clinical Child & Adolescent Psychology is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.