Senior Seminar - Screening and Diagnostic Tools

https://doi.org/10.1177/1362361318755318

Autism 2019, Vol. 23(2) 468 –476 © The Author(s) 2018 Article reuse guidelines: sagepub.com/journals-permissions DOI: 10.1177/1362361318755318 journals.sagepub.com/home/aut

Introduction

The most recent edition of the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5) introduced substantial revisions to the diagnostic criteria for autism (American Psychiatric Association, 2013). Key changes included a shift from triadic to dyadic symptom group- ings, and a consolidation of previously separate diag- nostic subcategories (i.e. autistic disorder, Asperger’s disorder, and pervasive developmental disorder not other- wise specified) into a single category of autism spectrum disorder (ASD). These primary changes have received a great deal of attention from scientific, clinical, and lay communities, primarily focused on concern about poten- tial effects on prevalence estimates and service eligibility (Buxbaum and Baron-Cohen, 2013; Grzadzinski et al., 2013; Halfon and Kuo, 2013; Volkmar and Reichow, 2013). As a result, a number of studies of sensitivity, spec- ificity, and diagnostic concordance between DSM-IV and DSM-5 have been conducted since the draft criteria were

first released (see, for review, Kulage et al., 2014; Smith et al., 2015). By contrast, the addition of severity level ratings, an equally significant change to the diagnostic criteria for ASD, has received little scientific attention.

As noted above, changes to DSM-5 were intended, in part, to address problems with inter-rater agreement on DSM-IV subcategories (Lord and Bishop, 2015; Ozonoff, 2012a, 2012b). Moving from three subcategories to a

Factors associated with DSM-5 severity level ratings for autism spectrum disorder

Micah O Mazurek1 , Frances Lu2, Eric A Macklin2,3 and Benjamin L Handen4

Abstract The newest edition of the Diagnostic and Statistical Manual of Mental Disorders (5th ed., DSM-5) introduced substantial changes to the diagnostic criteria for autism spectrum disorder, including new severity level ratings for social communication and restricted and repetitive behavior domains. The purpose of this study was to evaluate the use of these new severity ratings and to examine their relation to other measures of severity and clinical features. Participants included 248 children with autism spectrum disorder who received diagnostic evaluations at one of six Autism Treatment Network sites. Higher severity ratings in both domains were associated with younger age, lower intelligence quotient, and greater Autism Diagnostic Observation Schedule–Second Edition domain-specific symptom severity. Greater restricted and repetitive behavior severity was associated with higher parent-reported stereotyped behaviors. Severity ratings were not associated with emotional or behavioral problems. The new DSM-5 severity ratings in both domains were significantly associated with behavioral observations of autism severity but not with measures of other behavioral or emotional symptoms. However, the strong associations between intelligence quotient and DSM-5 severity ratings in both domains suggest that clinicians may be including cognitive functioning in their overall determination of severity. Further research is needed to examine clinician decision-making and interpretation of these specifiers.

Keywords autism spectrum disorder, diagnosis, DSM-5, need for support, severity level

1University of Virginia, USA 2Massachusetts General Hospital, USA 3Harvard Medical School, USA 4University of Pittsburgh School of Medicine, USA

Corresponding author: Micah O Mazurek, Curry School of Education, University of Virginia, 417 Emmet Street South, P.O. Box 400267, Charlottesville, VA 22904, USA. Email: [email protected]

755318AUT0010.1177/1362361318755318AutismMazurek et al. research-article2018

Original Article

Mazurek et al. 469

single ASD category with two domain-specific severity ratings allowed the diagnostic system to retain strong reli- ability for overall ASD category while allowing for a multi-dimensional assessment of severity. These new rat- ings provide a system for documenting an individual’s symptom severity in the areas of social communication and restricted and repetitive behavior (RRB). Each domain receives a rating of 1 (requiring support), 2 (requiring sub- stantial support), or 3 (requiring very substantial support) (American Psychiatric Association, 2013). Additional text explanation and some examples are provided for each level, but a lack of clear-cut operational definitions means that rating determinations remain somewhat subjective. As described in a recent review (Mehling and Tassé, 2016), it is not clear how clinicians will make these determinations or whether their ratings will reflect symptom severity alone or be influenced by other indices of impairment (such as cognitive functioning) or co-occurring symptoms (such as internalizing symptoms or challenging behavior).

Symptom-related functional impact is an important ele- ment of overall severity of psychopathology. The inclusion of this dimensional coding of symptom-specific severity has intuitive appeal in that it may help guide treatment planning and allow for examination of an individual’s pro- gress over time within a particular symptom domain. However, to date, the new severity scales have not been empirically validated against other indicators of severity. As a result, clinicians have little overt guidance in making these determinations. To our knowledge, there has been no published study examining how these severity ratings will be used in clinical practice, how they relate to other meas- ures of symptom severity, or how they relate to other child characteristics. This information is necessary for interpret- ing the clinical significance, validity, and utility of these ratings. The relationship between severity ratings and cog- nitive functioning may be particularly important to exam- ine, as intellectual impairment may be conflated with overall severity. In addition, determining whether ASD severity ratings are distinct from measures of general emo- tional and behavioral symptomatology will be important for evaluating discriminant validity.

Current study

The purpose of this study was to evaluate the DSM-5 severity level ratings in a large sample of children and adolescents with ASD. Our primary aims were to (1) describe the distribution of DSM-5 defined social com- munication and RRB severity ratings across our sample, (2) assess the relationship between DSM-5 severity rat- ings and a standardized measure of ASD severity, and (3) assess the relationship between DSM-5 severity ratings and other clinical features, particularly cognitive and behavioral functioning.

Methods

Participants and procedures

Participants consisted of 248 children and adolescents (ages 2–17 years, M = 6.4 years, SD = 4.0 years) with ASD enrolled in a larger study focused on DSM-5 criteria for ASD (Mazurek et al., 2017). All children received a com- prehensive diagnostic evaluation for autism at one of six Autism Treatment Network (ATN) sites: Children’s Hospital Los Angeles, Cincinnati Children’s Hospital Medical Center, Nationwide Children’s Hospital, University of Missouri, University of Pittsburgh Medical Center, and Vanderbilt University Medical Center. Each clinical diagnostic assessment was conducted in accord- ance with the standard ATN diagnostic process and included a review of records, a non-standardized diagnos- tic clinical interview, standardized observation using the Autism Diagnostic Observation Schedule–Second Edition (ADOS-2), cognitive assessment, and assessment of behavioral functioning. Additional measures were included when necessary on a case-by-case basis to further inform diagnostic determination. In total, 52% of the participants were assessed by a psychologist, 5.7% were assessed by a physician (i.e. developmental behavioral pediatrician, neu- rologist, pediatrician, or psychiatrist), and 42.3% were assessed by an interdisciplinary team (all teams included a psychologist and/or physician).

The study was approved by the Institutional Review Board at the clinical and data coordinating center at Massachusetts General Hospital and at each clinical site, and informed written consent from each family was obtained prior to participation. Families whose children were between the ages of 2 and 17 years 11 months and who were seen for an autism diagnostic evaluation were recruited for participation. Recruitment and enrollment continued until the target sample size was met. Only those meeting DSM-5 criteria for ASD were included in this study. Most children were male (82%) and Caucasian (76%), and most primary caregivers had received some post-secondary education (66%).

Measures

Demographics. Primary caregivers completed a demo- graphic questionnaire to report child age, sex, ethnicity, race, caregiver education level, and household income.

Autism symptom severity. The ADOS-2 (Lord et al., 2012) is a standardized diagnostic observational tool that assesses communicative behavior, social interaction skills, and repetitive behaviors and restricted interests. The ADOS-2 was administered at all sites by assessors with extensive experience and formal training on administration and scor- ing of the measure. The ADOS-2 comprises five different

470 Autism 23(2)

modules, one of which is selected for administration based on the child’s age and verbal ability. A continuous 10-point metric, the ADOS-2 calibrated severity score (CSS), has been developed as a measure of overall autism symptom severity (Esler et al., 2015; Gotham et al., 2009; Hus and Lord, 2014). The CSS was standardized to account for individual differences in age and language level. Higher CSS scores indicate greater symptom severity. Separate scores were calculated by domain: the social affect cali- brated severity score (SA-CSS) and the restricted and repetitive behavior calibrated severity score (RRB-CSS). Each domain score represents a continuous 10-point score that accounts for individual differences in age and lan- guage (Hus et al., 2014).

Two subscales from the Aberrant Behavior Checklist (ABC) (Aman and Singh, 1986) were included to assess parent-reported severity in social and repetitive behav- ior domains. The ABC is a 58-item caregiver-report questionnaire that measures current behavioral func- tioning across five empirically derived subscales. For the purpose of this study, the Social Withdrawal sub- scale (comprising 16 items assessing social isolation, withdrawal, and lack of social reciprocity) and the Stereotypic Behavior subscale (comprising seven items assessing repetitive behaviors and stereotyped move- ments) were examined as parent-report measures of symptom severity.

Intellectual ability. A range of measures were used across ATN sites to assess overall intelligence (Full Scale IQ), verbal intelligence (VIQ), and nonverbal intelligence (NVIQ). A small portion (10.9%) of the sample was administered a nonverbal measure of intelligence, the Leiter International Performance Scale—Third Edition, (Roid et al., 2013); therefore, only NVIQ scores were available for this subset of the sample. Intellectual testing could not be completed for 16.1% of the sample due to dif- ficulties participating or understanding task demands. As a result, valid Full Scale IQ scores were available for 181 children (73% of the total sample). Measures included the Stanford Binet Scales of Intelligence–Fifth Edition (24.6%) (Roid, 2003), the Wechsler Intelligence Scale for Children–Fourth Edition (3.6%) (Wechsler, 2003), the Wechsler Intelligence Scale for Children–Fifth Edition (6.9%) (Wechsler, 2014), the Wechsler Preschool and Pri- mary Scale of Intelligence–Third Edition (1.2%) (Wechsler, 2002), the Wechsler Abbreviated Scale of Intel- ligence–Second Edition (8.5%) (Wechsler, 2011), the Wechsler Adult Intelligence Scale—Fourth Edition (0.4%) (Wechsler, 2008), the Differential Ability Scales–Second Edition (3.6%) (Elliot, 2007), the Bayley Scales of Infant and Toddler Development–Third Edition (2%) (Bayley, 2006), or the Mullen Scales of Early Learning (MSEL, 22.2%) (Elliot, 2007). For those receiving the MSEL, the

Early Learning Composite Standard Score was used as a measure of Full Scale IQ.

Emotional and behavioral functioning. The Child Behavior Checklist (CBCL) (Achenbach and Rescorla, 2001) was administered to assess emotional and behavioral difficul- ties. The CBCL is a broad-band parent-report question- naire providing an overall assessment of symptoms (i.e. Total Problems score) as well as more specific summary and syndrome scales. Items are rated on a three-point scale (Not True to Very True). Two separate versions are available based on the child’s age, including younger (ages 1.5–5 years) and older (ages 6–18 years) versions. Although the specific syndrome scales differ across ver- sions, the Total and Internalizing and Externalizing Scale T-scores are comparable across versions. For this study, overall levels of both internalizing and externalizing problems were examined using Internalizing and Exter- nalizing composite T-scores. The Internalizing domain comprises mood and anxiety symptoms, while the Exter- nalizing domain includes behavioral problems, such as aggression and noncompliance.

Three additional subscales from the ABC (Aman and Singh, 1986) were included to assess additional challeng- ing behaviors, specifically: Irritability, Hyperactivity/ Noncompliance, and Inappropriate Speech.

DSM-5 checklist. After all diagnostic assessment proce- dures were conducted, clinicians completed a DSM-5 diagnostic checklist for each participant. The checklist contained seven symptoms grouped in two areas: (1) social communication deficits (three symptoms), and (2) RRBs (four symptoms). The clinician noted whether each symp- tom was “absent,” “present by history,” or “currently pre- sent,” consistent with DSM-5 descriptions (American Psychiatric Association, 2013). Additional checklist sec- tions included whether symptoms were present or absent in the early developmental period and whether impairment was present or absent. The checklist also included severity level ratings for both social communication and RRB on a three-point scale, consistent with DSM-5 criteria (Ameri- can Psychiatric Association, 2013).

Data analysis plan

Descriptive statistics (mean, standard deviation, range, and percentage) were calculated for demographic and pri- mary variables. To examine the distribution of DSM-5- defined social communication and RRB severity levels, cross-tabulation of the percentages at each severity level were calculated. The second and third research ques- tions were addressed by first conducting bivariate analyses to examine whether DSM-5 severity ratings were associ- ated with individual demographic (i.e. age and sex) or

Mazurek et al. 471

clinical features (i.e. ADOS-2 CSS domain scores, IQ score, internalizing symptoms, externalizing behaviors, and aberrant behaviors). DSM-5 severity scores are formally ordinal metrics with potentially unequal intervals between levels. We ran three models for each bivariate analysis and looked for agreement across the three models. The first model was a cumulative logistic regression model, which properly accounts for the variable intervals between levels but also assumes parallel cumulative odds across each pre- dictor. We tested the parallel cumulative odds assumption

with the proportional odds test. The second model was a binary logistic regression model, which dichotomized DSM-5 severity scores between requiring support versus requiring substantial or very substantial support. This divi- sion was selected because of the low prevalence of partici- pants scored as requiring very substantial support. The third model was a linear regression model, which assigned the values 1 through 3 to the three severity levels as a con- tinuous scale. The binary logistic model is correct but potentially less powerful than the cumulative logistic model. The cumulative logistic model is appropriate if the proportion odds assumption is met, but cumulative odds are difficult to communicate. The linear model is not for- mally correct, but the interpretation is easy. We focused on results for which there was agreement across all three models and thus an unambiguous conclusion of significant association. Future studies where power is more limited might choose to focus on inference from the cumulative logistic model for analyses of ordinal severity scales where the proportional odds assumption is met. Finally, we used cumulative and binary logistic and linear multiple regres- sion models to determine which clinical and demographic features were independent predictors of DSM-5 severity scores. For each model, we included all significant varia- bles from the bivariate models for each DSM-5 severity score.

Results

Demographic and clinical characteristics of the sample are presented in Table 1.

For DSM-5 social communication severity, 30% of the sample were rated as requiring support, 45% as requiring substantial support, and 25% as requiring very substantial support (Table 2). For DSM-5 RRB severity, 44% of the sample were rated as requiring support, 39% as requiring substantial support, and 17% as requiring very substantial support. In the cross-tabulation, 26% of the sample were rated as requiring support in both social communication and RRB domains and 28% were rated as requiring substantial support in both social communica- tion and RRB domains. Overall, social communication severity was greater than RRB severity (test for symme- try, p < 0.001), although there was substantial concord- ance between the two metrics (simple kappa = 0.52; 95% confidence interval (CI) 0.43, 0.60; p < 0.001). Basic sample characteristics across severity level are shown in Table 3.

Bivariate analyses

Greater social communication severity was associated with younger age, lower IQ, and higher ADOS-2 CSS scores in both social affect and RRB domains (Table 3).

Table 1. Demographic and clinical features.

% (n)

Sex Female 18.1 (45) Male 81.9 (203) Race Asian 1.2 (3) Black or African American 10.1 (25) Caucasian/White 76.2 (189) Other 7.3 (18) Ethnicity Hispanic/Latino 8.1 (20) Not Hispanic/Latino 85.9 (213) Parental education <High School 4.0 (10) High School 22.2 (55) Some College 33.9 (84) Bachelor’s degree 16.9 (42) Postgraduate 15.3 (38) Household income ⩽US$24,999 23.0 (57) US$25,000–49,999 22.6 (56) US$50,000–74,999 16.9 (42) US$75,000–99,999 12.1 (30) ⩾US$100,000 11.3 (28)

Mean (SD)

Age 6.4 (4.0); range: 2.0–17.6 IQ 76.1 (22.5); range: 33–127 Child Behavior Checklist (CBCL) Externalizing T-score 61.7 (11.6) Internalizing T-score 65.2 (9.6) Aberrant Behavior Checklist (ABC) Irritability 15.0 (10.0) Social Withdrawal 12.8 (8.7) Stereotypic Behavior 5.9 (4.9) Hyperactivity/Noncompliance 20.0 (11.8) Inappropriate Speech 3.7 (3.1) ADOS-2 SA-CSS 7.4 (1.9) ADOS-2 RRB-CSS 7.3 (2.1)

SA-CSS: social affect calibrated severity score, RRB-CSS: restricted and repetitive behavior calibrated severity score. ADOS-2: Autism Diagnostic Observation Schedule–Second Edition.

472 Autism 23(2)

Inferences of significant association were consistent (p < 0.05) for these variables across all three model types.

Greater RRB severity was associated with younger age, lower IQ, higher ABC Stereotypic Behavior subscale scores, higher ADOS-2 SA-CSS and RRB-CSS scores, and being male (Table 4). Inferences of significant associ- ation were consistent (p < 0.05) across all models for most of these variables, with the exception of ABC Stereotypic

Behavior, for which two out of three models indicated a statistically significant association (Table 5).

Multivariate analyses

Final multiple regression models indicated that age, IQ, and ADOS-2 SA-CSS were significant independent pre- dictors of social communication severity (p < 0.05 across

Table 2. Distribution of DSM-5 severity ratings across the total sample, n (%).

Restricted and repetitive behavior (RRB) severity Total

RRB Level 1 RRB Level 2 RRB Level 3

Social communication (SC) severity

SC Level 1 64 (25.8) 10 (4.0) 1 (0.4) 75 (30.2) SC Level 2 39 (15.7) 69 (27.8) 3 (1.2) 111 (44.8) SC Level 3 7 (2.8) 18 (7.3) 37 (14.9) 62 (25)

Total 110 (44.4) 97 (39.1) 41 (16.5) 248 (100)

Level 1 = requiring support, Level 2 = requiring substantial support, Level 3 = requiring very substantial support.

Table 3. Sample characteristics by severity level rating.

Restricted and repetitive behavior (RRB) severity

RRB Level 1 RRB Level 2 RRB Level 3

Social communication (SC) severity

SC Level 1 FSIQ = 91.5 (18.2) %FSIQ = 89 (57/64)

FSIQ = 95.8 (7.7) %FSIQ = 80 (8/10)

FSIQ = 61.0 (–) %FSIQ = 100 (1/1)

VIQ = 93.5 (19.3) %VIQ = 63 (40/64)

VIQ = 101.3 (8.3) %VIQ = 70 (7/10)

No VIQ (0/1)

NVIQ = 93.4 (19.6) %NVIQ = 72 (46/64)

NVIQ = 96.6 (8.4) %NVIQ = 90 (9/10)

No NVIQ (0/1)

Age = 9.5 (4.3) years Age = 7.4 (3.0) years Age = 10.8 (–) years % Module 1 = 6 (4/64) % Module 1 = 0 (0/10) % Module 1 = 0 (0/1)

SC Level 2 FSIQ = 78.1 (22.5) %FSIQ = 69 (27/39)

FSIQ = 69.6 (20.9) %FSIQ = 71 (49/69)

FSIQ = 73.7 (12.0) %FSIQ = 100 (3/3)

VIQ = 76.3 (24.4) %VIQ = 46 (18/39)

VIQ = 62.5 (16.3) %VIQ = 52 (36/69)

VIQ = 77.0 (18.4) %VIQ = 67 (2/3)

NVIQ = 83.5 (21.8) %NVIQ = 64 (25/39)

NVIQ = 76.9 (21.0) %NVIQ = 68 (47/69)

NVIQ = 83.0 (16.8) %NVIQ = 100 (3/3)

Age = 7.4 (4.3) years Age = 5.3 (2.9) years Age = 4.9 (1.2) years % Module 1 = 21 (8/39) % Module 1 = 45 (31/69) % Module 1 = 0 (0/3)

SC Level 3 FSIQ = 60.6 (15.1) %FSIQ = 71 (5/7)

FSIQ = 52.4 (6.3) %FSIQ = 39 (7/18)

FSIQ = 54.8 (9.2) %FSIQ = 65 (24/37)

VIQ = 54.5 (7.8) %VIQ = 29 (2/7)

VIQ = 46.7 (7.0) %VIQ = 39 (7/18)

VIQ = 52.8 (10.2) %VIQ = 49 (18/37)

NVIQ = 53.3 (6.6) %NVIQ = 57 (4/7)

NVIQ = 68.6 (20.8) %NVIQ = 67 (12/18)

NVIQ = 55.0 (10.9) %NVIQ = 54 (20/37)

Age = 3.6 (1.2) years Age = 4.7 (2.7) years Age = 3.5 (1.8) years % Module 1 = 57 (4/7) % Module 1 = 56 (10/18) % Module 1 = 57 (20/37)

IQ = M (SD) of Full Scale IQ (FSIQ), Verbal IQ (VIQ), and Nonverbal IQ (NVIQ) for each cell; % IQ = percentage and frequency of children for whom IQ was available; age = M (SD) of age for each cell; % Module 1 = percentage and frequency of children who were administered Module 1 of the ADOS-2 (intended for children with minimal verbal abilities).

Mazurek et al. 473

all models), and that age IQ, and ADOS RRB-CSS were significant independent predictors of RRB severity (p < 0.05 across all models) (Table 6).

Discussion

In our analysis of DSM-5 severity level ratings for ASD in a large sample of children and adolescents with ASD, we observed that 25% of children were rated as requiring support, the lowest severity, in both social communication

and repetitive behavior domains; 27% were rated as requiring substantial support, the intermediate severity, in both domains; and 15% were rated as requiring very sub- stantial support, the most severe symptoms, in both domains. Severity was largely consistent across domains, with only a handful of children receiving the lowest sever- ity ratings in one domain and most severe in the other. In general, social communication symptoms were rated at a higher level of severity than repetitive behaviors across the sample.

Table 4. DSM-5 social communication severity levels and clinical features: bivariate analyses.

Cumulative logit Binary logit Linear model

Odds ratio

95% CI p Odds ratio

95% CI p Slope estimate

95% CI p

Age 0.75 (0.70, 0.81) <0.001 0.77 (0.71, 0.83) <0.001 −0.093 (–0.113, –0.073) <0.001 Sex 0.90 (0.50, 1.63) 0.742 1.08 (0.54, 2.27) 0.827 −0.045 (–0.286, 0.197) 0.717 IQ 0.94 (0.92, 0.95) <0.001 0.94 (0.93, 0.96) <0.001 −0.019 (–0.023, –0.016) <0.001 CBCL Externalizing T-score 1.00 (0.98, 1.02) 0.965 0.99 (0.97, 1.02) 0.607 0.000 (–0.008, 0.009) 0.923 CBCL Internalizing T-score 0.98 (0.96, 1.01) 0.169 0.98 (0.95, 1.00) 0.100 −0.007 (–0.017, 0.003) 0.193 ABC Irritability 1.01 (0.98, 1.04) 0.644 1.00 (0.97, 1.03) 0.840 0.003 (–0.008, 0.014) 0.573 ABC Social Withdrawal 1.01 (0.97, 1.04) 0.689 1.00 (0.96, 1.03) 0.852 0.003 (–0.009, 0.015) 0.624 ABC Stereotypic Behaviora 1.05 (0.99, 1.12) 0.079 1.01 (0.95, 1.08) 0.758 0.020 (–0.001, 0.042) 0.063 ABC Hyperactivity 1.01 (0.99, 1.04) 0.343 1.01 (0.98, 1.03) 0.688 0.004 (–0.004, 0.013) 0.326 ABC Inappropriate Speech 0.94 (0.85, 1.03) 0.169 0.95 (0.86, 1.06) 0.368 −0.024 (–0.058, 0.010) 0.164 ADOS-2 SA-CSS 1.27 (1.11, 1.46) <0.001 1.31 (1.12, 1.54) <0.001 0.092 (0.042, 0.142) <0.001 ADOS-2 RRB-CSS 1.15 (1.03, 1.29) 0.018 1.16 (1.02, 1.32) 0.028 0.056 (0.012, 0.100) 0.014

CI: confidence interval; CBCL: Child Behavior Checklist; ABC: Aberrant Behavior Checklist; ADOS-2: Autism Diagnostic Observation Schedule– Second Edition; SA-CSS: social affect calibrated severity score; RRB-CSS: restricted and repetitive behavior calibrated severity score. aABC Stereotypic Behavior failed to meet the proportional odds assumption (p = 0.007).

Table 5. DSM-5 restricted and repetitive behavior severity levels and clinical features: bivariate analyses.

Cumulative logit Binary logit Linear model

Odds ratio

95% CI p Odds ratio

95% CI p Slope estimate

95% CI p

Age 0.76 (0.70, 0.82) <0.001 0.78 (0.72, 0.84) <0.001 –0.081 (–0.101, –0.060) <0.001 Sex 0.46 (0.24, 0.86) 0.017 0.46 (0.24, 0.88) 0.021 –0.285 (–0.519, –0.050) 0.018 IQ 0.96 (0.94, 0.97) <0.001 0.96 (0.94, 0.97) <0.001 –0.015 (–0.019, –0.011) <0.001 CBCL Externalizing T-Score 1.01 (0.99, 1.03) 0.533 1.00 (0.98, 1.02) 0.869 0.003 (–0.005, 0.011) 0.429 CBCL Internalizing T-Scorea 0.99 (0.96, 1.01) 0.388 0.98 (0.95, 1.00) 0.091 –0.002 (–0.012, 0.007) 0.643 ABC Irritability 1.02 (0.99, 1.05) 0.195 1.01 (0.98, 1.05) 0.354 0.008 (–0.002, 0.018) 0.131 ABC Social Withdrawal 0.99 (0.96, 1.02) 0.593 0.99 (0.95, 1.02) 0.408 –0.002 (–0.014, 0.010) 0.707 ABC Stereotypic Behavior 1.07 (1.00, 1.13) 0.036 1.06 (1.00, 1.13) 0.074 0.024 (0.003, 0.045) 0.026 ABC Hyperactivity 1.02 (0.99, 1.04) 0.134 1.02 (0.99, 1.04) 0.186 0.007 (–0.002, 0.016) 0.127 ABC Inappropriate Speech 1.02 (0.93, 1.11) 0.749 1.04 (0.94, 1.14) 0.468 0.002 (–0.032, 0.035) 0.919 ADOS-2 SA-CSS 1.20 (1.05, 1.38) 0.007 1.19 (1.03, 1.38) 0.019 0.071 (0.021, 0.121) 0.006 ADOS-2 RRB-CSSb 1.40 (1.22, 1.62) <0.001 1.45 (1.25, 1.70) <0.001 0.100 (0.058, 0.143) <0.001

CI: confidence interval; CBCL: Child Behavior Checklist; ABC: Aberrant Behavior Checklist; ADOS-2: Autism Diagnostic Observation Schedule– Second Edition; SA-CSS: social affect calibrated severity score; RRB-CSS: restricted and repetitive behavior calibrated severity score. aCBCL Internalizing Problems T-score failed to meet the proportional odds assumption (p = 0.005). bADOS-2 RRB-CSS failed to meet the proportional odds assumption (p = 0.049).

474 Autism 23(2)

Our findings indicate that clinician ratings of severity are consistent to some degree with both behavioral obser- vations and parental ratings of severity. Specifically, the results revealed significant associations between both social communication and RRB severity ratings and respective ADOS-2 domain scores. Significant associa- tions were also observed between RRB severity ratings and parent-reported symptoms of stereotyped behavior on the ABC. By contrast, parental ratings of social with- drawal on the ABC were not associated with DSM-5 rat- ings of social communication severity. The ABC subscales included in this study provide a narrow assessment of very specific types of RRB and social communication. Thus, future studies should include more comprehensive parent-report measures of the full range of both RRB and social communication functioning. It is also noteworthy that parent-reported behavioral and emotional problems were not significantly associated with DSM-5 symptom severity in either domain, providing some evidence that clinicians are not basing their severity ratings on general behavioral or emotional problems.

The results also revealed that intellectual functioning was strongly associated with both social communication and RRB severity ratings. Children with lower IQ had sig- nificantly greater clinician-rated severity in both domains. It could be the case that children who were more signifi- cantly affected by autism were also more likely to have global cognitive or developmental impairment. Alterna- tively, intellectual impairment may contribute indepen- dently to social communication deficits and repetitive behaviors above and beyond the effects of core ASD symptoms alone. Thus, the DSM-5 symptom severity rat- ings may reflect the combined manifestation of both symp- tom-specific and global developmental impairment. Given the wording of the new DSM-5 severity level descriptors (e.g. “requiring support”), clinicians may also have diffi- culty determining whether to assign ratings based on ASD

symptom severity alone (more consistent with text exam- ples) or based largely on need for support (more consistent with the level descriptors). If clinicians adhere to the latter interpretation, there may be greater potential for confla- tion of intellectual and symptom-related impairment. This poses problems for both inter-rater reliability and con- struct validity. Without more specific guidance, clinicians are likely to vary in the extent to which they classify severity based on domain-specific deficits, cognitive impairments, or need for support in activities of daily liv- ing. As shown in a recent descriptive study of children with ASD, there is a potential for significant discrepancy in severity classification depending on the measure and construct (Weitlauf et al., 2014). Further research is needed to better understand clinician decision-making and interpretation of the intended construct assessed by these new DSM-5 specifiers.

Age was also found to be inversely associated with DSM-5 symptom severity in both social communication and RRB domains. This is difficult to interpret within the context of this study because of the potential for sampling bias and may not reflect a true decrease in ASD severity with age. Children were recruited and enrolled into this study based on referral for autism diagnostic assessments. It is likely that individuals who were referred for an initial diagnostic assessment in adolescence generally had more subtle symptom presentation than those who were referred in early childhood. This would be consistent with prior research finding an inverse relationship between autism symptom severity and age at first diagnosis (Mazurek et al., 2014; Wiggins et al., 2006). Because of this, it is likely that the adolescents in our study had less severe symptoms than the larger population of adolescents with ASD. To fully examine the associations between age and DSM-5 symp- tom severity indicators, it would be most informative to enroll a broader population of individuals with ASD, not only those seen at the time of initial diagnosis.

Table 6. DSM-5 severity levels and clinical features: final multiple regression models.

Cumulative logit Binary logit Linear model

Odds ratio

95% CI p Odds ratio

95% CI p Slope estimate

95% CI p

Outcome variable: social communication severity level Age 0.82 (0.73, 0.91) <0.001 0.82 (0.72, 0.93) 0.002 –0.054 (–0.081, –0.028) <0.001 IQ 0.95 (0.93, 0.96) <0.001 0.95 (0.93, 0.97) <0.001 –0.015 (–0.019, –0.010) <0.001 ADOS-2 SA-CSS 1.30 (1.08, 1.57) 0.006 1.47 (1.16, 1.89) 0.002 0.065 (0.018, 0.113) 0.007 Outcome variable: restricted and repetitive behavior severity level Age 0.83 (0.74, 0.91) <0.001 0.83 (0.74, 0.93) 0.002 –0.046 (–0.072, –0.021) <0.001 IQ 0.97 (0.96, 0.99) 0.002 0.98 (0.96, 0.99) 0.014 –0.008 (–0.013, –0.004) <0.001 ADOS-2 RRB-CSS 1.69 (1.37, 2.12) <0.001 1.80 (1.42, 2.38) <0.001 0.109 (0.066, 0.153) <0.001

CI: confidence interval; CBCL: Child Behavior Checklist; ABC: Aberrant Behavior Checklist; ADOS-2: Autism Diagnostic Observation Schedule– Second Edition; SA-CSS: social affect calibrated severity score; RRB-CSS: restricted and repetitive behavior calibrated severity score.

Mazurek et al. 475

Limitations and future directions

As the first study of this type, the current findings provide an important first examination of the clinical application of DSM-5 ASD severity ratings across a large and well-charac- terized sample. The sample spanned a wide range of func- tioning and was typical of the male:female ratio found in population studies of ASD (Centers for Disease Control and Prevention (CDC), 2014). However, several factors may limit generalizability to the larger ASD population. First, the centers participating in our study were all located at aca- demic medical centers and specialize in ASD diagnosis, treatment, and research. Thus, the clinicians in our study may not be representative of the larger population of clinicians practicing in community-based or other settings. Future research should examine how clinicians in different settings may be using these DSM-5 severity ratings. It would also be informative to evaluate potential differences in clinical deci- sion-making across professional disciplines, as well as inter- rater reliability in assignment of severity level ratings.

Additional measurement limitations should also be con- sidered. First, we chose to include ADOS-2 CSS scores rather than raw scores because they were specifically designed to account for individual differences in age and language level. However, it should be noted that these CSS scores still do not fully account for the associations between autism symptoms and age and language. In addition, although ADOS-2 assessments were overseen by research- reliable clinicians at each site, we did not specifically track whether all assessments were directly administered by research-reliable clinicians. Another limitation is that we did not collect data related to adaptive functioning. Although many clinicians administered adaptive measures as part of their clinical evaluations, these data were not collected dur- ing this study. In the future, it would be informative to eval- uate the extent to which adaptive functioning correlates with clinician ratings of symptom severity. Overall, the current findings suggest that further guidance and more specific operational definitions may be helpful for clinicians assign- ing these new DSM-5 severity level ratings.

Acknowledgements

The authors are extremely grateful to all the families and clinicians who participated in this study.

Declaration of conflicting interests

Dr M.O.M has received research support from National Institute of Mental Health (NIMH), Autism Speaks, and Health Resources and Services Administration (HRSA). Ms F.L. has received research support from Autism Speaks and HRSA. Dr E.A.M. serves as a DSMB member for Acorda Therapeutics and Shire Human Genetic Therapies and receives research support from Adolph Coors Foundation, ALS Association, ALS Finding a Cure, Autism Speaks, Biotie Therapies, Michael J Fox Foundation, FDA, HRSA, NIH, and PCORI. Dr B.L.H. has received research

support from Curemark, Neuropharm, Lilly, Forest, Bristol Myers Squibb, Roche, Pediamed, Pfizer, and Autism Speaks.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This network activity was supported by Autism Speaks and coopera- tive agreement UA3 MC11054 through the US Department of Health and Human Services, Health Resources and Services Administration, Maternal and Child Health Research Program to the Massachusetts General Hospital. This work was conducted through the Autism Speaks Autism Treatment Network.

ORCID iD

Micah O Mazurek https://orcid.org/0000-0001-7715-6538

References

Achenbach TM & Rescorla L (2001) Manual for the ASEBA school-age forms & profiles: an integrated system of multi-informant assessment. Burlington, VT: University of Vermont, Research Center for Children, Youth & Families.

Aman M and Singh N (1986) Aberrant Behavior Checklist: Manual. East Aurora, NY: Slosson Educational Publications.

American Psychiatric Association (2013) Diagnostic and Statistical Manual of Mental Disorders (DSM-5). 5th ed. Washington, DC: APA.

Bayley N (2006) Bayley Scales of Infant and Toddler Development. 3rd ed. San Antonio, TX: Harcourt Assessment, Inc.

Buxbaum JD and Baron-Cohen S (2013) DSM-5: the debate con- tinues. Molecular Autism 4(1): 11.

Centers for Disease Control and Prevention (CDC) (2014) Prevalence of autism spectrum disorder among children aged 8 years—autism and developmental disabilities moni- toring network, 11 sites, United States, 2010. MMWR Surveill Summ 63(2): 1–21.

Elliot C (2007) Differential Abilities Scale—2nd Edition (DAS-II) Manual. 2nd ed. San Antonio, TX: Harcourt Assessment, Inc.

Esler AN, Bal VH, Guthrie W, et al. (2015) The autism diagnostic observation schedule, toddler module: Standardized sever- ity scores. Journal of Autism and Developmental Disorders 45(9): 2704–2720.

Gotham K, Pickles A and Lord C (2009) Standardizing ADOS scores for a measure of severity in autism spectrum disor- ders. Journal of Autism and Developmental Disorders 39(5): 693–705.

Grzadzinski R, Huerta M and Lord C (2013) DSM-5 and autism spectrum disorders (ASDs): an opportunity for identifying ASD subtypes. Molecular Autism 4(1): 12.

Halfon N and Kuo AA (2013) What DSM-5 could mean to chil- dren with autism and their families. JAMA Pediatrics 167(7): 608–613.

Hus V, Gotham K and Lord C (2014) Standardizing ADOS domain scores: separating severity of social affect and restricted and repetitive behaviors. Journal of Autism and Developmental Disorders 44: 2400–2412.

Hus V and Lord C (2014) The autism diagnostic observation schedule, module 4: Revised algorithm and standardized

476 Autism 23(2)

severity scores. Journal of Autism and Developmental Disorders 44(8): 1996–2012.

Kulage KM, Smaldone AM and Cohn EG (2014) How will DSM-5 affect autism diagnosis? A systematic literature review and meta-analysis. Journal of Autism and Developmental Disorders 44(8): 1918–1932.

Lord C and Bishop SL (2015) Recent advances in autism research as reflected in DSM-5 criteria for autism spec- trum disorder. Annual Review of Clinical Psychology 11: 53–70.

Lord C, Rutter M, DiLavore PC, et al. (2012) Autism Diagnostic Observation Schedule, Second Edition (ADOS-2) Manual (Part 1): Modules 1–4. 2nd ed. Torrance, CA: Western Psychological Services.

Mazurek MO, Handen BL, Wodka EL, et al. (2014) Age at first autism spectrum disorder diagnosis: The role of birth cohort, demographic factors, and clinical features. Journal of Developmental and Behavioral Pediatrics 35(9): 561–569.

Mazurek MO, Lu, Symecko H, et al. (2017) A prospective study of the concordance of DSM-IV and DSM-5 diagnostic cri- teria for autism spectrum disorder. Journal of Autism and Developmental Disorders 47(9): 2783–2794.

Mehling MH and Tassé MJ (2016) Severity of autism spectrum disorders: current conceptualization, and transition to DSM- 5. Journal of Autism and Developmental Disorders 46(6): 2000–2016.

Ozonoff S (2012a) Editorial: DSM-5 and autism spectrum disorders—two decades of perspectives from the JCPP. Journal of Child Psychology and Psychiatry 53(9): e4–e6.

Ozonoff S (2012b) Editorial perspective: autism spectrum dis- orders in DSM-5—an historical perspective and the need

for change. Journal of Child Psychology and Psychiatry 53(10): 1092–1094.

Roid GH (2003) Stanford-Binet Intelligence Scales. 5th ed. Itasca, IL: Riverside Publishing.

Roid GH, Miller LJ, Pomplun M, et al. (2013) Leiter-3: Leiter International Performance Scale. Torrance, CA: Western Psychological Services.

Smith IC, Reichow B and Volkmar FR (2015) The effects of DSM-5 criteria on number of individuals diagnosed with autism spectrum disorder: a systematic review. Journal of Autism and Developmental Disorders 45(8): 2541–2552.

Volkmar RF and Reichow B (2013) Autism in DSM-5: progress and challenges. Molecular Autism 4(1): 13.

Wechsler D (2002) Wechsler Preschool and Primary Scale of Intelligence. 3rd ed. San Antonio, TX: Psychological Corporation.

Wechsler D (2003) Wechsler Intelligence Scale for Children. 4th ed. San Antonio, TX: Psychological Corporation.

Wechsler D (2008) Wechsler Adult Intelligence Scale. 4th ed. San Antonio, TX: Psychological Corporation.

Wechsler D (2011) Wechsler Abbreviated Scale of Intelli- gence (WASI-II). 2nd ed. San Antonio, TX: Psychological Corporation.

Wechsler D (2014) Wechsler Intelligence Scale for Children. 5th ed. San Antonio, TX: NCS Pearson.

Weitlauf AS, Gotham K, Vehorn AC, et al. (2014) Brief report: DSM-5 “levels of support”: a comment on discrepant con- ceptualizations of severity in ASD. Journal of Autism and Developmental Disorders 44(2): 471–476.

Wiggins LD, Baio J and Rice C (2006) Examination of the time between first evaluation and first autism spectrum diagno- sis in a population-based sample. Journal of Developmental and Behavioral Pediatrics 27(2): S79–S87.