Review an article

� �

THEJOURNALOFCHILD "C._, PSYCHOLOGYANDPSYCHIATRY ~

Journal of Child Psychology and Psychiatry 54:2 (2013), pp 178–185 doi:10.1111/j.1469-7610.2012.02607.x

Diagnosis of autism spectrum disorders in 2-year-olds: a study of community practice

Christina M. Corsello,1 Natacha Akshoomoff,1,2 and Aubyn C. Stahmer1,2 1Rady Children’s Hospital San Diego, San Diego, CA; 2University of California, San Diego, CA, USA

Background: Longitudinal research studies have demonstrated that experienced clinicians using standardized assessment measures can make a reliable diagnosis of autism spectrum disorders (ASDs) in children under age 3. Limited data are available regarding the sensitivity and specifcity of these measures in community settings. The aims of this study were to determine how well a standardized diagnostic observational measure (Autism Diagnostic Observation Schedule – ADOS) functions alone, and with a brief parent measure within a community setting when administered by community clinicians. Methods: Clinical records for 138 children between the ages of 24 and 36 months of age who were evaluated for possible ASD or social/language concerns at a hospital-based developmental evaluation clinic were examined. Evaluations were conducted by community-based clinical psycholo- gists. Classifcation results obtained from standardized diagnostic measures were compared with case reviewer diagnosis, by reviewers blind to scores on diagnostic measures, using The Records-based Methodology for ASD Case Defnition that was developed by the Metropolitan Atlanta Developmental Disabilities Surveillance Program. Results: When compared with case review diagnosis, the ADOS demonstrated strong sensitivity and specifcity for both Autism versus Not Autism and ASD versus Nonspectrum (NS) diagnoses in this young sample. The Social Communication Questionnaire (SCQ), using the lower cutoff of ‡12, had adequate sensitivity when differentiating Autism from Not Autism, but weak sensitivity when differentiating ASD from NS, missing about 80% of the children with pervasive developmental disorder – not otherwise specifed. Using either the Modifed Checklist for Autism in Toddlers or the SCQ in combination with the ADOS did not result in improved specifcity over the ADOS alone and led to a drop in sensitivity when differentiating ASD from NS disorders. Conclusions: These results demonstrate that following best practice guidelines, the ADOS can be successfully incorporated into clinical practice with relatively good sensitivity and specifcity, and worked well with a referred sample of 2-year-olds. A parent questionnaire did not lead to any improvement in diagnostic classifcation above the ADOS used in isolation. Keywords: Autism, diagnosis, services research.

There are relatively few standardized diagnosticIntroduction measures that have demonstrated strong discrimi-

As awareness of the early signs of autism spectrum native ability in young children and predictive validity

disorders (ASD) increases, more physicians routinely over time (Lord et al., 2006). Two of the most well

screen for the presence of the disorder and children validated and commonly used measures in research

are referred to community diagnostic clinics with studies are the Autism Diagnostic Observation

suspicion of the disorder at younger ages. The Schedule (ADOS; Lord, Rutter, DiLavore, & Risi,

American Academy of Pediatrics has suggested rou- 2001) and the Autism Diagnostic Interview – Revised

tine screening for ASD at the 18- and 24-month well (ADI-R; Rutter, Le Couteur, and Lord, 2003). These

child visits (Johnson et al., 2007), which can lead to measures are considered the gold standard in re-

an increase in referrals for diagnosis at early ages. search protocols and have become widely known

Several studies have found that ASD can be reliably among clinical professionals. Both are available

diagnosed in children under 3 years of age by expe- through a US Publisher, but only the ADOS has

rienced, highly trained clinicians in specialty clinic become widely used by clinicians because the length

and research settings (Charman et al., 2005; Lord of the ADI-R makes it diffcult to incorporate into

et al., 2006; Turner & Stone, 2007) and that the clinical practice. In addition, studies using the ADI-R

greatest accuracy in diagnosis of young children is with children under 3 years of age have reported

achieved when using a standardized parent inter- mixed results, with several reporting an under-iden-

view and a standardized observational measure in tifcation of children with ASD primarily due to less

combination with clinical judgment (Corsello et al., children meeting criteria on the restricted behavior

2007; Lord et al., 2006; Risi et al., 2006). Given the area (Ventola et al., 2006; Wiggins & Robins, 2008).

emphasis on appropriate early intervention and the Although the ADOS is widely available and recom-

expense of intensive early intervention services, mended as one of the appropriate standardized

diagnostic accuracy is crucial. measures recommended in several Best Practice Guidelines as an appropriate standardized diagnostic

Confict of interest statement: No conficts declared. observation tool (California Department of Develop-

2012 The Authors. Journal of Child Psychology and Psychiatry 2012 Association for Child and Adolescent Mental Health. Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main St, Malden, MA 02148, USA

� �

doi:10.1111/j.1469-7610.2012.02607.x Diagnosis of autism spectrum disorders in 2-year-olds 179

mental Services, 2002; Filipek et al., 1999, 2000; National Research Council, 2001), little is known about the effectiveness of this measure when admin- istered by community clinicians who have not met the research training requirements and are not using the ADOS in combination with the ADI-R. The results from studies of the ADOS conducted by community- based clinicians who have achieved reliability are variable depending on clinic type and population evaluated. In an ASD specifc clinic, the ADOS dem- onstrated good sensitivity and specifcity in a sample of 2- to 8-year-old children (Mazefsky & Oswald, 2006). On the other hand, the ADOS was not as spe- cifc and resulted in more false-positive ASD classif- cations in a complicated community-based general developmental clinic sample of 2- to 16-year-old children (Molloy, Murray, Akers, Mitchell, & Man- ning-Courtney, 2011).

Clinicians working outside an ASD specialty clinic or research setting often have less rigorous training on standardized diagnostic measures, see a lower percentage of children with ASD, and are more likely to evaluate children who are diagnostically complex (Molloy et al., 2011). Clinic-referred samples may be more likely to include children from minority back- grounds, parents with a lower level of education, and families with more disadvantaged circumstances, all factors that have been associated with increased age of diagnosis (Mandell, Ittenbach, Levy, & Pinto- Martin, 2007; Mandell, Novak, & Zubritsky, 2005; Shattuck et al., 2009). Complicated diagnostic pre- sentations, combined with the often limited training of community clinicians on research based mea- sures, may affect measurement accuracy when used in community programs. In addition, most studies on community-based samples compare the classif- cation on the measure with clinical diagnosis that included the scores and observations from the measures. This decreases the objectivity and inde- pendence of the clinical diagnosis. The CDC has developed a method for giving a diagnosis based on a record review (Centers for Disease Control, 2007). This method allows for an outside reviewer to provide a diagnosis without scores or classifcations on measures.

The ADI-R is also available, but is less commonly used in clinical practice given the diffculty of obtaining insurance funding for the lengthy inter- view. Aside from the ADI-R, there are very few diag- nostic measures based on parent report. For this reason, many clinicians constrained by funding have resorted to using screening measures as well as parent questionnaires to obtain standardized parent information to augment their clinical interview. One such screening measure, the Social Communication Questionnaire (SCQ; Rutter, Bailey, and Lord, 2003), is based on the ADI-R and is more widely utilized given its brevity. The SCQ has been demonstrated to discriminate ASD from Nonspectrum (NS) diagnoses in children under age 5 with a reduced cutoff (Cor-

sello et al., 2007), but little is known about how accurate it is with children under age 3. Another screening measure, the Modifed Checklist for Aut- ism in Toddlers (M-CHAT; Robins, Fein, Barton, & Green, 2001) is widely used in children under the age of 30 months (Kleinman et al., 2008; Pandey et al., 2008; Robins et al., 2001). Although the M- CHAT can accurately identify many children with ASD, some children without an ASD screen positive for the disorder (Pandey et al., 2008), especially when a follow-up interview is not employed, which is often the case in community clinics (Pandey et al., 2008; Robins & Dumont-Mathieu, 2006). Despite this weakness, without other options clinicians may turn to this screener when evaluating very young children referred for an ASD.

Most of the evidence for making an accurate diagnosis in young children is based on studies conducted with experienced and highly trained staff in research settings. We investigated whether these results can be replicated within a community-based sample in a general developmental evaluation clinic with very young children when the clinician, in this case a clinical psychologist, has not received inten- sive training on standardized measures. This study was designed to address the following questions: (a) How well does the ADOS perform when used by community clinicians in a young community-based sample? (b) Does using a brief parent measure (M-CHAT or SCQ) in combination with the ADOS improve diagnostic accuracy in this young community sample?

Method Subjects

The sample consisted of 138 consecutive children between the ages of 24 and 36 months evaluated for an ASD at a children’s hospital developmental evaluation clinic, between October 2005 and August 2007. IQ was available for all but three of the children in the autism group. SCQ scores were available for 67 children, M- CHAT scores were available for 38 children (seven of these children also had the SCQ), and ADOS algorithm scores (communication + social + restricted and repet- itive + play) were available for 94 children (see Table 1). There was not a signifcant difference between diag- nostic groups in terms of age, F(2, 135) = 1.02, p = .36. There was a signifcant difference in IQ scores between diagnostic groups, F(2, 132) = 6.72, p < .01, with the Autism (AUT) group scoring signifcantly lower than both the pervasive developmental disorder – not other- wise specifed (PDD-NOS) and Nonspectrum (NS) groups.

There was not a signifcant difference in gender be- tween diagnostic groups (v 2 (2) = .53). Consistent with the 4:1 male to female ratio typical of children with ASD, male subjects comprised 86% of the AUT group and 90% of the PDD-NOS group. Interestingly, there were many more male subjects (81%) than female subjects in the NS group as well. There was not a signifcant difference between diagnostic groups in

2012 The Authors. Journal of Child Psychology and Psychiatry 2012 Association for Child and Adolescent Mental Health.

� �

180 Christina M. Corsello, Natacha Akshoomoff, and Aubyn C. Stahmer J Child Psychol Psychiatry 2013; 54(2): 178–85

Table 1 Age, IQ score, SCQ score, and ADOS total score by and interests (Rutter, Bailey, et al., 2003). Each item is clinical diagnosis checked as ‘yes’ or ‘no,’ and is assigned a point rating

Clinical diagnosis of ‘1’ (presence of abnormal behavior) or ‘0’ (absence of abnormal behavior). The frst item is not included in

Autism PDD-NOS Nonspectrum the scoring, as it indicates if the child has suffcient

(n = 56) (n = 50) (n = 32) verbal skills for language items to be scored. If the child is not scored as verbal, the six language items

Age (months) M 29.77 30.58 30.50 are skipped. The points are summed and result in a SD 3.16 3.39 2.82

24–35 24–35 24–35 total possible score of 0–33 for nonverbal children or

Range IQaM 63.38 73.20 76.28

0–39 for verbal children. Totals are compared to a

SD 4.18 17.58 21.76 cutoff of ‡15 for ASD. A lower cutoff score of ‡12 has Range 24–102 25–112 13–97 been suggested for children under the age of 5 years

SCQbM 17.16 12.44 9.17 (Corsello et al., 2007). There are two different versions

SD 6.16 6.35 5.38 of the SCQ: (a) a ‘current’ version designed for children Range 3–29 3–24 1–19 under the age of 5 years and (b) a ‘lifetime’ version

ADOSbM 23.74 14.08 8.25 designed for children 5 years of age or older, with all SD 3.72 6.06 6.25 questions based on lifetime or past behavior. Given Range 15–30 3–27 2–22 their young age, the current version was used for the

ADOS, autism diagnostic observation schedule; PDD-NOS, children in this study.

pervasive developmental disorder – not otherwise specifed; The ADOS (Lord et al., 2001) is a standardized

SCQ, social communication questionnaire; SD, standard devi- observational diagnostic measure. It is organized into

ation. four separate modules, based on the age and expressive aSignifcant difference between autism and nonspectrum language level of the child, ranging from preverbal groups. toddlers to verbally fuent adults. A child meets criteria bSignifcant difference between all three groups. for a classifcation of autism if the scores in the social

and communication domains and the total on the algorithm meet or exceed cutoff scores. All children in

2terms of race, (v (8) = .58) with Caucasian children this study were administered either module 1 or 2 of the ADOS. In this study, the majority of examiners had

comprising 41% of the sample, and a much smaller not achieved interrater reliability, although all had percentage of African American (2%) and Asian (3%) attended a clinical 2-day workshop on the measure. children. Fifteen percent of the sample identifed This was considered to be representative of the use of themselves as ‘Other’ and race remain unknown for this measure in a community clinic. ADOS total algo- 39% of the sample. There were a large percentage of rithm scores (communication + social + restricted and Hispanic children included in this sample (28%), which repetitive + play) were used for data analyses. is consistent with the Hispanic population in San Diego One of four developmental/cognitive measures was County. The majority of the sample was English also administered: the Mullen Scales of Early Learning speaking (72%). The remainder of the sample was (Mullen, 1995), the Bayley Scales of Infant Development bilingual (15%), Spanish speaking only (9%) or spoke – Second Edition (BSID-II; Bayley, 1993) or Bayley another language (4%). Twenty three percent of the Scales of Infant and Toddler Development – Third Edi- children in this study were enrolled in Medicaid. tion (Bayley-III; Bayley, 2005) or the Wechsler Preschool

and Primary Scales of Intelligence – Third Edition

Measures (WPPSI-III; Wechsler, 2002). In this study, Full Scale IQ scores consisted of the Early Learning Composite from

The M-CHAT is a 23-item, parent completed, screening the Mullen, the Full Scale IQ score from the WPPSI-III, questionnaire (Robins et al., 2001), including items the Mental Development Index from the BSID-II, or the assessing early communication and social behaviors mean of the Language and Cognitive standard scores as well as atypical behaviors suggestive of an ASD. from the Bayley-III. If the child’s scores were below the Each item is checked as ‘yes’ or ‘no,’ indicating lowest standard or scaled scores available, ratio IQs abnormality depending on the question. Six of these were calculated by taking the mean of the age equiva- items are identifed as critical items. A child’s score is lents on the measure and dividing by the child’s chro- suggestive of a possible ASD if it includes endorsement nological age. of abnormality on any two of the six critical items or The Records-based Methodology for ASD Case Def- any three of the 23 total items. The M-CHAT follow-up nition was developed by the Metropolitan Atlanta interview was not used as part of the evaluations be- Developmental Disabilities Surveillance Program cause data was collected prior to the publication of (MADDSP) of the Centers for Disease Control and Pre- studies indicating that specifcity was improved with vention. Records of children with possible ASD were the interview. The M-CHAT was used as a part of the reviewed and coded by clinicians and a determination of evaluation and clinical interviews were conducted with autism or ASD ‘caseness’ was made (Centers for Dis- the family. ease Control, 2007). The MADDSP ASD coding scheme

The SCQ is a 40-item, parent completed, screening includes operational defnitions of each diagnostic cri- questionnaire, based on the initial mandatory probes terion related to autism and each criterion is rated as from the original ADI (Le Couteur et al., 1989). It present or not present in a report. These are used to covers the areas of communication, reciprocal social determine if a case meets Diagnostic and Statistical interactions, and restricted and repetitive behaviors Manual, Fourth Edition, Text Revision (DSM-IV-TR)

2012 The Authors. Journal of Child Psychology and Psychiatry 2012 Association for Child and Adolescent Mental Health.

� �

doi:10.1111/j.1469-7610.2012.02607.x Diagnosis of autism spectrum disorders in 2-year-olds 181

criteria for a specifc ASD diagnosis. The reviewer determines a classifcation of ASD or not based on the information included in the report.

Procedure

Reports were reviewed for 138 children. Permission to review these records was obtained under the approval of the University of California, San Diego and Rady Children’s Hospital Human Research Protections Pro- grams. Each child was evaluated by one of the eight clinical psychologists in the clinic. The majority of children received the ADOS, either the M-CHAT or the SCQ, and a developmental assessment using either the BSID-II (N = 42), Bayley-III (N = 67), Mullen (N = 14) or WPPSI-III (N = 3). The clinicians had all attended a 2-day training on the ADOS and had consultation from one of the authors (CC or NA) who was research reli- able on the measure. The clinicians did not attend a research workshop or achieve interrater reliability on the measure. For 11 children, a developmental test score was not available and the Adaptive Behavior Composite of the Vineland Adaptive Behavior Scales, Second Edition (Vineland; Sparrow, Cicchetti, & Balla, 2005) was used instead. Each child was given a diagnosis by the psychologist conducting the evalua- tion who then wrote a clinical report that included a summary of the assessment with developmental scores and diagnostic classifcations on the standardized measures.

Written clinical reports were coded using the MAD- DSP ASD coding system. The record reviewers (CC and NA) were blind to scores on the measures and fnal clinical diagnosis when coding reports, with the goal of listing diagnostic criteria met and providing a second diagnosis that was not infuenced by scores on the measures. Report reviewers frst reviewed reports pro- vided by the CDC along with coding keys and estab- lished reliability at an 80% agreement for symptoms and at a 90% level for overall diagnosis. Thirty percent of the reports were double coded and reliability was maintained at 80% or better for diagnostic criteria and at 90% or better for overall diagnosis.

Design

We evaluated the diagnostic validity of the measures by comparing the classifcation of the measures alone and in combination with reviewer diagnosis. The discrimi- native validity of the measures was compared alone and in combination by examining the sensitivity (the ability to accurately classify children with an autism or ASD diagnosis: True Positives/(True Positives + False Nega- tives) and specifcity (the ability to accurately exclude children without an autism or ASD diagnosis: True Negatives/(True Negatives + False Positives). Compari- sons were made between children with autism (AUT) and those without autism (Not AUT: PDD-NOS and NS disorders), as well as between children with ASD (including Autism and PDD-NOS) and NS. Throughout the analyses, the groups will be referred to in these ways. Clinician and reviewer diagnoses were compared and diagnostic stability was reviewed for the subset of children who returned for a reevaluation as another check of the accuracy of the initial diagnosis.

Results Diagnostic validity of standardized measures

To determine the validity of the ADOS and SCQ in this community sample, the classifcations on the measures were compared with reviewer diagnosis, blinded to scores and classifcations on the mea- sures as well as clinician diagnosis, based on a review and coding of the written report. When compared with reviewer diagnosis, the ADOS dem- onstrated strong sensitivity and specifcity for both AUT versus Not AUT and ASD versus NS diagnoses (see Table 2). The SCQ, using the lower cutoff of ‡12 had adequate sensitivity when differentiating AUT from Not AUT, but weak sensitivity when differenti- ating ASD from NS, missing about 80% of the chil- dren with PDD-NOS. Consistent with previous studies, the M-CHAT captured the majority of the children with PDD-NOS and AUT in this 2-year-old sample, but also included 30% of the NS children. Using either the M-CHAT or the SCQ in combination with the ADOS did not result in improved specifcity over the ADOS alone and led to a dramatic drop in sensitivity for the ASD versus NS group primarily because the SCQ missed so many of the children with PDD-NOS (See Figure 1). Figure 1 depicts the percentage of children meeting the cutoff scores on each measure compared with their reviewer-identi- fed diagnosis.

To determine if age may have affected the diag- nostic discrimination of these two measures, the M-CHAT was evaluated for the younger age group for which it is recommended (30 months and under) and the SCQ evaluated for the children older than 30 months. When discriminating between ASD and NS in this younger group, the sensitivity (.95) and specifcity (.75) improved for the M-CHAT. However, the NS sample was extremely small (N = 4). For the SCQ, neither sensitivity (.63) nor specifcity (.33) improved when discriminating ASD from NS diag- noses in children over 30 months of age.

Table 2 Classifcation of autism (AUT) or not autism (Not AUT) and ASD or nonspectrum (NS) on screening and diagnostic measures compared to reviewer diagnosis

n Sensitivity Specifcity

AUT versus Not AUT ADOS 118 .84 .80 SCQ ‡ 12 67 .82 .64 M-CHAT 38 .88 .33 M-CHAT or SCQ & ADOS 84 .82 .65

ASD versus NS ADOS 118 .97 .85 SCQ ‡ 12 67 .69 .56 M-CHAT 38 .86 .67 M-CHAT or SCQ & ADOS 84 .76 .85

ADOS, autism diagnostic observation schedule; M-CHAT, modifed checklist for autism in toddlers; SCQ, social commu- nication questionnaire.

2012 The Authors. Journal of Child Psychology and Psychiatry 2012 Association for Child and Adolescent Mental Health.

182 Christina M. Corsello, Natacha Akshoomoff, and Aubyn C. Stahmer J Child Psychol Psychiatry 2013; 54(2): 178–85

100

90

80

70

60

50 ADOS Autism

40 ADOS ASD

30 ADOS NS

20

10

0 Autism PDD-NOS NS

Reviewer diagnosis

Diagnostic stability

Diagnosis often becomes clearer as children grow older (Lord et al., 2006). Diagnosis at follow-up appointments for the small number of children who were reevaluated was reviewed and compared with initial clinical diagnosis, rather than reviewer diag- nosis, as another gauge of diagnostic accuracy. Of the 138 children included in this study, 23% or 32 children were seen for a reevaluation when they were between 35 and 101 months of age, with 72% of the children falling between 4 and 6 years of age. Sev- enteen (53%) of the children remained in the same diagnostic category. As predicted, the AUT and NS diagnoses remained the most stable, with only two shifting from autism to PDD-NOS and two shifting from NS to ASD (one to autism and one to PDD-NOS). The PDD-NOS diagnoses had the most change, with four remaining in the PDD-NOS group, fve moving to autism and fve moving to NS.

All the 11 children who were initially diagnosed with AUT and later reevaluated met criteria for AUT on the ADOS when evaluated at the age of 2 years. This included the two children who moved from a clinical diagnosis of autism to PDD-NOS. All but two of the 14 children who were initially diagnosed with PDD-NOS fell within the ASD classifcation (N = 8) or autism classifcation (N = 4) on the ADOS when they were initially evaluated. Of the four children who remained within the PDD-NOS group, the three who were given the ADOS fell within the ASD classifca- tion on the measure. Of the fve children whose clinical diagnosis moved from PDD-NOS to Autism, three received an autism classifcation on the ADOS. All fve of the children who moved from PDD-NOS to NS had an ASD score on the ADOS with one falling within the classifcation of autism. This child was later diagnosed with selective mutism. Of the six children initially diagnosed as NS and later reeval- uated, only two were given the ADOS. One met cri- teria for ASD on the measure and was later diagnosed with autism, whereas the other was NS on the measure and later was diagnosed with an anxiety disorder.

Discussion The ADOS worked encouragingly well in this group of young children diagnosed by experienced psycholo- gists in a general developmental clinic. Although the clinicians had not received intensive training on the measure or achieved research reliability, the ADOS demonstrated very good diagnostic discrimination when compared with a diagnosis based on a report review by an expert on ASD who was blinded to cli- nician diagnosis and scores on diagnostic measures. The group of clinicians included in this study had received a 2-day training on the measure and worked closely with clinicians who had achieved research reliability on the ADOS and were available to provide

P er

ce n

ta ge

o f

ca se

s P

er ce

n ta

ge o

f ca

se s

100

90

80

70

60

50

40 SCQ AUT

30 SCQ NS

20

10

0

� �

■

□

□

■

□

Autism PDD-NOS NS

Reviewer diagnosis

Figure 1 Reviewer diagnosis agreement with the autism diag- nostic observation schedule (top) and the social communication questionnaire (bottom)

Clinician and reviewer diagnostic agreement

Clinicians and reviewers agreed 80% of the time. Disagreements were primarily between PDD-NOS and AUT with reviewers more likely to classify the diagnosis AUT than clinicians (See Table 3). Encouragingly, there were no disagreements between AUT and NS disorders. That is, a reviewer never classifed a child as NS when a clinician clas- sifed a child with AUT or vice versa. Of the fve children that the reviewer classifed as NS and the clinician classifed as PDD-NOS, three had received a nonspecifc ASD diagnoses from the clinician (‘Features of PDD-NOS’, ‘PDD-NOS provisional’ and ‘Rule out ASD’). Of the 19 children that were diag- nosed PDD-NOS by the clinician and AUT by the reviewer, all had a suffcient symptom count to meet criteria for a diagnosis of autism and 10 (53%) met criteria for autism on the ADOS.

Table 3 Agreement between clinical diagnosis and reviewer diagnosis

Clinical diagnosis Autism

Reviewer diagnosis

PDD-NOS Nonspectrum

Autism PDD-NOS Nonspectrum

55 19 0

1 27 4

0 5

27

PDD-NOS, pervasive developmental disorder – not otherwise specifed.

2012 The Authors. Journal of Child Psychology and Psychiatry 2012 Association for Child and Adolescent Mental Health.

� �

doi:10.1111/j.1469-7610.2012.02607.x Diagnosis of autism spectrum disorders in 2-year-olds 183

consultation and support as needed. Although this level of consultation may not be typical for commu- nity clinicians using the ADOS, this suggests that the ADOS can be an effective diagnostic tool even with very young children in a community-based clinic with experienced clinicians who receive con- sultative support.

In contrast with previous studies, classifcation on the parent measure did not improve diagnostic accuracy in this young sample and the combination of the ADOS and a parent measure led to a decrease in sensitivity when differentiating ASD from NS dis- orders. The parent questionnaires used by the cli- nicians in this study were designed for screening rather than diagnostic purposes. These screening questionnaires were completed by parents of young children who may not have the experience or exper- tise to accurately rate the diagnostic behaviors these measures are targeting. This may have contributed to lack of improvement in diagnostic accuracy over the ADOS alone. Most of the longitudinal research studies used the lengthier standardized diagnostic interview, the ADI-R, rather than a parent ques- tionnaire. The ADI-R is administered and scored by experienced clinicians who have a solid under- standing of how to interpret the behaviors that they are coding. Another potential reason for the ineffec- tiveness of the M-CHAT and SCQ as diagnostic measures in this study is that neither one was de- signed to cover the entire age span between 24 and 36 months. Brief diagnostic measures that could be easily incorporated into a clinical diagnostic assessment do not exist for this age group.

Although these measures did not improve the accuracy of an ASD diagnosis, the M-CHAT appeared to be effective as a screener in this sample, as the majority of ASD children were captured by this measure. This was interesting given that the major- ity of the children were older than the recommended age. It is important to note that the evaluations in- cluded in this study were conducted prior to the widespread introduction of the follow-up interview for the M-CHAT, which has led to improved speci- fcity (Pandey et al., 2008). The SCQ, which was designed and tested on an older sample of children initially, missed many children with PDD-NOS, indicating that it is not the best screener or diag- nostic measure in this young age group. This is similar to studies that have found that the ADI-R, on which the SCQ is based, tends to miss children in this age group (Corsello et al., 2007). More research is needed on the SCQ in younger ages, including an evaluation of subscales and items to determine if the pattern of symptoms missed parallels fndings in studies of the ADI-R. Further research may lead to an effective diagnostic and screening measure for children between 2 and 3 years of age, which is important given that this is the group most often initially identifed and referred and that two mea- sures in combination with clinical judgment leads to

the most accurate diagnosis in 2-year-old children suspected of an ASD.

Of the children who were initially evaluated, 22% were reevaluated providing an additional way of assessing diagnostic accuracy of the ADOS in this 2-year-old sample. The ADOS scores at initial eval- uation were fairly good predictors of later diagnosis. Only two children who initially were given a clinical diagnosis of autism and met criteria for autism on the measure, moved into the PDD-NOS group at follow up. The remainder continued to meet criteria for autism. The PDD-NOS group had more variability in diagnosis at follow up, which is consistent with previous studies. Although all but two of the children within this group had an ASD classifcation on the ADOS at the time of the initial evaluation, fve no longer met criteria for an ASD at follow up. The only misclassifcation of autism on the measure at initial evaluation was for a child who was later diagnosed with selective mutism. The remaining children who initially met criteria for PDD-NOS continued to meet for PDD-NOS or moved into a diagnosis of autism.

Surprisingly, there was fairly good agreement between reviewer diagnoses, blind to scores on measures and clinician diagnosis, based on report review and coding and the clinician’s clinical judg- ment. This is encouraging as the MADDSP ASD coding system has been used on a broader scale in an older sample of children. The strong agreement between reviewer and clinician suggests that this surveillance methodology also shows promise in a younger sample. However, reports were all from one clinic and were carefully written, included the same measures, and generally the same level of detail leading to greater ease for the reviewer. The use of the ADOS may infuence the observations of the clinician as well as the data that they include in their report ultimately also infuencing the diagnosis of the reviewer even though they were blind to measure scores and classifcation. Diagnostic agreement may not be as easy to achieve in other situations given the variability of measures used and content of reports in the larger community.

Limitations This study was based on the practices in a commu- nity clinic and report review. The stringent controls that are in place for a prospective research study were not in place in this study. Each child did not receive all three standardized diagnostic measures, limiting the conclusions that we can draw about the relationship between these measures. An attempt was made to provide a reviewer diagnosis indepen- dent of the clinician generated scores and classif- cation on diagnostic measures using a standardized record review strategy by a research clinician blind to the scores on the diagnostic measures and clini- cian diagnosis. However, the reviewer diagnosis is based on a report written by a clinician who used

2012 The Authors. Journal of Child Psychology and Psychiatry 2012 Association for Child and Adolescent Mental Health.

� �

184 Christina M. Corsello, Natacha Akshoomoff, and Aubyn C. Stahmer J Child Psychol Psychiatry 2013; 54(2): 178–85

specifc diagnostic measures to gather information and therefore, subject to the biases of the clinician who wrote the report. In addition, follow-up evalua- tions were only available for a small subset of the sample, again limiting the conclusions that can be made in terms of predictive value of the measure.

Summary Within this community-based clinic, the ADOS evi- denced good diagnostic discrimination in a 2-year- old sample when administered by clinicians who had received a 2-day clinical training followed by con- sultative support. Additional research is needed to identify or develop an appropriate effcient parent report tool to accompany a standardized diagnostic measure, as neither the SCQ nor the M-CHAT added

to the diagnostic accuracy of the ADOS in this sample.

Acknowledgements This study was supported by grants from the National Institutes of Health: K23MH071796 (to N.A.) and K01MH065325 (to A.C.S.). We also thank our research assistants, Rebecca Piñon Ruiz, Jasper Estabillo, and Therese Gadomski.

Correspondence Natacha Akshoomoff, Department of Psychiatry, 9500 Gilman Drive, 0115, University of California, San Diego, La Jolla, CA 92093-0115, USA; Email: nakshoom [email protected]

Key points

• The ADOS was effective in a 2-year-old sample in a community clinic when administered by psychologists who had received a 2-day training followed by consultation.

• The use of a parent screening measure did not improve diagnostic accuracy over the ADOS alone. • Diagnosis remained stable in the majority of children who were diagnosed with autism or nonspectrum

disorders and less stable in toddlers diagnosed with PDD-NOS. • There are no diagnostic parent interviews or questionnaires currently available for this age group that are

easily incorporated in a clinical evaluation.

References Bayley, N.. (1993). Bayley scales of infant development-second

edition (BSID-II) (2nd edn). San Antonio, TX: The Psycholog- ical Corporation.

Bayley, N.. (2005). Bayley scales of infant and toddler devel- opment, third edition (Bayley-III) (3rd edn). San Antonio, TX: Harcourt Brace.

California Department of Developmental Services. (2002). Autistic spectrum disorders: Best practice guidelines for screening. Sacramento, CA: Diagnosis and Assessment.

Centers for Disease Control. (2007). Prevalence of autism spectrum disorders – autism and developmental disabilities monitoring network, 14 sites, United States, 2002. Morbidity and Mortality Weekly Report Surveillance Summaries, 56, 12–28.

Charman, T., Taylor, E., Drew, A., Cockerill, H., Brown, J.A., & Baird, G. (2005). Outcome at 7 years of children diagnosed with autism at age 2: Predictive validity of assessments conducted at 2 and 3 years of age and pattern of symptom change over time. Journal of Child Psychology and Psychi- atry, 46, 500–513.

Corsello, C., Hus, V., Pickles, A., Risi, S., Cook, E., Leventhal, B., & Lord, C. (2007). Between a ROC and a hard place: Decision making and making decisions about using the SCQ. Journal of Child Psychology and Psychiatry, 48, 932– 940.

Filipek, P.A., Accardo, P.J., Ashwal, S., Baranek, G.T., Cook, E.H., Jr, Dawson, G., ... & Volkmar, F.R. (2000). Practice parameter: Screening and diagnosis of autism: Report of the quality standards subcommittee of the American academy of neurology and the child neurology society. Neurology, 55, 468–479.

Filipek, P.A., Accardo, P.J., Baranek, G.T., Cook, E.H., Jr, Dawson, G., Gordon, B., ... & Volkmar, F.R. (1999). The screening and diagnosis of autistic spectrum disorders.

Journal of Autism and Developmental Disorders, 29, 439– 484.

Johnson, C.P., Myers, S.M., & American Academy of Pediatrics Council on Children with Disabilities. (2007). Identifcation and evaluation of children with autism spectrum disorders. Pediatrics, 120, 1183–1215.

Kleinman, J.M., Robins, D.L., Ventola, P.E., Pandey, J., Boorstein, H.C., Esser, E.L., ... & Fein, D. (2008). The modifed checklist for autism in toddlers: A follow-up study investigating the early detection of autism spectrum disor- ders. Journal of Autism and Developmental Disorders, 38, 827–839.

Le Couteur, A., Rutter, M., Lord, C., Rios, P., Robertson, S., Holdgrafer, M., & McLennan, J. (1989). Autism diagnostic interview: A standardized investigator-based instrument. Journal of Autism and Developmental Disorders, 19, 363–387.

Lord, C., Risi, S., DiLavore, P.S., Shulman, C., Thurm, A., & Pickles, A. (2006). Autism from 2 to 9 years of age. Archives of General Psychiatry, 63, 694–701.

Lord, C., Rutter, M., DiLavore, P.C., & Risi, S. (2001). Autism diagnostic observation schedule. Los Angeles: Western Psy- chological Services.

Mandell, D.S., Ittenbach, R.F., Levy, S.E., & Pinto-Martin, J.A. (2007). Disparities in diagnoses received prior to a diagnosis of autism spectrum disorder. Journal of Autism and Devel- opmental Disorders, Pediatrics, 116, 1480–1486.

Mandell, D.S., Novak, M.M., & Zubritsky, C.D. (2005). Factors associated with age of diagnosis among children with autism spectrum disorders. Pediatrics, 116, 1480–1486.

Mazefsky, C.A., & Oswald, D.P. (2006). The discriminative ability and diagnostic utility of the ADOS-G, ADI-R, and GARS for children in a clinical setting. Autism, 10, 533–549.

Molloy, C.A., Murray, D.S., Akers, R., Mitchell, T., & Manning- Courtney, P. (2011). Use of the autism diagnostic observa- tion schedule (ADOS) in a clinical setting. Autism, 15, 143–162.

2012 The Authors. Journal of Child Psychology and Psychiatry 2012 Association for Child and Adolescent Mental Health.

� �

doi:10.1111/j.1469-7610.2012.02607.x

Mullen, E.M. (1995). Mullen scales of early learning (AGS ed.). Circle Pines, MN: American Guidance Service.

National Research Council. (2001). Educating children with autism. Washington, DC: National Academy Press.

Pandey, J., Verbalis, A., Robins, D.L., Boorstein, H., Klin, A., Babitz, T., ... & Fein, D. (2008). Screening for autism in older and younger toddlers wth the modifed checklist for autism in toddlers. Autism, 2, 513–535.

Risi, S., Lord, C., Gotham, K., Corsello, C., Chrysler, C., Szatmari, P., ... & Pickles, A. (2006). Combining information from multiple sources in the diagnosis of autism spectrum disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 45, 1094–1103.

Robins, D.L., Fein, & Dumont-Mathieu, T.M. (2006). Early screening for autism spectrum disorders: Update on the modifed checklist for autism in toddlers and other mea- sures. Journal of developmental and behavioral pediatrics, 27(Suppl. 2), S111–S119.

Robins, D.L., Fein, D., Barton, M.L., & Green, J.A. (2001). The modifed-checklist for autism in toddlers: An initial study investigating the early detection of autism and pervasive developmental disorders. Journal of Autism and Develop- mental Disorders, 31, 131–144.

Rutter, M., Bailey, A., & Lord, C. (2003). Social communication questionnaire. Los Angeles: Western Psychological Services.

Rutter, M., Le Couteur, A., & Lord, C. (2003). ADI-R. Autism diagnostic interview-revised. WPS edition. Los Angeles: Western Psychological Services.

Diagnosis of autism spectrum disorders in 2-year-olds 185

Shattuck, P.T., Durkin, M., Maenner, M., Newschaffer, C., Mandell, D.S., Wiggins, L., ... & Cuniff, C. (2009). Timing of identifcation among children with an autism spectrum disorder: Findings from a population-based surveillance study. Journal of the American Academy of Child and Adolescent Psychiatry, 48, 474–483.

Sparrow, S.S., Cicchetti, D.V., & Balla, D.A. (2005). Vineland adaptive behavior scales. II. Survey forms manual (2nd edn). Circle Pines, MN: American Guidance Service.

Turner, L.M., & Stone, W.L. (2007). Variability in outcome for children with an ASD diagnosis at age 2. Journal of Child Psychology and Psychiatry, 48, 793–802.

Ventola, P.E., Kleinman, J., Pandey, J., Barton, M., Allen, S., Green, J., ... & Fein, D. (2006). Agreement among four diagnostic instruments for autism spectrum disorders in toddlers. Journal of Autism and Developmental Disorders, 36, 839–847.

Wechsler, D. (2002). Wechsler preschool and primary scale of intelligence-third edition (WPPSI-III). San Antonio, TX: The Psychological Corporation.

Wiggins, L.D., & Robins, D.L. (2008). Brief report: Excluding the ADI-R behavioral domain improves diagnostic agreement in toddlers. Journal of Autism and Developmental Disorders, 38, 972–976.

Accepted for publication: 9 July 2012 Published online: 20 August 2012

2012 The Authors. Journal of Child Psychology and Psychiatry 2012 Association for Child and Adolescent Mental Health.