Discussion: Scholarly article review guideline

ORIGINAL ARTICLE

Screening for autism spectrum disorder in children with Down syndrome: An evaluation of the Pervasive Developmental Disorder in Mental Retardation Scale Vincent Pandolfia, Caroline I. Magyarb and Charles A. Dillc

aPsychology Department, Rochester Institute of Technology, Rochester, NY, USA; bDepartment of Paediatrics, Division of Neurodevelopmental and Behavioural Paediatrics, University of Rochester School of Medicine and Dentistry, Rochester, NY, USA; cPsychology Department, Hofstra University, Hempstead, NY, USA

ABSTRACT Background Children with Down syndrome (DS) are at risk for autism spectrum disorder (ASD). They are often diagnosed later than other children in part due to difficulty differentiating ASD-related impairment from that associated with DS. The psychometric properties of the Pervasive Developmental Disorder in Mental Retardation Scale (PDD-MRS) were examined with the aim of informing ASD screening guidelines for children with DS. Method Analysis of archival data from children aged 3 to 15 years with DS (N = 386) evaluated the reliability and validity of the PDD-MRS. Results Factor analyses yielded 2 factor-based scales: ASD and Emotional and Behavioural Problems. ASD reliably assessed ASD-specific symptoms, correlated with other ASD measures, and demonstrated good diagnostic accuracy. Emotional and Behavioural Problems assessed problems not diagnostic of ASD but may reflect part of the behavioural phenotype of DS and ASD. Conclusion The PDD-MRS appears to have utility in ASD screening for this population.

KEYWORDS autism; ASD; Down syndrome; PDD-MRS; assessment

Introduction

Research suggests the prevalence of autism spectrum dis- order (ASD) in children with Down syndrome (DS) may be higher than that observed in the general population. Estimates range from 5% to 39% (for a review, see Moss & Howlin, 2009; Reilly, 2009); many children with DS are diagnosed at a later age relative to other chil- dren and some may not be identified at all (e.g., Howlin, Wing, & Gould, 1995; Rasmussen, Börjesson, Wentz, & Gillberg, 2001). This situation is concerning, particularly as research indicates that children with DS and co-occur- ring ASD have lower cognitive, language, and adaptive levels, and have more behaviour problems than children with DS only (e.g., Magyar, Pandolfi, & Dill, 2012; Mol- loy et al., 2009), suggesting that the co-occurrence of ASD conveys additional morbidity. If not identified with a co-occurring ASD, then the child is unlikely to receive ASD-specific treatment.

Although the exact reasons for the late or lack of ASD diagnosis is unknown, it has been speculated that the sig- nificant delays and impairments seen in children with DS in the areas of communication and behaviour are attrib- uted to the child’s level of intellectual disability (ID) and associated speech and language impairments. These pro- blems are not viewed as symptomatic of a co-occurring ASD, a phenomenon called diagnostic overshadowing

(Reiss, Levitan, & Szyszko, 1982). However, the literature containing descriptions of children with DS and co- occurring ASD (DS + ASD) and those without ASD (DS) suggests that (a) heightened severity of impairment, and (b) differences in the form of the social communi- cation and repetitive behaviour can distinguish ASD- related impairments from those related to ID. This is more likely if a developmental approach to ASD assess- ment is applied (Dosen, 2005; Hepburn, Philofsky, Fidler, & Rogers, 2008) and if reliable and valid ASD assessment measures are used (e.g., Magyar et al., 2012).

The distinction between ASD-related symptomatol- ogy and DS-related impairment can be seen in both the social communication and repetitive behaviour domains of an ASD diagnosis. These include the pres- ence of more stereotyped and repetitive speech forms (when speech is present), limited or no gesturing for communicative purposes, limited interest in peers, increased probability of aggression in response to social approaches by peers, the absence of pretend and imagi- native play, the absence of functional play, such as demonstrating the tendency to line objects up, and pre- occupation with parts of objects (e.g., see Channell et al., 2015; Hepburn et al., 2008; for a review, see Reilly, 2009). These findings contribute greatly to advancing our knowledge of the potential clinical indicators that can

© 2017 Australasian Society for Intellectual Disability, Inc.

CONTACT Vincent Pandolfi [email protected]

JOURNAL OF INTELLECTUAL & DEVELOPMENTAL DISABILITY, 2018 VOL. 43, NO. 1, 61–72 https://doi.org/10.3109/13668250.2016.1271111

distinguish children with DS + ASD from children with DS only. Nevertheless, the challenge involved in accu- rately and systematically assessing for these indicators highlights the need for reliable and valid ASD screening measures.

A review of the ASD screening and health surveillance guidelines for all children published in 2007 (Johnson, Myers, & the Council on Children With Disabilities, 2007) and for children with DS published in 2011 by the American Academy of Pediatrics (Bull & the Com- mittee on Genetics, 2011), however, indicates that neither document provides sufficient research-based guidance to practitioners on the selection or use of reliable and valid ASD screening measures for children with DS. For example, the 2007 document recommends that all chil- dren be screened at the 18 and 24 month well-child visits using a standardised ASD screening measure. However, this recommendation was designed to assist practitioners in screening the general population of children. Although children with DS may be subsumed under this rec- ommendation, the timing of the screening assessment may not be consistent with the emergence of distinguish- able ASD symptoms in children with DS because of the developmental delays in the areas of language, communi- cation, and play. If screened at these very young ages for ASD, the practitioner may attribute relevant develop- mental impairments to the child’s intellectual and language impairments associated with the DS and not to the presence of ASD (i.e., diagnostic overshadowing). This may preclude further assessment at later ages when changes in the child’s development may lead to bet- ter differentiation of the manifest ASD symptoms.

The 2011 health surveillance guideline for children with DS includes a recommendation for ASD screening and anticipatory guidance beyond the 24-month level, but it does not include a recommendation for a specific ASD screening measure validated for use in children with DS. This is problematic because of the finding that of those practitioners who report using a standar- dised ASD screening measure per the 2007 American Academy of Pediatrics guidelines (Johnson et al., 2007), the Modified Checklist for Autism in Toddlers (M-CHAT; Robins, Fein, Barton, & Green, 2001) is the most commonly used ASD screening measure in the United States (Arunyanart et al., 2012), but this measure has not been validated for use with children with DS. This may limit its usefulness in screening children with DS who may require periodic screening throughout childhood for the emergence of ASD symptoms.

To our knowledge the only ASD general population screening measure to receive at least one thorough evalu- ation of its psychometric properties including diagnostic accuracy in children with DS is the Social

Communication Questionnaire – Lifetime Version (SCQ-L; Rutter, Bailey, & Lord, 2003), a 40-item care- giver-completed paper-and-pencil questionnaire. Magyar et al. (2012) evaluated the SCQ-L in a large (N = 447), well-characterised sample of children with DS aged 4 to 14 years 11 months, with and without ASD, and found that the measure was not only reliable but also that it discriminated between children with DS alone and those with DS + ASD. But although sensitivity was found to be good, specificity was not as robust. This suggests that practitioners may need to use an additional ASD screening measure or a different approach, such as an interview with the caregiver, to better specify the nature of the identified social communication and behavioural impairments. This might improve prac- titioners’ confidence in their decision on whether to refer a child for an ASD diagnostic assessment. This approach to assessment is consistent with the rec- ommendation that information from multiple sources is needed to improve clinical decision-making in ASD evaluation (Pandolfi & Magyar, 2014; Risi et al., 2006).

Present study

In the present study we aimed to expand the evidence base on psychometrically sound ASD screening measures for children with DS. This would provide prac- titioners an opportunity to consider more than one measure with evidence to support its use, enabling them to tailor the selection of measures to client charac- teristics and need. This may improve the chances of early ASD detection within this population.

We evaluated the psychometric properties of the Per- vasive Developmental Disorder in Mental Retardation Scale (PDD-MRS; Kraijer, 2006) in a large sample of children with DS. The PDD-MRS evaluates behaviours within the past 2–6 months consistent with an ASD diag- nosis. It was developed specifically for children and adults (2–70 years) with an ID and features a flexible administration format: data may be collected via clinical interview or through caregiver paper-and-pencil self- report. The PDD-MRS can be used for initial screening or for subsequent checks on a previous assessment out- come. In this study we specifically examined (a) internal structure of the scale through exploratory and confirma- tory factor analyses of the items, (b) scale reliability, (c) convergent and discriminant correlations with measures of ASD and cognition, (d) criterion validity indicated by differences in mean PDD-MRS scores between groups of children with and without ASD, and (e) diagnostic accu- racy through receiver operating characteristic (ROC) and predictive discriminant analyses. All analyses were com- pleted to better understand the extent to which PDD-

62 V. PANDOLFI ET AL.

MRS scores could be interpreted as indicators of ASD in children with DS.

Method

Design

Data collection Archival PDD-MRS data were used in this study (N = 386). Data were obtained from a statewide preva- lence study of medical and behavioural comorbidities of children with DS aged 3 years 4 months to 15 years 5 months. The prevalence study was approved by the University of Rochester’s institutional review board (RRRB12774) and written informed consent was obtained from the parents of all children. All data were coded and stored so that participant confidentiality was ensured and only those researchers with university approval were authorised to access the data.

The prevalence study used a three-tiered ascertainment procedure. Tier 1 assessment procedures included the mailing and completion of ASD screening measures: the M-CHAT (Robins et al., 2001; N = 440), the SCQ-L (Rut- ter, Bailey, & Lord, 2003; N = 438), telephone adminis- tration of the PDD-MRS (Kraijer, 2006; N = 386), as well as a medical history questionnaire. All children who screened positive (defined as positive on one or more ASD screening measures) and an approximately equal number of children who screened negative were evaluated at Tier 2 (N = 221) using the Autism Diagnostic Interview – Revised (ADI-R; Rutter, Le Couteur, & Lord, 2003), the Vineland Adaptive Behavior Scales – Second Edition – Parent/Caregiver Form (Sparrow, Cicchetti, & Balla, 2005), and the Repetitive Behaviour Scale – Revised (RBS-R; Bodfish, Symons, & Lewis, 1999). All children who screened positive on the ADI-R and an approxi- mately equal number of children who screened negative participated in the Tier 3 assessment (N = 71), which included administration of the Autism Diagnostic Obser- vation Schedule (ADOS; Lord, Rutter, DiLavore, & Risi, 2002), the Leiter International Performance Scale – Revised (Leiter-R; Roid & Miller, 1997), a norm-refer- enced measure of nonverbal intelligence, and the Child Behavior Checklist 1.5–5/6–18 (Achenbach & Rescorla, 2000, 2001), a parent-completed checklist to screen for common emotional and behavioural problems in children.

ASD diagnoses were made by an experienced evalu- ation team that included clinicians with extensive clinical and research experience in ASD and developmental dis- ability: a developmental paediatrician, a licensed clinical psychologist, and several trained clinical evaluators. A developmental approach to diagnosis (Dosen, 2005; Hep- burn et al., 2008) was used whereby an ASD diagnosis was

based on all data collected across all tiers on the child, eval- uated within the context of the child’s IQ and level of expressive language (verbal/nonverbal), and summarised ona Diagnosticand Statistical Manual of Mental Disorders (4th ed., text rev., DSM-IV-TR; American Psychiatric Association [APA], 2000) ASD checklist using clinical consensus guidelines developed bytheteam. Child partici- pants were diagnosed witheither DS only (n = 38) or DS + ASD (i.e., autistic disorder or pervasive developmental disorder not otherwise specified [PDD-NOS]; n = 33).

Participant characteristics

Data from all three tiers were used in the analyses. The total participant pool (N = 386) was divided to allow for exploratory (EFA) and confirmatory factor analyses (CFA) on separate matched groups. Participants were matched by age and then randomly assigned to either the EFA (n = 193) or CFA (n = 193) subgroup, and par- ticipants with Level 3 data (n = 71) were then used for the convergent and discriminant analyses. SPSS Version 21 software was used to generate descriptive data for the EFA and CFA subgroups, which are presented in Table 1.

Table 1. Descriptive data for model development and replication subgroups.

Variable

EFA subgroup CFA subgroup

n M SD n M SD

Age (years) 193 8.95 3.18 193 8.95 3.18 Vineland Adaptive Behavior Composite

102 68.09 11.90 111 65.85 11.84

Autism Diagnostic Interview – Reviseda

Reciprocal Social Interaction

106 7.25 5.46 114 8.03 5.63

Communication-Verbal 106 3.52 3.82 114 4.22 4.33 Communication- Nonverbal

106 2.34 3.77 114 2.23 3.62

Restrictive, Repetitive, and Stereotyped Behaviour and Interests

106 3.46 2.78 113 3.80 2.96

n No. % n No. %

Gender (male) 193 101 52.33 193 106 54.92 Raceb 190 192 White 173 91.05 180 93.75 Black 7 3.68 7 3.65 Asian 2 1.05 2 1.04 American Indian 1 0.53 2 1.04 > 1 category 7 3.68 1 0.52

Ethnicity Hispanic 187 11 5.88 190 6 3.16 Parent education 193 193 Less than high school 1 0.52 5 2.59 High school 26 13.47 28 14.51 Some college/ specialised training

40 20.73 35 18.13

College 126 65.28 124 64.25 Graduate degree 0 0.00 1 0.52

Note. EFA = exploratory factor analysis; CFA = confirmatory factor analysis. n = 193 for EFA and CFA subgroups. When n < 193, this was due to missing data.

aCurrent behaviour algorithm scores. bPercentages for EFA subgroup do not sum to 100 because of rounding error.

JOURNAL OF INTELLECTUAL & DEVELOPMENTAL DISABILITY 63

The groups were similar across all demographic and developmental variables. Significance tests were not con- ducted on these variables because of the use of matching and random assignment. The quantitative data were very similar across the EFA and CFA subgroups: the minor differences were not substantively meaningful and any statistically significant differences that occurred through random assignment could be attributed to Type I error. Thus, the subgroups had participants of the same age, approximately equal numbers of males and females, were composed mostly of participants who were white, had parents with college-level education, had similar ADI-R scores, and had Vineland adaptive behaviour scores that were well below the population mean.

Demographic data for Level 3 participants have been published elsewhere (Magyar et al., 2012) and those results are only summarised here. The group with DS (n = 38, M = 7.92 years, SD = 3.19 years) was slightly younger than the group with DS + ASD (n = 33, M = 8.97, SD = 2.51), but this difference was not significant at α < .05 (p = .132). As might be expected, the group with DS + ASD exhibited more developmental impairment than the group with DS only. The group with DS only showed significantly higher IQ scores (MDS = 52.38, SD = 14.57 vs. MDS + ASD = 41.93, SD = 6.74) and Vineland Adaptive Behavior Composite scores (MDS = 69.65, SD = 9.87 vs. MDS + ASD = 60.12, SD = 10.91) with both ps < .001. The DS + ASD group evidenced higher scores on the ADI-R Social Interaction (p < .001), Communication-Verbal (p = .004), and Repeti- tive Behaviour domains (p < .001), which indicated that these children demonstrated more ASD symptoms.

Instruments

This section contains a description of the measures used for the main data analyses (PDD-MRS, ADI-R, and the Leiter-R) and for subject characterisation (Vineland Adaptive Behavior Scales – Second Edition).

Pervasive Developmental Disorder in Mental Retardation Scale The PDD-MRS (Kraijer, 2006) is a 12-item ASD screen- ing measure developed for persons with ID. Item content was informed by several sources. They included the DSM-III-R (3rd ed., rev.; APA, 1987) criteria for the per- vasive developmental disorders of autistic disorder and PDD-NOS. It also included a literature review and file review of individuals with known ID and PDD. Items assess the quality of social interactions with adults and peers, language and speech problems, and several other behaviours such as obsessive interests, stereotyped beha- viours, self-injury, and erratic/unpredictable behaviour. Items are scored as present or absent during the

preceding 2–6 months, and the differentially weighted items are summed to provide a total score that indicates the likelihood of ASD. The test author reported that the differential weighting increased the instrument’s discri- minant validity, although no explanation was provided in the manual as to how the specific weights were deter- mined. Three qualitative ranges describe the total score’s screening outcome: “non-PDD” (Total Score = 0–6), “Doubtful PDD/non-PDD” (Total Score = 7–9), reflect- ing diagnostic uncertainty, and “PDD” (Total Score = 10–20).

The technical manual contains psychometric findings from the test development sample, which included chil- dren and adults with DS (see Kraijer, 2006). Internal consistency, interrater reliability, and test score stability were favourable. An EFA was completed on the 12 PDD-MRS items along with several developmental and demographic variables. It yielded the following four fac- tors: PDD, functioning level, gender, and age. However, factor analyses that only included the test items them- selves were not reported. The measure was found to gen- erally discriminate persons with the then-current diagnosis of PDD from persons without it. Despite the favourable reliability and validity data reported in the manual, independent replications of these findings have not been reported, and no independent study has examined the psychometric properties of the scale specifically in children with DS.

Social Communication Questionnaire – Lifetime Version The SCQ-L (Rutter, Bailey, & Lord, 2003) is a paper- and-pencil ASD screening measure that contains 40 items. Item content is related to the ADI-R. The measure assesses for qualitative impairments in recipro- cal social interaction and communication, and restricted, repetitive, and stereotyped behaviours. The measure is completed by a caregiver who is familiar with the child’s developmental history. The SCQ-L was included because it demonstrated favourable psychometric properties in a previous study of children with DS (see Magyar et al., 2012).

Autism Diagnostic Interview – Revised The ADI-R (Rutter, Le Couteur, & Lord, 2003) is a diag- nostic interview used to assess individuals suspected of having an ASD. Respondents are the caregivers familiar with the referred individual’s developmental history and present behaviour. The ADI-R captures information about functioning in three subdomains: Language/Com- munication, Reciprocal Social Interaction, and Restrictive, Repetitive, and Stereotyped Behaviours and Interests. Criterion-referenced diagnostic algorithm scores inform

64 V. PANDOLFI ET AL.

diagnostic decision-making. A current behaviour algor- ithm provides an assessment of the individual’s current status within each of the ADI-R subdomains. This was the algorithm used for the correlation analyses in this study because the current behaviour algorithm scores reflect functioning during a timeframe that is consistent with the PDD-MRS.

Leiter International Performance Scale – Revised The Leiter-R (Roid & Miller, 1997) is a standardised norm-referenced nonverbal measure of intelligence. It assesses visualisation and reasoning skills, as well as attention and memory. It provides IQ scores ranging from 30 to 170 (M = 100, SD = 15) and is appropriate for use with individuals aged 2 to 21 years. The Leiter- R demonstrates good reliability and validity, and the overall IQ can be used for educational and diagnostic decision-making (e.g., see Salvia & Ysseldyke, 2004).

Vineland Adaptive Behavior Scales – Second Edition The Vineland (Sparrow et al., 2005) is a standardised norm-referenced measure of adaptive behaviour, which is appropriate for use with individuals from birth to 90 years. The Parent/Caregiver Rating Form was used in this study to characterise participants’ level of function- ing. The Vineland provides the Adaptive Behavior Com- posite, which indicates the child’s overall level of functioning. It also provides scores for four specific adaptive behaviour domains: Communication, Socializa- tion, Daily Living Skills, and Motor Skills (ages birth–6 years, only). All Vineland scales have a mean of 100 and standard deviation of 15. The technical manual provides evidence of its reliability and validity.

Data analysis

LISREL 8.80 statistical software (Jöreskog & Sörbom, 2006) was used for the factor analyses, and SPSS Version 21 was used for all other analyses.

Data analysed All PDD-MRS items were dichotomously scored and were equally weighted (0 = negative for abnormal behav- iour, 1 = positive), with the exception of analyses of Krai- jer’s original weighted 12-item total score. Factor analyses of tetrachoric correlations among the dichoto- mous items could not include Items 4 (deviant language content) and 5 (deviant language production) because of the way the items are scored. The scores for Items 4 and 5 depend on the response to Item 3 (absence of expressive language). This dependency guaranteed empty cells in the bivariate frequency tables involving these items, which can result in biased tetrachoric correlations

(Greer, Dunlap, & Beatty, 2003). All subsequent analyses were conducted on (a) scores from the factor-based scales that emerged from the factor analyses; and (b) two PDD-MRS total scores: one based on the 10 dichot- omously scored items, which we called the Total-10, and the other was Kraijer’s 12-item weighted total score, which we called the Total-12.

Data analysis procedure Data were analysed in several steps. First, EFAs and CFAs were performed on the tetrachoric correlation matrices to determine what constructs were assessed by the PDD-MRS. EFAs and CFAs were each run on one half of the sample. CFAs followed the EFAs, which allowed us to determine if the EFA results could be repli- cated on a separate age-matched sample. Next, the fac- tors that emerged from the factor analyses were treated as factor-based measurement scales, and scale reliability analyses were conducted on them and for the Total-10 and Total-12. The reliability analyses were conducted on the CFA subgroup data because CFA parameters were used to compute the reliability indices (discussed below).

After the reliability analyses, additional analyses were conducted. The factor-based scales, the Total-10, and Total-12 were correlated with the ADI-R, SCQ-L, and Leiter-R IQ scores, which provided evidence pertaining to convergent and discriminant validity. Next, signifi- cance tests determined if the factor-based scale scores, the Total-10, and Total-12 could discriminate between groups of children with and without ASD. Finally, ROC and predictive discriminant analyses were com- pleted to determine how well the factor-based and total scores identified individual children with and without confirmed ASD diagnoses.

Results

Factor structure

Robust diagonally weighted least-squares (DWLS) esti- mation was used for all EFAs and CFAs. DWLS is appro- priate for ordered-categorical data and moderately sized samples (Flora & Curran, 2004; Wirth & Edwards, 2007; Yang-Wallentin, Jöreskog, & Luo, 2010).

Exploratory factor analysis We performed EFAs (n = 193) within the CFA frame- work (E/CFA; see Brown, 2006). In E/CFA the factor models to be tested are specified in the following way: (a) the factor variances are fixed to 1.0, (b) factor corre- lations are freely estimated, (c) each factor contains one “anchor item” that is freely estimated but whose loadings

JOURNAL OF INTELLECTUAL & DEVELOPMENTAL DISABILITY 65

on all other factors are fixed to zero, and (d) the remain- ing items are free to load on all factors (see Brown, 2006). The final solution presented here included Item 2 as the anchor for Factor 1 (ASD) and Item 9 for Factor 2 (Emotional and Behavioural Problems). We tested many other models with different anchor items but the one presented here was best with respect to model fit and interpretability.

Correlated two-factor and three-factor models were specified because the need to model independent factors would be indicated by very small factor correlations (Floyd & Widaman, 1995). An interpretable two-factor sol- ution emerged from the E/CFA. The three-factor model resulted in an inadmissible solution. The two-factor stan- dardised factor pattern coefficients are presented in Table 2.

Factor 1 appeared to reflect a broad ASD factor, and Factor 2 contained items assessing a range of problems, which we labelled Emotional and Behavioural Problems (EBP). The correlation between ASD and EBP of .47 was statistically significant (α < .05, p < .001) and indi- cated that the factors shared 22.1% of the variance. Although Table 2 shows that Items 6, 7, and 11 did not load significantly on either factor, the pattern of results and the items’ content allowed for the testing of competing factor models in the CFAs. Item 6 (outstand- ing and obsessive interests) appeared to load about evenly on both factors, Item 7 (stereotyped or unusual handling of objects) loaded much higher on the ASD factor, and Item 11 (highly erratic unpredictable behaviour) loaded higher on EBP. These results suggested that the PDD- MRS measured two related dimensions in this sample, which we labelled ASD and EBP.

Confirmatory factor analysis CFAs (n = 193) were conducted to verify the EFA find- ing. Because Item 6 loaded evenly on both factors in the E/CFA, we tested three versions of the two-factor model: one with Item 6 on ASD, one with Item 6 on EBP, and one with Item 6 loading on both factors. We also tested a one-factor model containing all 10 items (Total-10) because this reflected the most parsimonious model to test and because the PDD-MRS was originally developed to provide one total score.

We evaluated model adequacy in several ways. First, factor models were inspected for out-of-range parameter estimates (e.g., negative error variances), which can suggest an “incorrect” model. We then examined two psychometric fit indices. The root mean square error of approximation (RMSEA; Steiger & Lind, 1980) indicates the amount of misfit per degree of freedom, with values ≤ .05 indicating a good fit, values greater than .05 and ≤ .10 indicating an acceptable fit, and values > .10 indi- cating a poor fit (MacCallum, Browne, & Sugawara, 1996). The comparative fit index (CFI; Bentler, 1990) is an incremental fit index that evaluates model fit relative to a baseline model, which is usually a model containing uncorrelated variables. Values close to .95 suggest good fit (Hu & Bentler, 1999).

Results did not indicate out-of-range parameter esti- mates for any of the factor models. However, in the two-factor model where Item 6 loaded on both factors, its factor loading was not significant on EBP (–.05). In the two-factor model where Item 6 only loaded on EBP, we found an extremely high factor correlation of .90, which indicated nearly redundant factors. Therefore, these models were rejected.

The one-factor model (Total-10) and the two-factor model (ASD and EBP) with Item 6 on ASD both demon- strated acceptable parameter estimates, and all factor loadings were statistically significant. The fit indices sup- ported both models. For the one-factor model the RMSEA = .042 and CFI = .988, and for the two-factor model the RMSEA = .017 and CFI = .998. The factor cor- relation of .71 was statistically significant and indicated that the ASD and EBP factors shared 50.4% of the var- iance, an indication of adequate discriminant validity (see Brown, 2006). We performed the Satorra–Bentler chi-square difference test (Satorra & Bentler, 2001) to determine if the two-factor model provided a better fit than the one-factor model, and the statistically signifi- cant (α < .05) chi-square value indicated that it did (SBχ2diff = 42.06, df = 1, p < .001). Thus, we concluded that the two-factor model best fit the data. Table 3 con- tains the standardised factor pattern coefficients.

The median coefficient on ASD was .65, which meant that, on average, 42.3% of the item’s variance was

Table 2. Factor pattern coefficients from exploratory factor analysis: Two-factor solution (n = 193).

Item

Factor/p value

ASD p EBP p

1 Social interaction with adults 47 .004 30 .099 2 Social interaction with peers 96 < .001 – – 3 Expressive language absent/virtually absent

67 .033 02 .952

6 Outstanding and obsessive interests 37 .395 39 .390 7 Stereotyped or unusual handling of objects

40 .064 28 .267

8 Stereotyped manipulation of own body, no objects involved

54 .043 28 .412

9 Strong dependence on fixed patterns, routines, or rituals

– – 81 < .001

10 Self-injury 22 .401 48 .047 11 Highly erratic unpredictable behaviour –06 .889 53a .153 12 Unusual or unreasonable and excessive anxiety or panic

–24 .234 59 .008

Note. Items 2 and 9 were anchor items to satisfy identifying restrictions. ASD = ASD factor; EBP = Emotional and Behavioural Problems factor. Deci- mals were eliminated from all coefficients. ASD and EBP factors were posi- tively and significantly correlated (.47; p < .001).

aItem 11’s factor loading on EBP was not statistically significant but was larger than Item 10’s. Item 11’s loading was not significant because its standard error was larger than Item 10’s.

66 V. PANDOLFI ET AL.

accounted for by the factor. The median coefficient for EBP was .59: on average, 34.8% of the item’s variance was accounted for by the factor. These results showed that the relationships between the PDD-MRS items and their respective factors were relatively strong and in the expected direction. The CFA results were consist- ent with the EFA: the PDD-MRS measured two related dimensions in the sample, which we labelled ASD and EBP. The ASD and EBP factors were treated as factor- based measurement scales for the remaining analyses.

Scale reliability

Reliability indices were computed for the ASD and EBP scales and the two PDD-MRS total scores: the Total-10 and Total-12. CFA parameters were used to compute reliability for the factor-based scales (see Brown, 2006; Raykov, 1997, 2001), which was preferred to coefficient α because it has less restrictive assumptions (Green & Yang, 2009; Sijtsma, 2009). Thus, reliability analyses were conducted on the CFA sample data (n = 193). Reliability for ASD was .83 and .60 for EBP. Scale reliability for the Total-10 was computed using the CFA parameters for the one-factor model, and an esti- mate of .83 was obtained. Because CFA parameters were not available for the Kraijer Total-12, scale reliability was estimated using Guttman’s λ-2. Although based on the same assumptions as α, Guttman’s internal consistency estimate is less affected by violations of its assumptions (Sijtsma, 2009). We found λ-2 = .67.

Correlations with other variables

Evidence for convergent and discriminant validity for ASD, EBP, the Total-10, and Total-12 was evaluated by examining their correlations (α < .05) with the ADI-R subdomains, the SCQ-L, and IQ. These correlations

were calculated on Tier 3 sample data (n = 71) because these children were administered the ADI-R and Lei- ter-R. Findings are presented in Table 4.

ASD correlated significantly with the ADI-R’s Social Interaction (p < .001), Verbal (p = .001) and Nonverbal Communication (p = .009), and Repetitive Behaviour (p < .001) subdomains. It also correlated significantly with the SCQ-L total score (p < .001) and negatively with IQ (p = .006). ASD shared more variance with the ADI-R subdomains (r2 = .23– .49) and the SCQ-L (r2 = .55) than with IQ (r2 = .12).

EBP correlated significantly with the ADI-R Social Interaction (p < .001), Verbal Communication (p = .002), and Repetitive Behaviour (p < .001) subdo- mains but not with Nonverbal Communication (p = .068). EBP correlated significantly with the SCQ-L (p < .001) and negatively with IQ (p = .040). Relative to ASD, EBP generally shared less variance with the ADI- R subdomains (r2 = .13– .21) and the SCQ-L (r2 = .23). EBP also shared less variance with IQ (r2 = .07) than it did with the ADI-R and SCQ-L.

Results for the Total-10 and Total-12 were similar to those found for ASD. The Total-10 correlated signifi- cantly with the ADI-R Social Interaction (p < .001), Ver- bal (p < .001) and Nonverbal Communication (p = .004), and Repetitive Behaviour subdomains (p < .001). It also correlated significantly with the SCQ-L (p < .001) and negatively with IQ (p = .003). However, it shared more variance with the ADI-R subdomains (r2 = .28– .52) and SCQ-L (r2 = .58) than it did with IQ (r2 = .13). The Total-12 correlated significantly with the ADI-R’s Social Interaction (p < .001), Verbal (p < .001) and Nonverbal Communication (p = .018), and Repetitive Behaviour (p < .001) subdomains and with the SCQ-L (p < .001). Its negative correlation with IQ (p = .014) was also sig- nificant. The Total-12 shared far more variance with the ADI-R subdomains (r2 = .20– .48) and the SCQ-L (r2 = .56) than it did with IQ (r2 = .09).

Table 3. Factor pattern coefficients for the two-factor model from confirmatory factor analysis (n = 193).

Item

Factor

ASD EBP

1 Social interaction with adults 65 – 2 Social interaction with peers 87 – 3 Expressive language absent/virtually absent 65 – 6 Outstanding and obsessive interests 66 – 7 Stereotyped or unusual handling of objects 65 – 8 Stereotyped manipulation of own body, no objects involved 55 – 9 Strong dependence on fixed patterns, routines, or rituals – 58

10 Self-injury – 61 11 Highly erratic unpredictable behaviour – 60 12 Unusual or unreasonable and excessive anxiety or panic – 27

Note. ASD = ASD factor; EBP = Emotional and Behavioural Problems factor. Decimals were eliminated from all coefficients. All coefficients statistically significant, with p < .001 except for Item 12. Its factor loading was statisti- cally significant, with p = .018. ASD and EBP factors were positively and sig- nificantly correlated (.71; p < .001).

Table 4. Convergent and discriminant correlations for PDD-MRS factor and total scores (n = 71).

ADI-R

SI Va NVb RB SCQ-L IQc

ASD 70*** 48** 49** 59*** 74*** –34** EBP 45*** 46** 36 44*** 48*** –26* Total-10 72*** 55*** 53** 63*** 76*** –36** Total-12 67*** 53*** 45* 69*** 75*** –30*

Note. PDD-MRS = Pervasive Developmental Disorder in Mental Retardation Scale; ADI-R = Autism Diagnostic Interview – Revised; SI = Social Inter- action; V = Communication-Verbal; NV = Communication-Nonverbal; RB = Restricted, Repetitive Behaviour; SCQ-L = Social Communication Questionnaire – Lifetime Total Score; ASD = ASD factor-based scale score; EBP = Emotional and Behavioural Problems factor-based scale score; Total- 10 = PDD-MRS 10-item Total Score; Total-12 = PDD-MRS 12-item Total Score based on Kraijer scoring algorithm.

an = 44. bn = 27. cn = 65. *p < .05. **p < .01. ***p < .001.

JOURNAL OF INTELLECTUAL & DEVELOPMENTAL DISABILITY 67

In sum, ASD, the Total-10, and the Total-12 exhibited generally positive and moderate correlations with the ADI-R and SCQ-L and fairly small negative correlations with IQ. When compared to the ASD, Total-10, and Total-12 scales, EBP correlated to a lesser extent with the ADI-R subdomains and the SCQ-L. The pattern of correlations provided evidence for convergent and discri- minant validity.

Criterion-related evidence

Two-tailed independent t tests and the Dunn–Bonfer- roni correction for multiple comparisons (αDB < .0125) to keep the experimentwise alpha at .05 (αEW < .05) were used to compare the mean ASD, EBP, Total-10, and Total-12 scores of two subgroups: children with DS (n = 38) and children with DS + ASD (n = 33). Table 5 contains the results.

The group of children with DS + ASD had means that were significantly higher (all ps < .001) on the ASD, Total-10, and Total-12 than the group of children with DS alone. Hedge’s g (see Hedges & Olkin, 1985) effect size estimates indicated large mean differences. These effect sizes indicated that 83.7–89.4% of children with DS + ASD exceeded the mean scores of the group of chil- dren with DS only.

The mean EBP score was not significantly different (p = .246) between the two groups. The effect size was small (g = 0.28) and indicated that 61% of individuals with DS + ASD exceeded the mean score of the group of children with DS.

Diagnostic accuracy

ROC analyses Several ROC analyses determined which PDD-MRS scores best distinguished individuals with DS + ASD from individuals with DS only. The area under the

curve (AUC) indicates the proportion of cases correctly classified. A value of 1.0 indicates perfect diagnostic accuracy and .50 indicates chance-level classification. The results are presented in Table 6.

Table 6 shows that the AUCs for the ASD factor, Total-10, and Total-12 were statistically significant (all ps < .001; αDB < .0125 using the Dunn–Bonferroni cor- rection for multiple comparisons). The AUC for EBP was not statistically significant (p = .276) and its AUC was significantly lower (αDB < .008) than the AUCs for ASD (z = 3.28, p < .001), Total-10 (z = 4.43, p < .001), and Total-12 (z = 3.28, p < .001). No other significant differences were found among the AUCs. The ASD fac- tor-based scale evidenced the best trade-off between sen- sitivity (.909) and specificity (.605), with a cut-off score of 1.5. Adjusting all of the cut scores upward did not appreciably improve the generally low specificity values without significantly sacrificing sensitivity.

Predictive discriminant analyses Because scale reliability, criterion-related validity, and diagnostic accuracy were generally unfavourable for the EBP scale, predictive discriminant analysis (PDA) allowed us to see if EBP items limited the diagnostic accuracy of the Total-10 and Total-12. Items comprising EBP are included in Total-10 and Total-12 but not ASD. PDAs produced hit rates for ASD, EBP, Total-10, and Total-12 across all subjects (n = 71) and within the sub- groups of children with DS and children with DS + ASD. All PDAs were cross-validated on 1000 bootstrap samples using the “leave-one-out” method (Lachen- bruch, 1967, 1968).

The more stringent maximum chance criterion of 53.5% was used for the null value instead of 50%, which is typical for two-group analyses. With n = 71, and 38 participants with DS, one would have correctly classified 53.5% (38/71) by simply assigning all partici- pants to the group with DS. The maximum chance cri- terion is possible for overall group but not for separate group hit rates. For the separate subgroup hit rate

Table 5. Significance tests on mean PDD-MRS factor and total scores between diagnostic subgroups.

Score

Group means and standard deviations

DS DS + ASD p ga % Above meanb of group with DS

ASD 1.55 (1.52) 3.61 (1.75) < .001 1.25 89.4 EBP 1.34 (1.15) 1.67 (1.19) .246 0.28 61.0 Total-10 2.89 (2.38) 5.27 (2.28) < .001 1.01 84.4 Total-12 6.32 (4.24) 10.39 (3.98) < .001 0.98 83.7

Note. PDD-MRS = Pervasive Developmental Disorder in Mental Retardation Scale; DS = group with Down syndrome only (n = 38); DS + ASD = group with Down syndrome and ASD (n = 33); ASD = ASD factor-based scale score. EBP = Emotional and Behavioural Problems factor-based scale score; Total-10 = PDD-MRS 10-item Total Score; Total-12 = PDD-MRS 12-item Total Score with Kraijer algorithm.

aHedges’ g (Hedges & Olkin, 1985). bApproximate percentage of scores in the group with DS + ASD that fell above the mean for the group of children with DS.

Table 6. Diagnostic accuracy of PDD-MRS factor and total scores (n = 71).

ASD EBP Total-10 Total-12

AUC .806* .575 .773* .769* 95% CI [.700, .911] [.442, .709] [.663, .882] [.656, .882] Sensitivity .909 .818 .879 .848 Specificity .605 .289 .526 .526 Cut Score 1.5 0.5 2.5

Note. Diagnostic accuracy statistics for children with DS only versus children with DS + ASD. nASD = 33, nDS = 38. ASD = ASD factor-based scale score; EBP = Emotional and Behavioural Problems factor-based scale score; Total- 10 = PDD-MRS 10-item Total Score; Total-12 = PDD-MRS 12-item Total Score with Kraijer algorithm.

*p < .001, which was statistically significant at α < .0125 using Dunn–Bonfer- roni correction to keep αEW < .05. For the EBP factor, p = .276.

68 V. PANDOLFI ET AL.

analyses the proportional chance criterion was used for the null value.

ASD had a statistically significant hit rate of 74.6% (z = 3.57, p < .001) and the EBP hit rate was 53.5% (z = 0.00, p < 1.00), which was no better than chance. When the two scales were used together, the hit rate was 73.2%, which was statistically significant (z = 3.33, p = .001) but was less than the hit rate of ASD alone. For these reasons, the EBP factor was not included in the remaining PDAs.

Table 7 contains PDA results for the ASD factor and two total scores. One can see the overall hit rates across all subjects, and hit rates for the group of children with DS only and the group of children with DS + ASD.

Table 7 shows that the ASD (74.6%), Total-10 (69.0%), and Total-12 (69.0%) scales all had statistically significant overall hit rates. McNemar’s tests compared the diagnostic accuracy of ASD versus the Total-10 and Total-12 scales. These two tests were not significant at a traditional alpha level [.05; both χ2(1) = 2.67, p = .103] but were consistent with the pattern of diagnos- tic accuracy found in the ROC analyses: the ASD scale had the highest hit rate. Table 7 also shows the improve- ment over chance (I) index. The ASD scale had an I = 48.9%, which indicated that this scale led to about 49% fewer classification errors than if classification were done by chance. It had the best improvement over chance results in comparison to the Total-10 and Total-12 (both Is = 37.6%).

Hit rates were also computed for the group of children with DS and the group with DS + ASD separately. As seen in Table 7, all three scales evidenced statistically

significant hit rates for the group with DS + ASD; how- ever, only ASD had a significant hit rate for the group with DS. This means that ASD was the only scale that accurately identified members of both diagnostic sub- groups: children with and without ASD.

Discussion

The increased recognition that a subset of children with DS can also have ASD highlights the need for early identification and treatment to reduce the morbidity associated with this diagnostic co-occurrence. Evaluation for ASD in children, including children with DS, is guided by published professional health guidelines (Bull & the Committee on Genetics, 2011; Johnson et al., 2007) but challenges exist for many practitioners in distinguishing ASD symptoms from the developmen- tal impairments associated with DS. This may be particu- larly true if they do not apply a developmental assessment model and use reliable and valid ASD assess- ment measures (Magyar et al., 2012). The professional practice guidelines on ASD assessment do not provide sufficient information on reliable and valid ASD screen- ing measures for use in children with DS.

Prior to this study of the PDD-MRS, to our knowl- edge, the only ASD screening measure validated for use in children with DS was the SCQ-L (Magyar et al., 2012). The present study sought to add to this limited research base by examining the psychometric properties of the PDD-MRS in a well-characterised sample of chil- dren with DS with and without ASD. Overall findings indicate that the PDD-MRS can be considered a useful measure for ASD screening of children with DS aged 3–15 years. The findings are especially relevant given that diagnosis is often delayed in this population relative to children without DS.

One of the two scales that emerged from the factor analyses, the ASD factor-based scale, contained those items that assess for the core ASD impairments. This includes impairments in social communication and interaction and the presence of repetitive behaviour and activities, which is consistent with the current DSM-5 (5th ed., APA, 2013) conceptualisation of ASD. The factor was found to be reliable and it correlated sig- nificantly with the SCQ-L, the ADI-R, and with IQ, although it shared more variance with the ASD measures than with IQ. These findings suggest that the scale is not merely a measure of developmental level.

The ASD factor also evidenced better diagnostic accuracy compared to the other three scales that were evaluated: the factor analytically derived EBP scale, the Total-10, and Total-12 scales. Even though children were diagnosed with DSM-IV-TR criteria within the

Table 7. Predictive discriminant analysis for ASD factor, Total-10, and Total-12 scores (n = 71). Predictor variable Hit rate (%) z p I(%)

ASD Overall 74.60 3.57a <.001 48.89 Subgroups DS + ASD 75.76 3.37b < .001 DS only 73.68 2.49b .013

Total-10 Overall 69.00 2.62a .009 37.63 Subgroups DS + ASD 66.67 2.33b .020 DS only 71.05 2.17b .030

Total-12 Overall 69.00 2.62a .009 37.63 Subgroups DS + ASD 72.73 3.02b .003 DS only 65.79 1.52b .129

Note. nASD = 33; nDS = 38. I(%) = Improvement over chance index; ASD = ASD factor-based scale score; Total-10 = PDD-MRS 10-item Total Score; Total-12 = PDD-MRS 12-item Total Score with Kraijer algorithm. All significance tests conducted at α < .025 using Dunn–Bonferroni correction to keep αEW < .05 for each predictive analysis.

aNull value based on maximum chance criterion. bNull value based on pro- portional chance criterion.

JOURNAL OF INTELLECTUAL & DEVELOPMENTAL DISABILITY 69

context of a developmental approach to assessment, the newly released DSM-5 does not reflect significant sub- stantive changes to the diagnostic criteria. In fact, the DSM-5 indicates that those children diagnosed with autistic disorder, Asperger disorder, or PDD-NOS using DSM-IV-TR criteria should now be diagnosed with ASD (APA, 2013). Results of this study indicate that, within the context of a developmental approach to assessment, the ASD factor-based scale can be used in routine ASD screening in children with DS.

The second factor that emerged from the analysis was the EBP factor. This factor-based scale contained items that included rigidity, self-injury, unpredictable behav- iour, and extreme anxiety. These problems are not specific to an ASD diagnosis but are often observed in children with a wide range of developmental disorders including ASD, DS, and ID. These problems may be seen as part of the behavioural phenotype of DS and co-occurring ASD. The EBP factor shared far less var- iance with the ASD measures than the ASD factor, and its performance in the diagnostic accuracy tests was poor. It seems the presence of these items on the PDD- MRS introduces “noise” into the scoring algorithms of the two total scores, thus leading to more false positives for ASD. However, this factor seems to have some poten- tial value in identifying other problems that may warrant additional assessment, particularly if the caregiver reports concerns about such behaviours being related to functional impairment or distress. In these situations, the Child Behaviour Checklist (Achenbach & Rescorla, 2000, 2001) or the Aberrant Behaviour Checklist (Aman, Singh, Stewart, & Field, 1985) could be adminis- tered to further assess the child for a co-occurring emotional and/or behaviour disorder. Both of these measures have psychometric evidence to support their use in children with ASD and ID (Kaat, Lecavalier, & Aman, 2014; Pandolfi, Magyar, & Dill, 2009, 2012).

The ASD factor may prove particularly useful in tele- health service delivery models, a model that is increas- ingly being approved by many health insurance companies in the United States, which can improve access for families to primary care. Moreover, the ASD factor if used as an interview can be used in combination with paper-and-pencil ASD screeners, such as the SCQ-L, where interview data are needed to further distinguish the manifest ASD symptoms, a practice consistent with the recommendation that information from multiple sources can improve clinical decision-making in ASD evaluation (Pandolfi & Magyar, 2014; Risi et al., 2006).

This study has some limitations. One was the necess- ary removal of two PDD-MRS items from the factor ana- lyses due to the way the PDD-MRS is scored. Scores for Items 4 (deviant language content) and 5 (deviant

language production) depend on the response to Item 3 (absence of expressive language). If a child does not have adequate expressive language, then Items 4 and 5 must always be scored 0. This dependency can lead to biased estimates of inter-item correlations and inaccur- ate factor analytic results. Although this pattern of scores may align with clinical presentation for many children with DS because of speech-language impairments, future consideration should be given to resolving this psycho- metric issue so that these atypical behaviours can be included in evaluations of the measure.

Another limitation has to do with generalisation of the findings. The study used data from participants recruited from New York State living outside of New York City. Although the sample’s demographics were generally consistent with other published research in ASD and DS, it was representative of participants from predominantly white, well-educated, middle-class families. Therefore, replication of this study’s method- ology with a sample of participants from more diverse geographic locations and demographic subgroups seems warranted. It is also recommended that this study be replicated with children with DS younger than 3 years of age to determine if the ASD factor- based scale can be useful for the 18- and 24-month ASD screening recommended by the American Acad- emy of Pediatrics (Johnson et al., 2007).

Acknowledgements

The authors thank the parent/child evaluation team and the parents and children who participated in the study at the Uni- versity of Rochester Medical Centre.

Disclosure statement

No potential conflict of interest was reported by the authors.

Funding

This work was supported by the AUCD under Grant RTO1 2005-1/2-08.

References

Achenbach, T. M., & Rescorla, L. A. (2000). Manual for the ASEBA Preschool Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, and Families.

Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA School-Age Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, and Families.

Aman, M. G., Singh, N. N., Stewart, A. W., & Field, C. J. (1985). The Aberrant Behavior Checklist: A behavior rating

70 V. PANDOLFI ET AL.

scale for the assessment of treatment effects. American Journal of Mental Deficiency, 89, 485–491.

American Psychiatric Association. (1987). Diagnostic and stat- istical manual of mental disorders (3rd ed., rev.). Washington, DC: Author.

American Psychiatric Association. (2000). Diagnostic and stat- istical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.

American Psychiatric Association. (2013). Diagnostic and stat- istical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.

Arunyanart, W., Fenick, A., Ukritchon, S., Imjaijitt, W., Northrup, V., & Weitzman, C. (2012). Developmental and autism screening: A survey across six states. Infants & Young Children, 25, 175–187. doi:10.1097/IYC. 0b013e31825a5a42

Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238–246. doi:10.1037/ 0033-2909.107.2.238

Bodfish, J. W., Symons, F. J., & Lewis, M. H. (1999). The Repetitive Behavior Scale: A test manual. Morganton, NC: Western Carolina Center Research Reports.

Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York, NY: Guilford Press.

Bull, M. J., & the Committee on Genetics. (2011). Health supervision for children with Down syndrome. Pediatrics, 128, 393–406. doi:10.1542/peds.2011-1605

Channell, M. M., Phillips, B. A., Loveall, S. J., Conners, F. A., Bussanich, P. M., & Klinger, L. G. (2015). Patterns of autism spectrum symptomatology in individuals with Down syn- drome without comorbid autism spectrum disorder. Journal of Neurodevelopmental Disorders, 7, 1–9. doi:10. 1186/1866-1955-7-5

Dosen, A. (2005). Applying the developmental perspective in the psychiatric assessment and diagnosis of persons with intellectual disability: Part I – assessment. Journal of Intellectual Disability Research, 49, 1–8. doi:10.1111/j. 1365-2788.2005.00656.x

Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9, 466–491. doi:10.1037/1082-989X.9.4.466

Floyd, F. J., & Widaman, K. F. (1995). Factor analysis in the development and refinement of clinical assessment instru- ments. Psychological Assessment, 7, 286–299. doi:10.1037/ 1040-3590.7.3.286

Green, S. B., & Yang, Y. (2009). Commentary on coefficient alpha: A cautionary tale. Psychometrika, 74, 121–135. doi:10.1007/s11336-008-9098-4

Greer, T., Dunlap, W. P., & Beatty, G. O. (2003). A Monte Carlo evaluation of the tetrachoric correlation coefficient. Educational and Psychological Measurement, 63, 931–950. doi:10.1177/0013164403251318

Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta- analysis. San Diego, CA: Academic Press.

Hepburn, S., Philofsky, A., Fidler, D. J., & Rogers, S. (2008). Autism symptoms in toddlers with Down syndrome: A descriptive study. Journal of Applied Research in Intellectual Disabilities, 21, 48–57. doi:10.1111/j.1468- 3148.2007.00368.x

Howlin, P., Wing, L., & Gould, J. (1995). The recognition of autism in children with Down syndrome—implications

for intervention and some speculations about pathology. Developmental Medicine & Child Neurology, 37, 406–414. doi:10.1111/j.1469-8749.1995.tb12024.x

Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. doi:10.1080/10705519909540118

Johnson, C. P., Myers, S. M., & the Council on Children With Disabilities. (2007). Identification and evaluation of chil- dren with autism spectrum disorders. Pediatrics, 120, 1183–1215. doi:10.1542/peds.2007-2361

Jöreskog, K., & Sörbom, D. (2006). LISREL 8.80 [Computer soft- ware]. Lincolnwood, IL: Scientific Software International, Inc.

Kaat, A. J., Lecavalier, L., & Aman, M. G. (2014). Validity of the Aberrant Behavior Checklist in children with autism spectrum disorder. Journal of Autism and Developmental Disorders, 44, 1103–1116. doi:10.1007/s10803-013-1970-0

Kraijer, D. W. (2006). Pervasive Developmental Disorder in Mental Retardation Scale manual: Second revised edition. PITS B.V. Leiden: Psychologische Instrumenten Tests En Services.

Lachenbruch, P. A. (1967). An almost unbiased method of obtaining confidence intervals for the probability of misclassification in discriminant analysis. Biometrics, 23, 639–645. doi:10.2307/2528418

Lachenbruch, P. A. (1968). On expected probabilities of mis- classification in discriminant analysis, necessary sample size, and a relation with multiple correlation coefficient. Biometrics, 24, 823–834. doi:10.2307/2528873

Lord, C., Rutter, M., DiLavore, P. C., & Risi, S. (2002). Autism Diagnostic Observation Schedule. Los Angeles, CA: Western Psychological Services.

MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covari- ance structure modeling. Psychological Methods, 1, 130–149. doi:10.1037/1082-989X.1.2.130

Magyar, C. I., Pandolfi, V., & Dill, C. A. (2012). An initial evaluation of the social communication questionnaire for the assessment of autism spectrum disorders in children with Down syndrome. Journal of Developmental & Behavioral Pediatrics, 33, 134–145. doi:10.1097/dbp. 0b013e318240d3d9

Molloy, C. A., Murray, D. S., Kinsman, A., Castillo, H., Mitchell, T., Hickey, F. J., & Patterson, B. (2009). Differences in the clinical presentation of Trisomy 21 with and without autism. Journal of Intellectual Disability Research, 53, 143–151. doi:10.1111/j.1365-2788.2008.01138.x

Moss, J., & Howlin, P. (2009). Autism spectrum disorders in genetic syndromes: Implications for diagnosis, intervention and understanding the wider autism spectrum disorder population. Journal of Intellectual Disability Research, 53, 852–873. doi:10.1111/j.1365-2788.2009.01197.x

Pandolfi, V., & Magyar, C. I. (2014). Assessment of co-occur- ring emotional and behavioral disorders in youth with ASD using the Child Behavior Checklist 6–18. In V. B. Patel, V. R. Preedy, & C. Martin (Eds.), The comprehensive guide to autism (pp. 2799–2812). New York, NY: Springer.

Pandolfi, V., Magyar, C. I., & Dill, C. A. (2009). Confirmatory factor analysis of the Child Behavior Checklist 1.5–5 in a sample of children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 39, 986– 995. doi:10.1007/s10803-009-0716-5

JOURNAL OF INTELLECTUAL & DEVELOPMENTAL DISABILITY 71

Pandolfi, V., Magyar, C. I., & Dill, C. A. (2012). An initial psy- chometric evaluation of the CBCL 6–18 in a sample of youth with autism spectrum disorders. Research in Autism Spectrum Disorders, 6, 96–108. doi:10.1016/j.rasd.2011.03. 009

Rasmussen, P., Börjesson, O., Wentz, E., & Gillberg, C. (2001). Autistic disorders in Down syndrome: Background factors and clinical correlates. Developmental Medicine & Child Neurology, 43, 750–754. doi:10.1111/j.1469-8749.2001. tb00156.x

Raykov, T. (1997). Estimation of composite reliability for con- generic measures. Applied Psychological Measurement, 21, 173–184. doi:10.1177/01466216970212006

Raykov, T. (2001). Estimation of congeneric scale reliability using covariance structure analysis with nonlinear con- straints. British Journal of Mathematical and Statistical Psychology, 54, 315–323. doi:10.1348/000711001159582

Reilly, C. (2009). Autism spectrum disorders in Down syn- drome: A review. Research in Autism Spectrum Disorders, 3, 829–839. doi:10.1016/j.rasd.2009.01.012

Reiss, S., Levitan, G. W., & Szyszko, J. (1982). Emotional disturbance and mental retardation: Diagnostic oversha- dowing. American Journal of Mental Deficiency, 86, 567– 574.

Risi, S., Lord, C., Gotham, K., Corsello, C., Chrysler, C., Szatmari, P., … Pickles, A. (2006). Combining information from multiple sources in the diagnosis of autism spectrum disorders. Journal of the American Academy of Child & Adolescent Psychiatry, 45, 1094–1103. doi:10.1097/01.chi. 0000227880.42780.0e

Robins, D. L., Fein, D., Barton, M. L., & Green, J. A. (2001). The Modified Checklist for Autism in Toddlers: An initial study investigating the early detection of autism and perva- sive developmental disorders. Journal of Autism and

Developmental Disorders, 31, 131–144. doi:10.1023/ A:1010738829569

Roid, G. H., & Miller, L. J. (1997). Leiter International Performance Scale – Revised. Los Angeles, CA: Western Psychological Services.

Rutter, M., Bailey, A., & Lord, C. (2003). Social Communication Questionnaire. Los Angeles, CA: Western Psychological Services.

Rutter, M., Le Couteur, A. L., & Lord, C. (2003). Autism Diagnostic Interview – Revised. Los Angeles, CA: Western Psychological Services.

Salvia, J., & Ysseldyke, J. E. (2004). Leiter international perform- ance scale – revised. In J. Salvia & J. E. Ysseldyke (Eds.), Assessment in special and inclusive education (9th ed., pp. 352–354). Boston, MA: Houghton Mifflin Company.

Satorra, A., & Bentler, P. M. (2001). A scaled difference chi- square test statistic for moment structure analysis. Psychometrika, 66, 507–514. doi:10.1007/BF02296192

Sijtsma, K. (2009). On the use, the misuse, and very limited usefulness of Cronbach’s alpha. Psychometrika, 74, 107–120. doi:10.1007/s11336-008-9101-0

Sparrow, S. S., Cicchetti, D. V., & Balla, D. A. (2005). Vineland Adaptive Behavior Scales (2nd ed.). Circle Pines, MN: American Guidance Service.

Steiger, J. H., & Lind, J. M. (1980, May). Statistically based tests for the number of common factors. Paper presented at the meeting of the Psychometric Society, Iowa City, IA.

Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: Current approaches and future directions. Psychological Methods, 12, 58–79. doi:10.1037/1082-989X.12.1.58

Yang-Wallentin, F., Jöreskog, K. G., & Luo, H. (2010). Confirmatory factor analysis of ordinal variables with misspecified models. Structural Equation Modeling, 17, 392–423. doi:10.1080/10705511.2010.489003

72 V. PANDOLFI ET AL.

Copyright of Journal of Intellectual & Developmental Disability is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.