Systematic Review

TelehealthandAustism.Sutherland..pdf

Home >Nursing homework help >Systematic Review

INT J LANG COMMUN DISORD, MARCH/APRIL 2019, VOL. 54, NO. 2, 281–291

Special Issue

Telehealth and autism: Are telehealth language assessments reliable and feasible for children with autism?

Rebecca Sutherland†§, David Trembath‡¶, Marie Antoinette Hodge§, Veronica Rose‡¶‖ and Jacqueline Roberts† †Autism Centre of Excellence, Griffith University, Mt Gravatt, QLD, Australia ‡School of Allied Health Sciences, Gold Coast Campus, Griffith University, QLD, Australia §Child Development Unit, Children’s Hospital at Westmead, Westmead, NSW, Australia ¶Menzies Health Institute Queensland, Gold Coast, QLD, Australia ‖Neurodisability and Rehabilitation Research, Murdoch Children’s Research Institute; Department of Paediatrics, University of Melbourne

(Received January 2018; accepted October 2018)

Abstract

Background: Access to timely and appropriate speech–language pathology (SLP) services is a significant challenge for many families. Telehealth has been used successfully to treat a range of communication disorders in children and adults. Research examining the use of telehealth for children with autism has focused largely on diagnosis, parent-implemented interventions, and behavioural interventions involving interactions between clinicians and parents. There is, however, very limited research into the use of telehealth directly to assess or intervene with children with autism. This paper reports the outcomes of a study of telehealth language assessments with primary school-aged children with autism. Aims: To evaluate the reliability and feasibility of telehealth language assessments for school-aged children with autism. Methods & Procedures: The language skills of 13 children with autism aged 9–12 who attended mainstream schools or support classes were assessed using the Clinical Evaluation of Language Fundamentals—4th Edition. An SLP delivered and scored four subtests of the assessment via telehealth from a remote location. A second SLP at the same location as the child co-scored the online subtests to provide a measure of reliability and delivered the remaining subtests. The local SLP completed checklists in both conditions to provide observations regarding behaviour. Parent feedback was elicited via survey. Outcomes & Results: There was strong interrater reliability between the telehealth and face-to-face conditions (correlation coefficients ranged from r = 0.919 to 0.990 across the subtests and Core Language Score) and good agreement between clinicians on all measures. Analysis using the Wilcoxon Signed Rank test indicated no significant differences in children’s behaviour between the telehealth and face-to-face conditions, although variation between individuals was observed. Parents provided generally positive feedback about the use of telehealth for the assessments. Conclusions & Implications: The findings of this study provide preliminary support the use of telehealth assessments of school-aged children with autism. Comparison of telehealth and face-to-face assessment scores showed high agreement and correlation, and while the children showed individual differences in their behaviour during the telehealth sessions, there was no clear difference between the conditions at the group level. The findings sug- gest that telehealth may present a reliable and feasible approach to the assessment of language for children with autism in some circumstances as a primary or adjunct service model, while acknowledging that individual differ- ences among these children may be important to consider when planning both assessment and intervention via telehealth.

Address correspondence to: Rebecca Sutherland, Autism Centre of Excellence, Griffith University, Messines Ridge Road, Mt Gravatt QLD 4101, Australia; e-mail: [email protected]

DOI: 10.1111/1460-6984.12440

282 Rebecca Sutherland et al.

What this paper adds What is already known on the subject Telehealth has been used successfully to assess and treat a range of communication disorders in adults and children. It has been used to help parents of children with autism implement interventions, but there is little research about the use of telehealth directly with the children themselves.

What this paper adds to existing knowledge This study indicates that the use of telehealth to assess the language skills of children with autism is reliable, and that while children showed individual differences in their behaviour during the sessions, there was no clear difference between the telehealth and face-to-face conditions at the group level.

What are the potential or actual clinical implications of this work? Telehealth may be a useful primary or adjunct way of delivering assessment for children with autism, but it is important for clinicians to consider the needs of individual children and families.

Introduction

Access to timely, affordable and locally delivered speech– language services remains a barrier for many children with speech and language difficulties, particularly in re- gional, rural and remote areas in which speech pathol- ogy services are limited (Australian Institute of Health and Welfare (AIHW) 2009, Community Affairs Ref- erences Committee 2014). In the Australian context, a 2014 Senate inquiry found that not only were there fewer speech–language pathologists (SLPs) working out- side urban areas, but also the ratio of therapists per 100 000 people declined as rurality increased (Commu- nity Affairs References Committee 2014). Telehealth is one way of addressing some of the barriers associated with staffing and access issues and it has been shown to be an acceptable and viable alternative to face-to-face speech–language services in a range of specialties, includ- ing dysphagia assessment and intervention (Collins et al. 2017, Ward et al. 2013, Sharma et al. 2013), stuttering therapy (O’Brian et al. 2014, Bridgman et al. 2016), and assessment and intervention for speech-sound dis- orders, including childhood apraxia of speech (Grogan- Johnson et al. 2013, Thomas et al. 2016, Waite et al. 2012). Telehealth has also been used to provide for- mal assessments of child language with young children with no known language difficulties (Waite et al. 2010), as well as children with reading difficulties (Suther- land et al. 2017). There has, however, been a limited focus on children with more complex neurodevelop- mental disabilities, including autism spectrum disorder (ASD).1

ASD is a neurodevelopmental disability character- ized by functional impairments in social communication and social interactions, as well as restricted, repetitive patterns of behaviour, interests or activities (American Psychiatric Association (APA) 2013). Social communi- cation skills and impairments in this group vary widely. Some individuals experience significant impairments in

all areas of communication (including language, speech and pragmatic functions), with minimal intentional communication use and/or significantly disordered language, while others with autism may have more subtle difficulties with conversation and impaired prag- matic language, and higher level language difficulties (Gernsbacher et al. 2015). For those with autism and a comorbid language disorder, difficulties may include impairments in structural language and receptive language abilities (Loucas et al. 2008), grammatical errors or limited vocabulary (Wittke et al. 2017). Given the communication difficulties experienced by many children and adults with autism, SLPs have an important role in assessing and providing intervention for this population (Speech Pathology Australia 2016). For families in rural and remote areas, however, accessing appropriate services is challenging as there are fewer speech pathologists in these areas, and even fewer professionals experienced in autism (Taylor et al. 2016).

Telehealth has been found to be a viable method of service provision for families of children with autism who are unable to access local services, particularly in the preschool population. It has been used to provide parent training in early intervention (e.g., Ingersoll and Berger 2015, Ingersoll et al. 2016, Pickard et al. 2016, Meadan et al. 2016, Wainer and Ingersoll 2015, Simacek et al. 2017), with positive outcomes reported for com- munication and other skills, including behaviour and imitation, as well as to support older children with autism and co-occurring anxiety (Hepburn et al. 2015) and to provide functional behavioural assessment and functional communication training for children with behaviour support needs (Lindgren et al. 2016, Suess et al. 2014) via parent training. Diagnostic assessments for autism have been completed via telehealth with adults (Schutte et al. 2015). Diagnostic assessments via telehealth for children have also been researched, with parents coached via telehealth to deliver assessment

Telehealth and autism 283

items to their children (Reese et al. 2013, Reese et al. 2015).

While early intervention, diagnostic and behaviour support services for people with autism have been pro- vided via telehealth, few studies have focused on SLP services for children with autism directly, particularly those of school age. In a recent systematic search and review of the literature (Sutherland et al. 2018) only one speech pathology-based study was identified (Boisvert et al. 2012), involving a single participant undergoing language intervention. Further, the majority of stud- ies concerning autism and telehealth involved training parents or teachers to facilitate assessments or inter- vention, rather than the use of telehealth to interact directly with the child on the autism spectrum. As a result, little is known about the response of children with autism to telehealth, nor whether telehealth ser- vices that have been established with other groups of children can be reliably and feasibly used with this population.

The potential response of children and young people with autism to telehealth services is not known. While it is commonly suggested anecdotally that children and young people on the autism spectrum have a high in- terest in, or use of, computers and other electronic me- dia, there is limited research to allow us to understand whether, and how, this is different from other children (Mazurek et al. 2012, Orsmond and Kuo 2011) and how this might relate to telehealth. Small-scale research has suggested that children with autism may engage in more screen-based activities including watching televi- sion than children without autism (Must et al. 2014); however, larger studies (Montes 2016) have found no significant differences between groups. In the one study in the SLP literature noted above, Boisvert et al. (2012) examined this issue in a case study of an 11-year-old boy who was provided with language intervention in both telehealth and face-to-face settings, targeting the same goal. Results indicated that the child achieved greater levels of accuracy in the telehealth condition and it was noted anecdotally that the child was more attentive and actively participated in the telehealth sessions with little or no redirection required. The authors argued that the telehealth setting may provide a more consistent and predictable social environment, reducing anxiety and consequent adverse behaviours; however, there is very limited high-quality research, with adequate sample size and research design, to guide clinical decision-making regarding the use of telehealth with children on the autism spectrum.

Given the clinical need to provide services for chil- dren with autism, limited services located in rural and regional areas and limited research regarding tele- health services delivered directly to children with autism, the aim of the present study was to investigate the

reliability of language assessments for children on the autism spectrum, delivered via telehealth, replicating the design of a previous study in this area (Sutherland et al. 2017). The secondary aim was to explore the feasibility of the use of telehealth with children with autism, exploring their behavioural responses to tele- health and face-to-face assessment conditions. Based on previous research, it was hypothesized that the reliabil- ity of the assessments delivered via telehealth would be high (Waite 2010, withheld for blind review). How- ever, given the limited information currently available about child responses to telehealth, we had no specific hypotheses about the relationship between child be- haviour and the telehealth and face-to-face assessment conditions.

Materials and methods

Design

A method comparison design (Bland and Altman 1999) was used to measure the agreement between the scores for each language assessment subtest recorded by the telehealth SLP compared with those made simultane- ously by the face-to-face SLP. Method comparison allows researchers to establish variation between two methods of measurement. In this study, it meant that the reliabil- ity of the telehealth assessment compared with face-to- face assessment could be measured without a test–retest effect.

The independent variable in the present study was the condition under which the assessment was con- ducted (telehealth versus face to face). The dependent variables were: (1) the language assessment scores given by the telehealth and face-to-face SLPs which were com- pared to provide a measure of reliability; and (2) be- haviour observation scores made by the face-to-face SLP in the face-to-face and telehealth conditions, providing measures of the feasibility of the telehealth condition with respect to the children’s behaviour.

Participants

Eligibility criteria

To be included in the study, children were required to meet the following criteria: (1) aged 9–12 years (to allow the use of a single set of core language assess- ment subtests); (2) attending a school placement in a mainstream environment or support class within a mainstream school; (3) parent perceptions of adequate verbal language to participate in a standardized lan- guage assessment; and (4) a parent-reported commu- nity diagnosis (e.g., by paediatrician) of ASD (inclu- sive of DSM-IV diagnoses such as Asperger syndrome).

284 Rebecca Sutherland et al.

Table 1. Participant characteristics

Participant Age (years;months) Gender School placement CLS % Lowest AE Highest AE SCQ

1 10;1 Male Support class 85 16 6;09 12;03 20 2 10;0 Female Mainstream 100 50 9;02 16;03 13 3 11;1 Male Mainstream 108 66 9;02 10;09 23 4 11;11 Male Mainstream 93 32 8;01 12;03 13 5 12;2 Male Mainstream 78 7 8;02 11;09 8a

6 9;5 Male Mainstream 55 0.1 <7;0 7;09 27 7 11;4 Male Mainstream 73 4 8;09 10;09 19 8 11;0 Male Support class 68 2 <7;0 7;10 20 9 11;10 Female Mainstream 113 81 9;05 >21;11 13

10 9;9 Male Mainstream 85 16 6;09 >9;11 b

11 12;3 Male Mainstream 90 25 7;09 19;03 13 12 12;0 Female Mainstream 87 19 9;03 10;03 16 13 11;9 Male Mainstream 59 0.3 6;10 9;10 6a

Notes: a Autism diagnosis confirmed by ADOS-2 assessment. b ADOS only. Age = age at assessment; CLS, CELF-4 Core Language Score; %, percentile rank of CLS on CELF-4; Lowest AE, lowest age equivalent on any CELF-4 subtest; Highest AE, highest age equivalent on any CELF-4 subtest; SCQ, Social Communication Questionnaire score.

Although not part of the eligibility criteria, it eventu- ated that all children spoke English as their primary language.

Characteristics

Participants were 13 children (three female), with a mean age 11.12 years (range = 9 years 5 months–12 years 3 months), recruited through community contacts and via open invitations on social media.

Core Language Scores as determined by the face-to- face SLP’s scoring during the assessment, ranged from 55 to 113. The Social Communication Questionnaire (SCQ; Rutter et al. 2008) was used to confirm the di- agnosis of autism as part of the study. Severity on the SCQ ranged from 6 to 27. While the accepted cut- off score on the SCQ is 15, the manual allows for a lower cut-off score for comprehensive assessment and several studies have indicated that a cut-off score of 11 provides appropriate sensitivity (Norris and Lecavalier 2010). Two children scored below this cut-off; both un- derwent independent diagnostic assessment using the Autism Diagnostic Observation Schedule—2 (ADOS- 2) as part of regular clinical care and met the cut-off on the algorithm for a diagnosis of autism. A third child was diagnosed using the ADOS-2 as part of routine clinical care and the SCQ was not completed by the parents. Ten children attended mainstream classes and two attended a supported class within a mainstream environment. Table 1 describes the children’s charac- teristics in more detail. Parents were provided with a Social Story (Gray and Garand 1993) to read with their children, which explained the assessment procedure and provided photographs of the clinicians they would meet and the equipment they could expect to see. Parents were encouraged to read these with their children, but

the extent to which this occurred was not controlled by the researchers.

Materials

Assessment tool

Subtests of the Clinical Evaluation of Language Funda- mentals 4th Edition, Australia & New Zealand (CELF-4; Semel et al. 2003) were used in both the telehealth and face-to-face assessment conditions. The CELF-4 (Australia and New Zealand) is an individually admin- istered clinical tool for the identification, diagnosis and follow-up evaluation of language skill deficits in school- age children, normed on Australian children. This for- mal, standardized assessment assesses both receptive and expressive language using a number of subtests that target aspects of vocabulary, syntax, morphology and auditory comprehension. Permission was granted by Pearson to digitize images from CELF-4. These images were scanned and uploaded to the telehealth application. CELF-4 has robust psychometric properties with stabil- ity coefficients ranging from 0.77 to 0.94, interrater reliability ranging from 0.88 to 0.99 and internal con- sistency for the Core Language Score (a composite score indicating the presence or absence of a language impair- ment along with the severity of the impairment) ranging from 0.97 across age ranges. Four subtests are used to compute the Core Language Score in the 9–12-year age group: Concepts and Following Directions, Recall- ing Sentences, Formulated Sentences and Word Classes. The subtests are each scored in different ways: responses on the Concepts and Following Directions and Word Classes subtests are either correct or incorrect (0 or 1); Recalling Sentences responses can be scored from 0 to 3 depending on the number of errors made; and responses

Telehealth and autism 285

on the Formulated Sentences subtest are scored either 0, 1 or 2. The Formulated Sentences subtest requires the administering clinician to make a judgement based on the number of grammatical and/or syntactical errors made by the child. Because of this, Formulated Sen- tences has the lowest level of interrater reliability and some variation between scorers is tolerated in the scor- ing. The other subtests are less subjective but do rely on the clinician accurately seeing, hearing, and recording responses in order to score them accurately.

Equipment

The assessments were conducted using standard in- ternet access (accessed variously from a private home, university and hospital) with consumer-grade equip- ment. The telehealth SLP used a commercially available laptop (PC) computer with built-in speakers, which was connected to a separate monitor (via HMDI cable) with a consumer-grade webcam (Logitech C920) attached. The separate monitor was needed to allow for the full telehealth interface to be displayed. The child site used standard Department of Health computers and internet access with consumer-grade webcams (LogitechC920) and speakers (Logitech PC Z130), along with a commer- cially available touch screen from Dell (model S2240T). The telehealth application used was developed by Coviu (formerly National Information Communications Tech- nology Australia—NICTA). This secure, web-based in- terface included synchronized image viewing and re- motely visible click-markers for pointing at images as well as video conference functionality.

Procedure

Three experienced SLPs were involved in administering and scoring the language assessments. One SLP (the first author) completed all face-to-face administrations with the children; 12 telehealth assessments were completed by a second SLP and one by a third SLP. The face-to-face SLP and the third SLP were familiar with the telehealth application. The second SLP had not used the applica- tion previously but was familiar with videoconferencing technology and other communication applications.

Children were located in Westmead, NSW, with the face-to-face SLP. The telehealth assessment was delivered remotely from Melbourne, VIC, by the telehealth SLP for 12 children and from a second location in Westmead, NSW, for one child (due to timing and clinic constraints of the research team and the family).

The telehealth application allowed the child to see and hear the telehealth SLP and view the digitized im- ages concurrently. For subtests with no visual stimuli the child saw the SLP only. The telehealth SLP could see and hear the child, as well as the visual stimuli and

instructions. The child provided responses to the tele- health SLP over the webcam. The telehealth clinician could also see touch markers immediately when the child touched the screen to select answers on one subtest (Concepts and Following Directions).

The telehealth SLP delivered the four Core Lan- guage subtests via the telehealth application and scored the children’s responses. The face-to-face SLP was in the same location as the child and facilitated the tele- health assessments by turning on and logging in to the computer, testing sound and vision, and troubleshoot- ing throughout. The face-to-face SLP co-scored the four subtests as an observer. Scores determined by the face-to-face SLP were taken as the reference point in the comparisons. Two further subtests (Understanding Paragraphs and Word Definitions) were completed by the face-to-face clinician with the child under standard assessment procedures. These face-to-face assessments were completed either before or after the telehealth ses- sion, based on clinician preference resulting in nonran- domized counterbalanced delivery order. All scoring was completed at the time of assessment.

The face-to-face SLP also completed behaviour ob- servation checklists regarding the children’s attention and other behaviours across both the telehealth and face- to-face conditions in order to provide a measure of the children’s reactions to, and ability to cope with, the two assessment conditions. The subtests were all completed in a single session of up to 60 min in duration. A par- ent was present with their child for the whole assessment and completed questionnaires and the survey during the child’s assessment.

Measures

Reliability

Reliability of the telehealth assessments, and agreement between the assessments, were measured by comparing the CELF-4 scores recorded by the telehealth SLP with those recorded simultaneously by the face-to-face SLP. These measures were collected and scored as per the stan- dard administration instructions for the CELF-4 and Core Language Scores and severity rankings assigned accordingly.

Behaviour

Measures of behaviour were gained using a behaviour observation rating scale originally used and described in full by (withheld for review), that was adapted from the Clinical Evaluation of Language Fundamentals— Preschool, 2nd edition, Australian and New Zealand (CELF-P2; Wiig et al. 2006) behaviour checklist. Ad- ditions were made to the original checklist to allow

286 Rebecca Sutherland et al.

clinicians to rate children’s apparent anxiety and the types of distractors (e.g., test equipment and non-test items, such as the camera or computer). Other scale items in the published checklist include those describ- ing activity and interaction levels, attention to task and distractibility. The observations in each condition (e.g., ‘never’, ‘sometimes’, ‘always’) were assigned a score from 0 to 2 and a total behaviour score was calculated.

Parent satisfaction

A simple, three-question survey was used to ask parents to rate their comfort with the telehealth assessment, and their perception of their child’s comfort level (i.e., ‘How comfortable were you with the telehealth assess- ment?’; ‘How comfortable do you think your child was with the telehealth assessment?’). A basic three-point Likert scale (‘Not comfortable’, ‘somewhat comfortable’, ‘very comfortable’) was used. An open question was in- cluded to elicit comments on the assessment from the parents.

Data analysis

Bland–Altman Limits of Agreement (Bland and Alt- man 1999) analyses were used to measure agreement of scores across the two conditions. This type of anal- ysis allows a new measurement or assessment (in this case, assessment delivered via telehealth) to be com- pared with a standard or well-established method (face- to-face assessments of language). Simultaneous scoring and analysis with this method eliminates issues regard- ing test–retest effects and allows for the same tool to be used in the two methods. In Bland–Altman analyses, a score of zero indicates perfect agreement. The greater the number for each plot point, the greater the magni- tude of the disagreement between the assessors in the two conditions. Calculations of 95% limits of agree- ment for each comparison (average difference ± 1.96 SD (standard deviations) of the difference) were made for each subtest scaled score, as well as the Core Lan- guage Score. Spearman’s correlation was used to provide measures of correlation (interrater reliability) for the scaled scores of each subtest and the Core Language Score. Behaviour, as measured by the total score on the behaviour observation checklists comparing behaviour during the telehealth subtests with behaviour during the traditional face-to-face subtests, was analyzed using the Wilcoxon Signed Ranks Test to test for any consistent differences between the pairs of behaviour observations (i.e., across the telehealth and face-to-face subtests). Par- ent data was collated and themes were developed from the open comment responses. These analyses were com- pleted using Statistical Package for the Social Sciences (SPSS).

Table 2. Correlation coefficients for the Clinical Evaluation of Language Fundamentals—4th edition (CELF-4) subtest

scaled scores

Test n Correlation p-value

Core Language Score 13 .993 <0.01 Concepts and Following

Directions 13 .967 <0.01

Recalling Sentences 13 .989 <0.01 Formulated Sentences 13 .943 <0.01 Word Classes—Total 13 .965 <0.01

Results

Twelve of the 13 assessments were completed in full. One face-to-face assessment was curtailed when the child be- came distracted and distressed about the correct answers and wanted to see the answers in the test materials dur- ing the assessment. Switching to the online assessment ameliorated this problem, although the child remained highly active throughout the assessment and had diffi- culties participating in the four telehealth subtests. No assessments were stopped or cancelled due to technical difficulties. One parent commented on less-than-ideal sound quality in the first assessment, but this did not impact on the completion of the assessment at the time. External speakers were used for subsequent sessions.

Reliability

Evidence of strong correlations between the conditions was found, with correlations between the standard and scaled scores for the subtests ranging from 0.943 to 0.993 (table 2). The Bland–Altman plots (figure 1) showed no trend in differences between the assessors and the level of agreement between assessors did not dif- fer across ability levels. There was greater agreement in subtests that required less clinical judgement and inter- pretation due to more prescriptive scoring procedures.

In addition, there was complete agreement on the severity levels of language disorder between the clinicians for 10 of the 13 children. For three children, the Core Language Scores were on the cusp of two clinical severity levels and small difference in the scores meant that a different severity level was ascribed for these children in the two conditions.

Behaviour

A Wilcoxon Signed Ranks Test was used to examine possible differences in behaviour between the two con- ditions. Overall, the difference between the pairs of scores was not significant (Z = –1.078; p = 0.281), despite the wide variation at the individual level. Four children showed higher behaviour scores (more obser- vations of inattention, distraction and/or anxiety) in the

Telehealth and autism 287

Figure 1. Bland–Altman plots illustrating interrater agreement across subtests.

288 Rebecca Sutherland et al.

Table 3. Parent comments regarding satisfaction

Parent number Comment

1 Very easy assessment but very comprehensive. My son really enjoyed the whole assessment. He loves computers and telephoning friends etc. so this was a great fit!

2 Had to think when doing anything and appeared unsure, appeared nervous as well

5 I think it is great and will help a lot of people 6 Anything that gets more information about

children’s well-being is a must for any parents or carers

7 I think the audio was a bit poor and (child) may have been having trouble understanding, otherwise it was good

10 I think (child) particularly enjoyed participating in the assessment via the webcam

12 If no technology issues, seems as good as an in-person assessment

13 This interesting to question and answers on computer [sic]

face-to-face condition and seven had higher scores in the telehealth condition. Two children’s scores were the same in both conditions. A moderately strong correlation was also noted between behaviour in the face-to-face con- dition and the SCQ scores (r = .660, p = 0.019) and moderate correlation between the telehealth behaviour and the SCQ scores which approached significance (r = 0.540, p = 0.07).

Parent satisfaction

Ten of the 13 parents completed and returned the sat- isfaction surveys. All parents (100%) felt ‘definitely’ comfortable with the telehealth assessments and all agreed that their child felt ‘definitely’ or ‘somewhat’ comfortable with the assessments. Eight parents of the thirteen students provided answers to an invitation to comment on the assessments; all responses are recorded in table 3.

Discussion

This study aimed to examine the reliability of language assessments delivered to children on the autism spec- trum via telehealth using consumer-grade equipment. In addition, it aimed to evaluate the feasibility of pro- viding these assessments via the reactions of children on the autism spectrum to the telehealth assessment by measuring behavioural observations, including level of activity, distractibility and signs of anxiety across the face-to-face and telehealth conditions.

The agreement between the assessment scores ob- tained by the telehealth and face-to-face SLPs was high, suggesting that the scores obtained in each setting were similar. This result is in line with the findings of both Sutherland et al. (2017), which involved children with reading difficulties using a similar assessment model, and Waite (2010), who assessed younger children using a customized system in a university setting. The present study adds to previous findings by extending the as- sessments to a population that has had limited research attention (i.e., school-aged children with autism). While telehealth has been used to provide training to parents and teachers of children with autism, there has been very limited research published regarding the involvement of children themselves with telehealth services. This study addresses the concerns raised by Sutherland et al. (2018) regarding the lack of direct telehealth service provision to children with autism and is among the first to undertake formal assessments with children with autism in which the child and clinician interact directly via telehealth.

Of particular interest in the current study were the response of the participants with autism to the telehealth assessments. In the single study regarding telehealth language pathology services with a child with autism (Boisvert et al. 2012), anecdotal information about the child’s performance suggested that the child was less dis- tracted and more responsive in the telehealth condition. The finding of reduced distractibility in the telehealth condition applied to four of the participants in the cur- rent study. For seven of the participants, the measures of behaviour were better in the face-to-face condition, with better attention and less distractibility noted. Two students showed no difference in behaviour across the conditions. Further research with a larger and more di- verse group of children is needed to better understand the impact of autism on telehealth participation, but our preliminary study suggests that children’s responses to telehealth are likely to be highly individual. While it is often suggested that children on the autism spectrum are readily and frequently engaged in using computers, research has not found that this is true of children with autism to a greater degree than neurotypical children (Montes 2016). It is unclear what role an intrinsic in- terest in computers and technology may have played in the current study. Parents were not asked about their child’s pre-assessment use of, or comfort with, comput- ers and telecommunication but an interest in computers was mentioned by one parent; it is possible that this may have played a role in behaviour for some children.

The analyses revealed a moderately strong correla- tion between behaviour in both assessment conditions and autism severity as measured by the SCQ, suggest- ing that the formal assessment situation may be more difficult for students with more pronounced character- istics of autism. Certainly the characteristics of autism

Telehealth and autism 289

can make these types of assessments more challenging and careful consideration of the timing, environment and administration of assessment is warranted (Paynter 2015). Further examination regarding the preparation of children for both face-to-face and telehealth services is warranted and may be an important factor in consid- erations around telehealth provision in clinical settings. The present study attempted to provide support by giv- ing parents a Social Story to read with their children in order to prepare them for the assessment session. How- ever, it is not known whether parents read it with their children and thus it is not known whether the children who coped better with the telehealth condition were better prepared for the assessment through the read- ing of the story or other preparations by the parent. It is likely that further exploration of good practice in preparing children to participate in telehealth services may be valuable when considering working with this population.

Limitations

The present research includes a small sample of chil- dren with autism, with lower numbers than comparable studies of telehealth language assessments (Sutherland et al. 2017, Waite et al. 2010). While the results with respect to strong reliability, even in this small sample, builds confidence in findings, the presence of moderate- to-strong but nevertheless non-significant correlations between SCQ scores and behaviour may reflect under- powered analyses. Replication of this study with a larger sample size is warranted to further explore these relation- ships. Behaviour ratings were completed by the assessing clinician; ideally, blinded assessors would have enhanced the validity of the results. In addition, the children in- volved represent only a small portion of children on the autism spectrum; children with intellectual disabil- ities or more significant communication disorders were not included. As this was one of the first studies in- vestigating the use of telehealth to interact directly with children with autism, it was important to control for the possible impact of other variables such as cognitive and language skills. Given the positive findings of this small- scale study, it will be important to expand this work to a broader range of cognitive abilities and language skills. As with previous studies, only the core language subtests were assessed under the telehealth, rather than a more comprehensive assessment of all language and communication skills. Future studies could examine the use of a broader range of language assessments, includ- ing assessment of pragmatics and higher level language skills. In addition, while the results of this study indi- cate that telehealth and face-to-face scores on telehealth language assessments are similar, it does not provide in- formation about whether children with autism would

perform better in terms of language scores one condi- tion over the other. Ideally, future telehealth assessment studies could involve the use of assessments with alter- nate forms to allow for more direct comparison between language abilities demonstrated in telehealth and face- to-face conditions.

Conclusions

The findings of the current study provide evidence to support the use of telehealth in the language assessment of some school-aged children with autism. High levels of agreement and correlation were found between the telehealth and face-to-face scores, indicating that the telehealth system was reliable for the formal assessment of language. Individual differences were noted in the be- haviour of the children with autism across the telehealth and face-to-face conditions but there was no clear dif- ference as a group between conditions. Telehealth may be a useful primary or adjunct service model for chil- dren on the autism spectrum provided that clinicians are aware of, and can respond flexibly to, the individual differences shown by this group of children.

Acknowledgements

The authors acknowledge Silvia Pfeiffer of Coviu, and A/Professor Natalie Silove, Suzi Drevensek and Marcia Williamsz of The Child Development Unit, The Children’s Hospital at Westmead.The au- thors wish to thank the children and families involved with this study. Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.

Note

1. The term ‘autism’ will be used in this paper and includes autism spectrum disorder (ASD) (DSM-5), autism spectrum disorders including Asperger’s disorder (DSM-IV) and autism spectrum condition (ASC). The term ‘child/ren with autism’ is also used in this paper. The authors acknowledge that debate exists regarding the use of person first/identity first language.

References

AMERICAN PSYCHIATRIC ASSOCIATION (APA), 2013, Diagnostic and Statistical Manual of Mental Disorders: DSM-5 (Washington, DC: APA).

AUSTRALIAN INSTITUTE OF HEALTH AND WELFARE (AIHW), 2009, Health and Community Services Labour Force (National Health Labour Force Series NO. 42. Cat. No. HWL 43). Canberra, ACT: AIHW.

BLAND, J. M. and ALTMAN, D. G., 1999, Measuring agreement in method comparison studies. Statistical Methods in Medical Research, 8, 135–160.

BOISVERT, M., HALL, N., ANDRIANOPOULOS, M. and CHACLAS, J., 2012, The multi-faceted implementation of telepractice to service individuals with autism. International Journal of Tel- erehabilitation, 4, 11–23.

290 Rebecca Sutherland et al.

BRIDGMAN, K., ONSLOW, M., O’BRIAN, S., JONES, M. and BLOCK, S., 2016, Lidcombe Program Webcam Treatment for Early Stuttering: a randomized controlled trial. Journal of Speech, Language, and Hearing Research, 59, 932–939.

COLLINS, A., BURNS, C. L., WARD, E. C., COMANS, T., BLAKE, C., KENNY, L., GREENUP, P. and BEST, D., 2017, Home- based telehealth service for swallowing and nutrition man- agement following head and neck cancer treatment. Journal of Telemedicine and Telecare, 23, 866–872.

COMMUNITY AFFAIRS REFERENCES COMMITTEE, 2014, Prevalence of Different Types of Speech, Language and Communication Disorders and Speech Pathology Services in Australia (Canberra, ACT: Community Affairs References Committee).

GERNSBACHER, M. A., MORSON, E. M. and GRACE, E. J., 2015, Language development in autism. In G. Hickok and S. Small (eds), Neurobiology of Language (San Diego: Elsevier Academic Press), pp. 879–886

GRAY, C. A. and GARAND, J. D., 1993, Social stories: improving responses of students with autism with accurate social infor- mation. Focus on Autistic Behavior, 8, 1–10.

GROGAN-JOHNSON, S., SCHMIDT, A. M., SCHENKER, J., ALVARES, R., ROWAN, L. E. and TAYLOR, J., 2013, A comparison of speech sound intervention delivered by telepractice and side-by-side service delivery models. Communication Disorders Quarterly, 34, 210–220.

HEPBURN, S. L., BLAKELEY-SMITH, A., WOLFF, B. and REAVEN, J. A., 2015, Telehealth delivery of cognitive–behavioral inter- vention to youth with autism spectrum disorder and anxiety: a pilot study. Autism, 20, 207–218.

INGERSOLL, B. and BERGER, N. I., 2015, Parent engagement with a telehealth-based parent-mediated intervention program for children with autism spectrum disorders: predictors of pro- gram use and parent outcomes. Journal of Medical Internet Research, 17, e227.

INGERSOLL, B., WAINER, A. L., BERGER, N. I., PICKARD, K. E. and BONTER, N., 2016, Comparison of a self-directed and therapist-assisted telehealth parent-mediated intervention for children with ASD: a pilot RCT. Journal of Autism and De- velopmental Disorders, 46, 2275–2284.

LINDGREN, S., WACKER, D., SUESS, A., SCHIELTZ, K., PELZEL, K., KOPELMAN, T., LEE, J., ROMANI, P. and WALDRON, D., 2016, Telehealth and autism: treating challenging behavior at lower cost. Pediatrics, 137, S167–S175.

LOUCAS, T., CHARMAN, T., PICKLES, A., SIMONOFF, E., CHANDLER, S., MELDRUM, D. and BAIRD, G., 2008, Autistic symptoma- tology and language ability in autism spectrum disorder and specific language impairment. Journal of Child Psychology and Psychiatry, 49, 1184–1192.

MAZUREK, M. O., SHATTUCK, P. T., WAGNER, M. and COOPER, B. P., 2012, Prevalence and correlates of screen-based media use among youths with autism spectrum disorders. Journal of Autism and Developmental Disorders, 42, 1757–1767.

MEADAN, H., SNODGRASS, M. R., MEYER, L. E., FISHER, K. W., CHUNG, M. Y. and HALLE, J. W., 2016, Internet-based parent- implemented intervention for young children with autism: a pilot study. Journal of Early Intervention, 38, 3–23.

MONTES, G., 2016, Children with autism spectrum disorder and screen time: results from a large, nationally representative US study. Academic Paediatrics, 16, 122–128.

MUST, A., PHILLIPS, S. M., CURTIN, C., ANDERSON, S. E., MASLIN, M., LIVIDINI, K. and BANDINI, L. G., 2014, Comparison of sedentary behaviors between children with autism spectrum disorders and typically developing children. Autism, 18, 376– 384.

NORRIS, M. and LECAVALIER, L., 2010, Screening accuracy of level 2 autism spectrum disorder rating scales: a review of selected instruments. Autism, 14, 263–284.

O’BRIAN, S., SMITH, K. and ONSLOW, M., 2014, Webcam delivery of the Lidcombe Program for Early Stuttering: a Phase 1 clinical trial. Journal of Speech, Language, and Hearing Research, 57, 825–830.

ORSMOND, G. I. and KUO, H.-Y., 2011, The daily lives of adolescents with an autism spectrum disorder: discretionary time use and activity partners. Autism, 15, 579–599.

PAYNTER, J. M., 2015, Assessment of school-aged children with autism spectrum disorder. Journal of Psychologists and Coun- sellors in Schools, 25, 104–115.

PICKARD, K. E., WAINER, A. L., BAILEY, K. M. and INGERSOLL, B. R., 2016, A mixed-method evaluation of the feasibility and acceptability of a telehealth-based parent-mediated interven- tion for children with autism spectrum disorder. Autism, 20, 845–855.

REESE, R., BRAUN, M., HOFFMEIER, S., STICKLE, L., RINNER, L., SMITH, C., ELLERBECK, K., JAMISON, R., WENDLAND, M., JARRETT, L. and HADORN, M., 2015, Preliminary evidence for the integrated systems using telemedicine. Telemedicine and e-Health, 21, 581–587.

REESE, R. M. J., R., WENDLAND, M., FLEMING, K., BRAUN, M. J., OETH SCHUTTLER, J. and TUREK, J., 2013, Evaluating inter- active videoconferencing for assessing symptoms of autism. Telemedicine and e-Health, 19, 671–677.

RUTTER, M., BAILEY, A. and LORD, C., 2008, Social Communication Questionnaire (SCQ) (Los Angeles, CA: Western Psychological Services).

SCHUTTE, J. L., MCCUE, M. P., PARMANTO, B., MCGONIGLE, J., HANDEN, B., LEWIS, A., PULANTARA, I. W. and SAPTONO, A., 2015, Usability and reliability of a remotely administered adult autism assessment, the Autism Diagnostic Observation Schedule (ADOS) Module 4. Telemedicine and e-Health, 21, 176–184.

SEMEL, E., WIIG, E. and SECORD, W., 2003, Clinical Evaluation of Language Fundamentals, 4th edition (San Antonio, TX: Harcourt Assessment, Psychological Corporation).

SHARMA, S., WARD, E. C., BURNS, C., THEODOROS, D. and RUSSELL, T., 2013, Assessing dysphagia via telerehabilitation: patient perceptions and satisfaction. International Journal of Speech– Language Pathology, 15, 176–183.

SIMACEK, J., DIMIAN, A. F. and MCCOMAS, J. J., 2017, Communi- cation intervention for young children with severe neurode- velopmental disabilities via telehealth. Journal of Autism and Developmental Disorders, 47, 744–767.

SPEECH PATHOLOGY AUSTRALIA, 2016, Clinical Guideline: Evidence Based Speech Pathology Practice for Individuals with Autism Spectrum Disorder (Melbourne, VIC: Speech Pathology Asso- ciation of Australia).

SUESS, A. N., ROMANI, P. W., WACKER, D. P., DYSON, S. M., KUHLE, J. L., LEE, J. F., LINDGREN, S. D., KOPELMAN, T. G., PELZEL, K. E. and WALDRON, D. B., 2014, Evaluating the treatment fidelity of parents who conduct in-home functional com- munication training with coaching via telehealth. Journal of Behavioral Education, 23, 34–59.

SUTHERLAND, R., TREMBATH, D., HODGE, A., DREVENSEK, S., LEE, S., SILOVE, N. and ROBERTS, J., 2017, Telehealth language assessments using consumer grade equipment in rural and urban settings: feasible, reliable and well tolerated. Journal of Telemedicine and Telecare, 23, 106–115.

SUTHERLAND, R., TREMBATH, D. and ROBERTS, J., 2018, Telehealth and autism: a systematic search and review of the literature.

Telehealth and autism 291

International Journal of Speech–Language Pathology, 20, 324– 336.

TAYLOR, L., BROWN, P., EAPEN, V., MIDFORD, S., PAYNTER, J., QUARMBY, L., SMITH, T., MAYBERY, M., WILLIAMS, K. and WHITEHOUSE, A., 2016, Autism spectrum disorder diagnosis in Australia: are we meeting best practice standards? (Aus- tralia: Autism Co-operative Research Centre, Brisbane).

THOMAS, D. C., MCCABE, P., BALLARD, K. J. and LINCOLN, M., 2016, Telehealth delivery of Rapid Syllable Transitions (ReST) treatment for childhood apraxia of speech. Interna- tional Journal of Language and Communication Disorders, 51, 654–671.

WAINER, A. L. and INGERSOLL, B. R., 2015, Increasing access to an ASD imitation intervention via a telehealth parent training program. Journal of Autism and Developmental Disorders, 45, 3877–3890.

WAITE, M. C., THEODOROS, D. G., RUSSELL, T. G. and CAHILL, L. M., 2010, Internet-based telehealth assessment of language

using the CELF-4. Language, Speech, and Hearing Services in Schools, 41, 445–458.

WAITE, M. C., THEODOROS, D. G., RUSSELL, T. G. and CAHILL, L. M., 2012, Assessing children’s speech intelligibility and oral structures, and functions via an internet-based telehealth system. Journal of Telemedicine and Telecare, 18, 198–203.

WARD, E. C., BURNS, C. L., THEODOROS, D. G. and RUSSELL, T. G., 2013, Evaluation of a clinical service model for dyspha- gia assessment via telerehabilitation. International Journal of Telemedicine and Applications, 2013, 918526–918526.

WIIG, E. H., SECORD, W. A. and SEMEL, E. M., 2006, Clinical Evaluation of Language Fundamentals: Preschool, 2nd Edition (Marrickville NSW: Harcourt Assessment).

WITTKE, K., MASTERGEORGE, A. M., OZONOFF, S., ROGERS, S. J. and NAIGLES, L. R., 2017, Grammatical language impairment in autism spectrum disorder: exploring lan- guage phenotypes beyond standardized testing. Frontiers in Psychology, 8, 532.

Copyright of International Journal of Language & Communication Disorders is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.