Systematic Review

RESEARCH/Original Article

Telehealth language assessments using consumer grade equipment in rural and urban settings: Feasible, reliable and well tolerated

Rebecca Sutherland, BSPath (Hons), MHthSc (Developmental Disability)

1,2 , David Trembath, BAppSc (Speech Pathology),

MaAppSc (Communication Sciences and Disorders), PhD 3 ,

Antoinette Hodge, BA (Psychology Hons), DPsy, MClin Neuropsych

2 , Suzi Drevensek, BSc (Psychology), BAppSc (Speech

Pathology Hons) 2 , Sabrena Lee, BSPth (Hons)

2 , Natalie Silove,

MBBS, FRACP 2

and Jacqueline Roberts, BA (Hons), DipTeach, BAppSc (Speech Pathology), PhD

1

Abstract

Introduction: Telehealth can be an effective way to provide speech pathology intervention to children with speech and

language impairments. However, the provision of reliable and feasible standardised language assessments via telehealth to

establish children’s needs for intervention and to monitor progress has not yet been well established. Further, there is limited

information about children’s reactions to telehealth. This study aimed to examine the reliability and feasibility of conducting

standardised language assessment with school-aged children with known or suspected language impairment via a telehealth

application using consumer grade computer equipment within a public school setting.

Method: Twenty-three children (aged 8–12 years) participated. Each child was assessed using a standardised language assess-

ment comprising six subtests. Two subtests were administered by a speech pathologist face-to-face (local clinician) and four

subtests were administered via telehealth. All subtests were completed within a single visit to the clinic service, with a break

between the face to face and telehealth sessions. The face-to-face clinician completed behaviour observation checklists in the

telehealth and face to face conditions and provided feedback on the audio and video quality of the application from the child’s

point of view. Parent feedback about their child’s experience was elicited via survey.

Results: There was strong inter-rater reliability in the telehealth and face-to-face conditions (correlation coefficients ranged

from r¼0.96–1.0 across the subtests) and good agreement on all measures. Similar levels of attention, distractibility and anxiety

were observed in the two conditions. Clinicians rated only one session of 23 as having poor audio quality and no sessions were

rated as having poor visual quality. Parent and child reactions to the use of telehealth were largely positive and supportive of

using telehealth to assess rural children.

Discussion: The findings support the use of telehealth in the language assessment of school-aged children using a web

application and commercially available computer equipment. This reliable and innovative service delivery model has the poten-

tial to be used by speech pathologists to provide assessments to children in remote communities.

Keywords

Telehealth, language assessment, rural, school-aged children

Date received: 6 October 2015; Date accepted: 23 November 2015

Introduction

Australia is a vast country with a relatively small popula- tion of approximately 24 million people

1 concentrated in

urban areas, with limited health, education and other ser- vices in rural areas.

2 A recent senate inquiry into speech

pathology services in Australia 3 found evidence of a short-

age of speech pathologists, particularly outside urban areas,

1 Autism Centre of Excellence, Griffith University, Australia

2 NSW Centre for Effective Reading & Child Development Unit, The

Children’s Hospital at Westmead, Australia 3 Menzies Health Institute Queensland, Australia

Corresponding author:

Rebecca Sutherland, Child Development Unit, The Children’s Hospital at

Westmead, Locked Bag 4001, Westmead NSW 2145, Australia.

Email: [email protected]

Journal of Telemedicine and Telecare

0(0) 1–10

! The Author(s) 2016

Reprints and permissions:

sagepub.co.uk/journalsPermissions.nav

DOI: 10.1177/1357633X15623921

jtt.sagepub.com

J Telemed Telecare OnlineFirst, published on January 13, 2016 as doi:10.1177/1357633X15623921

at Xi’an Jiaotong University on June 4, 2016jtt.sagepub.comDownloaded from

and that the ratio of speech pathologists to population falls as population density declines, from 25.9 per 100,000 of population in major cities to 5.9 per 100,000 of population in very remote areas. As a result, people in rural and remote areas experience significant barriers to accessing ser- vices.

4 The lack of services creates a significant risk for poor

social and educational outcomes for those with language impairments living in rural and remote areas of Australia.

Telehealth in speech-language pathology

Telehealth may overcome some of the difficulties associated with providing accessible speech pathology services and has been used successfully to treat a range of conditions includ- ing adult speech and language disorders,

5 dysphagia,

6,7

childhood fluency 8 and speech sound disorders.

9

Yet despite evidence for its utility in delivering inter- ventions, only a small number of studies have examined the role for telehealth in standardised language assess- ment. Language assessments are crucial in establishing diagnoses, guiding intervention and documenting progress in children with language difficulties.

The use of telehealth in the delivery of standardised language assessments has been examined in two stu- dies,

10,11 using method comparison designs, while the

use of a language screening tool was evaluated in a further study.

12 In all three studies, the authors reported good

reliability between face-to-face and online scoring of par- ticipant responses. Waite et al.

11 trialled the use of the

Clinical Evaluation of Language Fundamentals - 4 th

edi- tion (CELF-4) using a custom-built Internet-based tele- health application with store and forward capabilities and touch screen at the participant’s end. The picture stimuli were digitised. An audio recording of the stimulus for one subtest assessment was used, along with live pres- entation of the remaining three subtest materials. As with Eriks-Brophy et al.

10 and Ciccia et al.

12 , a method com-

parison design 13

was utilised with simultaneous scoring by a face-to-face and online assessor, in this case between two assessment rooms in a university clinic. Students and clin- icians were randomised to face-to-face or online condi- tions. Very good levels of inter- and intra-rater agreement were found on all measures with some vari- ation between subtests (k range from 0.93 to >0.99). A limitation of this study, as noted by Taylor et al.,

13 is that

it was not conducted in real world settings but rather under ideal research conditions. In addition, none of the studies reported client or clinician satisfaction and only Ciccia et al. reported on parent satisfaction. Finally, the studies did not include comparisons of child behaviour across face-to-face and online conditions.

The data to date regarding telehealth assessment is pro- mising, however the current lack of information about the reliability and feasibility of conducting telehealth assess- ments in routine clinical settings with commercially avail- able equipment, as well as parent, child and clinician responses to the approach, represents a significant gap in the literature. Research is needed to examine the

effectiveness of telehealth language assessment in the con- text of existing service delivery models, as well as to begin to address identified barriers to speech pathologists using telehealth. These include beliefs that parents hold negative attitudes towards its use,

14 and that telehealth can only be

used in specialised research or university settings, rather than real world clinical services.

15 Other concerns held by

speech pathologists who do not use telehealth include fear of technology and concerns regarding their ability to develop rapport with remote clients.

15

The aims of this study were to determine whether, within an existing service, a web-based telehealth applica- tion using consumer grade, commercially available com- puter equipment could be used to provide a formal language assessment that is feasible, reliable and well-tol- erated by participants and their families. In this study, children were in the same location as a second speech path- ologist (face-to-face speech pathologist) whose role was to facilitate the assessment, replicating a real-world setting in which children could be supported by teachers or other school support staff during assessment. It was hypoth- esised that under these conditions, standardised language assessment delivered via telehealth would be feasible, reli- able and well tolerated by participants and their families.

Method

Ethical approval was received from the Sydney Children’s Hospital Network (SCHN) Human Research Ethics Committee (HREC reference: LNR/12/SCHN/408). Approval was also obtained from NSW Department of Education. Written consent was obtained from parents, and children were provided with an illustrated social story to support their understanding of the project.

Design

A method comparison design was used 16

in order to meas- ure the agreement between the scores for each language assessment subtest recorded by the telehealth speech path- ologist compared with those made simultaneously by the face-to-face speech pathologist. This method allows researchers to quantify the variation between two methods of measurement for the same subject. The independent variable was the condition under which the assessment was conducted (telehealth vs face-to-face). The dependent variables (DVs) were: (a) number of scheduled sessions that were completed successfully (DV1) and face-to-face speech pathologists ratings of audio and visual quality (DV2) which provided measures of feasibility; (b) the lan- guage assessment scores given by the telehealth and face-to-face speech pathologists which were compared to provide a measure of reliability (DV3); and (c) behaviour observation scores made by the face-to-face speech path- ologist in the face-to-face and telehealth conditions, pro- viding a measure regarding the behaviour of the children and how well the telehealth condition was tolerated (DV4), along with parent survey information (DV5).

2 Journal of Telemedicine and Telecare 0(0)

at Xi’an Jiaotong University on June 4, 2016jtt.sagepub.comDownloaded from

Participants

Participants were 23 children (18 male, five female) aged 8– 12 years (mean¼9 years 11 months) with a history of read- ing difficulties and known/suspected language impairment, attending mainstream schools. They were referred to a specialist reading centre, a service for children with reading difficulties who live in rural and remote NSW, Australia. Children were referred by their schools based on their poor response to reading interventions and attended one of two rural ‘hubs’ or a suburban hub (name withheld) for multi- disciplinary assessment, including a thorough language assessment. The participating students were a sample of convenience, based on those students attending the service. There were no exclusionary criteria; however, children did not participate if they had had a language assessment using the CELF-4 within the previous six months.

Three experienced (M¼13.7 years of work experience, range 10–19 years) local speech pathologists consented to participate in the project (i.e. co-scoring and delivering assessments) as part of their regular work. They were all experienced in administering the paper form of the CELF- 4 and regularly used technology for communication in the workplace (e.g. videoconferencing with colleagues). Two of the three speech pathologists reported having limited experi- ence in delivering speech and/or language intervention via computer (e.g. using Adobe Connect). None had used the current telehealth interface prior to the project. The tele- health speech pathologist had four years clinical experience and was familiar with the language assessment tool. She reported familiarity with a range of technology applications (e.g. Skype, computer use) but had less experience using technology for communication in the workplace (e.g. video- conferencing) and no experience of telehealth prior to the project.

Materials

Assessment tool. The CELF-4 17

is a standardised, individu- ally administered clinical tool for the identification, diagno- sis and follow-up evaluation of language skill deficits in school-age children, adolescents and young adults. Permission was granted by Pearson Australia to digitise images of the CELF-4 for use in online administration. The test images were scanned and converted to JPEG files which were then uploaded to the application. The published psychometric properties of the CELF-4 include stability coefficients ranging from 0.77–0.94, internal consistency for the Core Language Score (a composite score indicating the presence or absence of a language impairment along with the severity of the impairment) ranging from 0.97 across age ranges.

17 The Core Language Score is based on four core

subtests. For the 5–8 year-old age group these are Concepts and Following Directions, Word Structure, Recalling Sentences and Formulated Sentences. Students aged 9–12 years completed also three of these subtests (Concepts and Following Directions, Recalling Sentences and Formulated Sentences) as well as Word Classes which

yields an expressive and a receptive score. Inter-scorer reli- abilities are described as 0.99 or 1.0 across all CELF-4 Australian subtests. The core subtests for 9–12 year olds include two subtests that require the administrator to score a response as correct/incorrect (Concepts and Following Directions; Word Classes – Receptive). Two subtests require the administering clinician to make a subjective judgement with reference to scoring guidelines to score a response as correct or incorrect (Word Classes – Expressive) or to score 0, 1 or 2 based on the number of grammatical and/or syn- tactical errors made by the child (Formulated Sentences). The final subtest (Recalling Sentences) can be scored as 0, 1, 2 or 3 based on the number of errors made during imme- diate repetition of a spoken sentence. While scoring for this subtest is not subjective, the reliability of scoring is depend- ent on precise notation of the child’s response. All partici- pants completed the four core subtests via the telehealth application. Two additional subtests were administered by the face-to-face speech pathologist, along with any other clinically relevant assessments required.

Equipment. The assessment was delivered via a telehealth application developed by National Information Communications Technology (ICT) Australia (NICTA). This web-based interface included synchronised image viewing and remotely visible click-markers for pointing at images as well as video conference functionality. The computers used were standard issue [name withheld for review] devices with consumer grade webcams (Logitech C920) and speakers (Logitech PC Z130). A commercially available touch screen from Dell (model S2240T) was sup- plied to the local sites. All four sites (telehealth setting and the three hubs) were located within Department of Education, public schools and used the Internet connec- tion available to these sites. The application was designed to allow a speech pathologist using telehealth to complete the assessment with the child supported by a school staff member (teacher or teacher aide) in their local setting. For the purpose of this research, a speech pathologist acted as a support person to facilitate assessments (rather than a teacher or teacher aide) in order to co-score the assessment tasks delivered by the telehealth speech. This allowed for the comparison of scores for the study without adding extra team members to the assessment environment.

Procedure

Training. The telehealth speech pathologist was trained in the use of the interface by the developer and a senior speech pathologist (RS) in a single session. The face-to- face speech pathologists were shown the interface in a group setting and subsequently given information and an opportunity to practice using the application in a ses- sion with the telehealth speech pathologist. This informa- tion session provided details on how to log into the application, camera positioning, room set up and general troubleshooting. Training for both the telehealth and face- to-face speech pathologists took less than 30 min.

Sutherland et al. 3

at Xi’an Jiaotong University on June 4, 2016jtt.sagepub.comDownloaded from

A group session was held with all speech pathologists, as well as other senior speech pathologists employed by the service, to practice scoring the subjective subtests in accordance with the CELF-4 manual instructions. This provided the telehealth and face-to-face speech patholo- gists with an opportunity to review standard scoring guidelines and to participate in a practical component where the clinicians practiced scoring a series of sample sentences. Reliability was not formally established, how- ever consensus was reached for each example in accord- ance with the CELF-4 scoring guidelines. All assessments were carried out as per the CELF-4 manual.

Assessment procedure. The assessments were conducted from the metropolitan telehealth site in Westmead, NSW, to the three hub sites in rural NSW (Hub 1 and Hub 3) and suburban Sydney, NSW (Hub 2). The telehealth speech pathologist administered the four core subtests of the CELF-4 via the telehealth application and scored the chil- dren’s responses (approximately 45–60 min). The face-to- face speech pathologist was with the child during the tele- health assessment and facilitated the assessment by turning on the computer, logging into the application and checking vision and sound. She then co-scored the assessment as an observer. The face-to-face speech pathologist also com- pleted two non-core CELF-4 subtests and other language assessments (as clinically indicated) with the student face- to-face (approximately 30–60 min depending on clinical need). These face-to-face assessments were completed either before or after the telehealth session depending on the scheduling demands of the hub, resulting in non- randomised counter-balanced delivery order (17/23 com- pleted face-to-face assessment tasks first). All subtests were completed within a single visit to the clinic service, with a break between the face-to-face and telehealth ses- sions. The face-to-face speech pathologist completed behaviour observations of the children in both face-to- face and telehealth conditions in order to provide feedback about the child’s behaviour that may not have been able to be seen by the telehealth clinician. The face-to-face clinician also provided feedback on the audio and video quality of the application from the child’s point of view.

Measures

Measures of feasibility (DV1, DV2). The telehealth speech pathologist kept records regarding the number of sched- uled sessions completed and reported any issues resulting in non-completion of the assessment due to child, clin- ician, or technological factors (DV1). The telehealth and face-to-face speech pathologists both recorded observa- tions regarding the technology during each session, noting the audio and video quality on a three-point scale (poor, OK, good) (DV2).

Measures of reliability (DV3). Measures of reliability involved both the telehealth and face-to-face speech pathologists simultaneously scoring the core subtests of the CELF-4.

These measures were collected and scored as per the standard administration instructions for the CELF-4. Core Language Scores were calculated and severity rank- ings of language impairment were assigned as per the administration guidelines. Method comparison analysis and Pearson’s correlation were used to determine level of agreement and reliability.

Measures of behaviour (DV4). A behaviour observation rating scale was adapted from the Clinical Evaluation of Language Fundamentals – Preschool, 2

nd edition,

Australian and New Zealand (CELF-P2) 18

behaviour checklist. No such checklist accompanies the CELF-4 but the CELF-P2 checklist contained relevant items for this population. Observations about anxiety and the types of distractors (test equipment and non-test items) were added to those describing activity and interaction levels, attention to task and distractibility. The rating scale is presented in Table 1.

Measures of parent satisfaction (DV5). Parents completed a simple survey following the telehealth assessment that asked ‘Do you think your child felt comfortable during the (telehealth) assessment?’ and ‘Did you feel comfort- able during the (telehealth) assessment’ with four options: ‘yes definitely’, ‘yes, somewhat’, ‘no’ and ‘don’t remem- ber’. The survey was completed and returned anonym- ously to the research team.

Results

Feasibility

Overall 100% (23 of 23) assessment sessions were com- pleted; no sessions were discontinued due to technological or other difficulties. Ratings of audio and visual qual- ity (‘poor’ to ‘good’) were converted to numerical scores (0, 1, 2). Audio was rated as ‘good’ in 74% of sessions, and inconsistent or ‘OK’ in a further 22%. Only one session was rated as ‘poor’. The average audio rating for Hub 1 was 1.88 (mode¼2, range 1–2), for Hub 2 it was 1.43 (mode¼2, range 0–2) and for Hub 3 1.75 (mode¼2, range 1–2). Audio problems occurred more frequently in one hub; these were mostly related to low volume and words occasionally cutting out. The difficulties were not significant enough to require any adjustment to the assess- ment, other than feedback from the local clinician to the telehealth clinician that she needed to speak louder. Visual quality was rated as ‘good’ in 83% of sessions. No sessions were rated as ‘poor’. The average visual quality rating for Hub 1 was 1.88 (mode¼2, range 1–2), for Hub 2 was 1.71 (mode¼2, range 1–2) and for Hub 3, 1.88 (mode¼2, range 1–2).

Reliability

Bland-Altman analyses were used to assess the level of agreement for the Core Language Scores and the raw

4 Journal of Telemedicine and Telecare 0(0)

at Xi’an Jiaotong University on June 4, 2016jtt.sagepub.comDownloaded from

scores of the four core subtests by plotting the mean (x-axis) versus the difference (y-axis) between the two measures (telehealth and face-to-face) (Figure 1). In these analyses, zero equals perfect agreement, and the larger the number for each plot point the greater the dis- agreement between assessors. Calculations of 95% limits of agreement for each comparison (average differ- ence�1.96 standard deviations of the difference) were made for each subtest. Plots showed no clear trend in difference between the assessors, and the variance did not appear to differ with the mean. Furthermore, evidence of strong correlations was found between conditions for all measures ranging from 0.96–1.0 as shown in Table 2. There was complete agreement on severity level for 22 of

the 23 children (96%) with one student’s rating only dif- fering by one severity level.

Behaviour

Behaviour, as measured by the behaviour observation checklist, was analysed using paired t-tests. Initial analysis of the data indicated differences between the hubs, with one hub showing greater variability between the face-to- face and telehealth speech pathologists. Behaviour data was subsequently analysed in three groups, according to the location of the child. The two rural hubs showed no difference in observed behaviour between the conditions. In contrast, the suburban hub showed greater levels of behaviour difficulty in the telehealth condition for some students, however this difference did not reach significance at 0.05 (see Table 3).

Parent satisfaction

Thirteen parents of the 23 participants returned the survey anonymously (56%). All parents described their child and themselves as being ‘somewhat comfortable’ or ‘definitely comfortable’ with the telehealth assessment. Table 4 pre- sents parents’ comments regarding their experience. No parents indicated that they felt uncomfortable with the assessment. Many of the comments from parents indi- cated that their child had found the experience positive and a number of parents reflected positively on their own experience or opinion of the telehealth assessments. Two parents expressed concern: one parent felt their child lost concentration when he got a question wrong; the second parent stated their child ‘did like to use the com- puter and the videoconferencing. . .. but he would prefer to talk to a person face-to-face’.

Discussion

This study aimed to examine the reliability and feasibility of conducting standardised language assessment with school-aged children with known or suspected language impairment via a telehealth application using consumer grade computer equipment within a public school setting. The feedback from all the speech pathologists involved suggested that the system used was feasible, with the tele- health speech pathologist able to complete all assessments with good levels of audio and visual quality. Given that the system is designed for speech pathologists to deliver assessments online within schools, and for children to be supported in the assessment session by teachers or teach- ing assistants (rather than local speech pathologists), the feasibility of the system and the ability to assess children reliably was of particular interest.

In terms of reliability, a high level of agreement was reached between the telehealth and face-to-face speech pathologists on all subtest raw scores, scaled scores and core language scores, indicating that the responses received and recorded by the telehealth speech pathologist

Table 1. Behaviour rating scale.

The child was: Compliant

Compliant with support

Noncompliant

The child’s activity level was: Appropriate

Too active

Not active enough

The child was fidgety/restless: Never

Some of the time

Most of the time

The child required breaks: Never

At appropriate times

Too frequently (interfered with

testing)

The child maintained attention: The whole time

Most of the time

Some of the time

Never

The child was distracted by test-

related factors or equipment

(e.g. webcam, computer,

mouse, etc.)

Never

Occasionally

Often

The child was distracted by non-

test-related factors (e.g. envir-

onmental sounds, window,

pictures on walls etc.):

Never

Occasionally

Often

The child engaged in off-task

behaviours:

Never

Occasionally

Often

The child demonstrated fatigue,

boredom and/or frustration

during the test:

Never

Rarely

Sometimes

Often

Fatigue, boredom and/or frus-

tration became evident:

Never

After 15 min

After 10 min

After 5 min

The child demonstrated anxiety/

nervousness during the test:

The whole time

Most of the time

Some of the time

Never

The child requested help, sup-

port or clarification:

Never

Occasionally

Often

Sutherland et al. 5

at Xi’an Jiaotong University on June 4, 2016jtt.sagepub.comDownloaded from

F ig

u r e

1 .

B la

n d

A lt m

a n

p lo

ts a n d

sc a tt

e rp

lo ts

il lu

st ra

ti n g

in te

r- ra

te r

a g re

e m

e n t

a n d

re li a b il it y

a c ro

ss su

b te

st s.

6 Journal of Telemedicine and Telecare 0(0)

at Xi’an Jiaotong University on June 4, 2016jtt.sagepub.comDownloaded from

F ig

u r e

1 .

C o n ti n u e d .

Sutherland et al. 7

at Xi’an Jiaotong University on June 4, 2016jtt.sagepub.comDownloaded from

were very similar to those recorded by the face-to-face speech pathologist. These results are in line with those reported by Waite and colleagues

11 in their study of 5–9

year-olds whose language was assessed using the CELF-4 via telehealth. Their study found very good agreement between telehealth and face-to-face scorers when assessing children using custom built equipment in a university set- ting, with some variation between subtests (k range from 0.93 to >0.99). The present study adds to their findings by completing the assessments in real time and remote loca- tions with consumer grade, commercially available equip- ment that is readily available in the community, and is user friendly. The telehealth clinician was able to learn to use the system easily and the face-to-face speech path- ologists support for the children required minimal train- ing. It is anticipated that education staff in regional schools will be able to provide appropriate support for telehealth assessments in the future.

The behaviour of individual children under telehealth conditions and their reaction in comparison to typical face-to-face sessions has not been systematically examined in research published to date. This study found few differ- ences in the clinician-reported child interaction, attention or anxiety levels across the two conditions when analysed in separate hubs. Differences were observed, however, between the hubs, with one clinician reporting that the children showed more behaviour difficulties when engaged in the telehealth assessments, although the differences between conditions did not reach significance. This hub also experienced more audio and visual difficulties than

the other two hubs and it is possible that these difficulties impacted on some students’ attention and engagement. In addition, the effect of different interaction styles between the hub therapists cannot be ruled out. It will be import- ant for children involved in telehealth assessments to be supported to ensure that they are able to attend and inter- act appropriately.

The present study found good levels of parent satisfac- tion with the technology, in line with the findings of Ciccia et al.

12 who found high levels of parent satisfaction with

speech and language screening conducted via Skype, and

Table 3. Paired sample t-test comparing telehealth and face-to-face behaviour ratings in each hub.

Paired differences

t df Sig. (2-tailed)Mean Std. deviation Std. error mean

95% confidence interval of the difference

Lower Upper

Hub 1 0.37500 0.74402 0.26305 –0.24702 0.99702 1.426 7 0.197

Hub 2 2.71429 3.25137 1.22890 –0.29273 5.72130 2.209 6 0.069

Hub 3 0.87500 1.88509 0.66648 –0.70098 2.45098 1.313 7 0.231

Table 2. Correlation coefficients for the Clinical evaluation of lan-

guage fundamentals – 4th edition (ELF-4) subtest raw scores.

Test scores n Correlation p-Value

Subtest scaled scores

Concepts & Following Directions 23 0.99 <0.001

Formulated Sentences 23 0.97 <0.001

Word Classes (Receptive) 19 1.0 <0.001

Word Classes (Expressive) 19 0.96 <0.001

Recalling sentences 23 0.96 <0.001

Core Language Score 23 0.99 <0.001

Table 4. Parent comments regarding satisfaction.

Parent Comment

1 No issues at all.

2 Was very relaxed and enjoyable.

4 I believe this is a great idea. This could expand to the bush;

where services are in need. Keep up the good work.

5 My child thought (the telehealth speech pathologist) was

very friendly. I thought she did very well at making him

feel comfortable and was very good at judging when he

needed a break.

7 (Child) done (sic) very well, I was very happy.

8 (My child), who was reluctant to attend the centre, stated

after this videoconferencing assessment that he ‘liked

coming’ and was ‘having fun’. A statement never made

before when in conjunction with education and/or

learning.

9 (My child) seemed excited and happy to be doing work on

computer and video chat.

10 (My child) enjoyed himself and said it was fun.

11 It showed us some of the areas where (my child) has diffi-

culty. I felt the videoconference was very worthwhile.

12 I found when he got a question wrong he would loose (sic)

concentration then would take another three to four

questions to get back on track.

13 (My child) did like to use the computer and the videocon-

ferencing was quite a novelty for him but he would prefer

to talk to a person face-to-face without the computer link

up. I also thought the sound through the computer was

difficult at times to hear/understand. (My child) spent a lot

of time concentrating on understanding what was being

said.

8 Journal of Telemedicine and Telecare 0(0)

at Xi’an Jiaotong University on June 4, 2016jtt.sagepub.comDownloaded from

a preference for parents to access this service compared to attending separate speech and language assessments. Similarly, Dunkley and colleagues

14 found that rural resi-

dents expressed willingness to trial services provided in this way, in part because of their isolation and lack of alternative services. The results of the current study indicated positive attitudes towards the use of telehealth in this setting with all families indicating that both they and their children felt comfortable using the telehealth system.

The barriers to using telehealth described by a number of authors have been somewhat addressed in this study.

15,19 The study attempted to mitigate financial and

access barriers by conducting assessment in local schools, using commercially available equipment with standard school-based computer equipment and Internet access. While proportionately fewer rural and remote homes have Internet access compared to homes in urban areas,

20 schools generally have access even in very

remote areas and all government schools in New South Wales (NSW), for example, have been connected to the Internet since 1996.

21 Barriers arising from technology

difficulties were addressed using a simple to access, web- based application that required no download or installa- tion and no technical knowledge beyond accessing a web page. Cultural barriers, including reluctance to engage in telehealth and the belief that telehealth is restricted to research settings were addressed in this study by showing that the use of telehealth in a real-world setting, outside a specialist research facility

15 is feasible. The effectiveness of

using telehealth in a clinical setting was reflected in the good reliability of scores across conditions, in high parent satisfaction ratings, and in how well the telehealth assess- ments were tolerated by the children as demonstrated by the behaviour observations across conditions.

Limitations

The present study has a number of limitations. As with Waite and colleagues,

11 only the four core language subt-

ests were administered rather than all the subtests required for a comprehensive assessment. Additionally, only mod- erate numbers (n¼23) were involved in this cohort and no comparison group was included. While some counterba- lancing to control for order effects occurred naturally, this was not randomised. However, in keeping with the real- world clinical caseloads, no children were excluded from the assessment. All those who could be assessed by both the telehealth and face-to-face speech pathologists during the study timeframe were included, with no refu- sals or withdrawals from the study.

Future research

The findings of the present study support the use of a telehealth application to assess the language of children with reading difficulties in a real-world situation using commercially available equipment. Future studies should investigate the feasibility and tolerability of telehealth

assessments of children with diverse learning needs, including those with additional social and behavioural difficulties that may impact on assessment administration (e.g. children with autism spectrum disorder). The use of telehealth to undertake all subtests of formal assessment, along with other language measures such as narrative assessments and conversation should also be considered. Finally, it will be important to examine the practicalities of using of telehealth systems for language assessments in clinic and education settings with the support of educators or teaching assistants to manage the assessments.

Acknowledgements

The authors wish to acknowledge Silvia Pfeiffer, Terence

Percival, Rosemary Hollowell of the National ICT Australia (NICTA), and Marcia Williamsz, Kate Baggett, Fiona Eastley, Chantelle Medley, Sylvia Pope, Jenny Rayner, Neale Waddy of

The NSW Centre for Effective Reading, NSW Department of Education and The Child Development Unit, The Children’s Hospital at Westmead.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support

for the research, authorship, and/or publication of this article: this work was supported by the NSW Department of Education. National ICT Australia (NICTA) provided in-kind support in the form of the design of the assessment website interface.

References

1. Australia Bureau of Statistics. Australia demographic statistics (cat. no. 3101.0). Canberra, ACT: ABS, 2014.

2. Australian Institute of Health and Welfare (AIHW). Health and community services labour force. National health labour force series number 42. Canberra, ACT: AIHW, 2009.

3. Community Affairs References Committee. Prevalence of dif- ferent types of speech, language and communication disorders and speech pathology services in Australia. Canberra, ACT:

Community Affairs References Committee, 2014. 4. O’CallaghanAM,McAllisterLandWilsonL.Barrierstoaccess-

ing rural paediatric speech pathology services: Health care con- sumers’ perspectives. Aust J Rural Health 2005; 13: 162–171.

5. Constantinescu G, Theodoros D, Russell T, et al. Assessing disordered speech and voice in Parkinson’s disease: A telereh- abilitation application. Int J Lang Commun Disord 2010; 45:

630–644. 6. Sharma S, Ward EC, Burns C, et al. Assessing dysphagia via

telerehabilitation: Patient perceptions and satisfaction. Int J

Speech Lang Pathol 2013; 15: 176–183. 7. Ward EC, Sharma S, Burns C, et al. Validity of conducting

clinical dysphagia assessments for patients with normal to

mild cognitive impairment via telerehabilitation. Dysphagia 2012; 27: 460–472.

8. Lewis C, Packman A, Onslow M, et al. A phase II trial of telehealth delivery of the Lidcombe Program of Early

Stuttering Intervention. Am J Speech Lang Pathol 2008; 17: 139–149.

Sutherland et al. 9

at Xi’an Jiaotong University on June 4, 2016jtt.sagepub.comDownloaded from

9. Grogan-Johnson S, Schmidt AM, Schenker J, et al. A com-

parison of speech sound intervention delivered by teleprac- tice and side-by-side service delivery models. Commun Disord Q 2013; 34: 210–220.

10. Eriks-Brophy A, Quittenbaum J, Anderson D, et al. Part of

the problem or part of the solution? Communication assess- ments of Aboriginal children residing in remote communities using videoconferencing. Clin Linguist Phon 2008; 22:

589–609. 11. Waite MC, Theodoros DG, Russell TG, et al. Internet-

based telehealth assessment of language using the

CELF–4. Lang Speech Hear Serv Sch 2010; 41: 445–458. 12. Ciccia AH, Whitford B, Krumm M, et al. Improving the

access of young urban children to speech, language and

hearing screening via telehealth. J Telemed Telecare 2011; 17: 240–244.

13. Taylor OD, Armfield NR, Dodrill P, et al. A review of the efficacy and effectiveness of using telehealth for paediatric

speech and language assessment. J Telemed Telecare 2014; 20: 405–412.

14. Dunkley C, Pattie L, Wilson L, et al. A comparison of rural

speech-language pathologists’ and residents’ access to and attitudes towards the use of technology for speech-language

pathology service delivery. Int J Speech Lang Pathol 2010;

12: 333–343. 15. May J and Erikson S. Telehealth: Why not? Perspectives of

speech pathologists not engaging in telehealth. Journal of Clinical Practice in Speech-Language Pathology 2014; 16:

147–151. 16. Bland JM and Altman DG. Measuring agreement in method

comparison studies. Stat Methods Med Res 1999; 8: 135–160.

17. Semel E, Wiig E and Secord W. Clinical evaluation of lan- guage fundamentals, 4th edition. San Antonio, TX: Harcourt Assessment, Psychological Corporation, 2003.

18. Wiig EH, Secord W, Semel EM. CELF preschool 2: Clinical evaluation of language fundamentals preschool. Pearson/ PsychCorp, 2004.

19. Jang-Jaccard J, Nepal S, Alem L, et al. Barriers for deliver- ing telehealth in rural Australia: A review based on Australian trials and studies. Telemed J E Health 2014; 20: 496–504.

20. Australia Bureau of Statistics. Online @ Home, Australian Social Trends, cat. no. 4102.0. Canberra, ACT: ABS, 2011.

21. Standing Council on School Education and Early

Childhood (SCSEEC). National report on schooling in Australia. 1996.

10 Journal of Telemedicine and Telecare 0(0)

at Xi’an Jiaotong University on June 4, 2016jtt.sagepub.comDownloaded from