Systematic Review

Data extraction and Outcomes Classification

Data from the seven articles that met predetermined inclusion criteria outlined by the

QAT (Table 3) and passed the extraction portion of the screening process for further analysis

was extracted and is presented in Table 4. Each article was broken down by authors, publication

year, country, study aims, study design, sample size/setting, intervention, and findings/outcomes.

Statistical Analysis

The articles selected for this systematic review each utilized at least two methods for statistical

analysis. Methods varied across studies and included the following: Analysis of Variance

(ANOVA), Multivariate Analysis of Variance (MANOVA), simple linear regression, method

comparison analysis, Pearson’s correlation, Bland-Altman analyses (limits/measures of

agreement, plots), t-tests (independent and paired), Wilcoxon signed rank test, Kolmogorov-

Smirnov Test for Normality, Bonferroni procedure, Cohen’s kappa (weighted and unweighted),

Intraclass Correlation Coefficients (ICC), Spearman’s Rank Correlation Coefficient, Systematic

Analysis of Language Transcripts (SALT), and power analysis. Only one study (article 3)

specified using the Statistical Package for the Social Sciences (SPSS) software to perform

statistical analyses procedures.

Interrater Reliability

Weighted percent agreement was used to calculate the interrater reliability of the

researchers who independently completed the screening of titles and abstracts and then

proceeded to review the full-text articles. Percent agreement for the screening of titles and

abstracts was computed through Covidence, the online screening research tool utilized during

this process. Weighted percent agreement for the titles and abstracts screening round was 96%

and 73% for the full-text review round.

Results

Participant Characteristics

Majority of the studies (57%) focused on individuals with either suspected impairment in

language, hearing, speech, and/or academic performance, previously diagnosed impairments

(e.g., language impairment, Specific Learning Disorder with impairment in reading), or a

combination of both (suspected and previously diagnosed). Two studies included in this review

focused on individuals with Autism Spectrum Disorder (ASD) and one study only included

typically developing children. Five of the seven studies specified the inclusion of both female

and male participants, with the majority of participants being male in all five studies. Participant

ages ranged from the youngest being 4-years-old to the oldest being 13-years-old (as reported).

Sample sizes ranged between 6 to 37 participants with most studies (71%) including more than

20 participants. A more detailed breakdown of each study's specific participant characteristics

can be found in Table 4.

Telepractice Technology

In general, most studies offered a description of their chosen telehealth technology. The

researchers typically communicated with participants in real time using video conferencing

software, either commercially produced (e.g., Adobe ConnectPro) or custom made for research

purposes. In four out of seven studies, research teams used custom-made software programs.

Commonly used hardware included commercially available computers, laptops, webcams,

microphones, and/or comprehensive teleconferencing systems. Regarding the internet

connection, a study utilized a 128-Kbs Internet link, while another study accessed the internet

through plug-in dongles, mobile hotspots, wired LAN and/or Wi-Fi hotspot. The rest of the

studies did not provide thorough details about their internet connection.

Feasibility/Language Outcomes

Most of the articles reviewed reported no significant difference between telehealth and face-to-

face scores, three of them reported high agreement between telehealth and face-to-face scores.

Alt, M. and Humphrey, M. (2012) reported no significant difference in behavioral ratings

between the two methods. Regarding behavioral measures, Sutherland et al. (2019) suggests that

the responses of children with ASD to telehealth are likely to be highly individual. Sutherland et

al. (2017) determined the system for telehealth delivery was feasible and reported the reactions

of the parents and children on the use of telehealth were largely positive and supportive to assess

rural children. Raman et al. (2019) found that using multiple internet options at both sites helped

overcome technical challenges related to connectivity during screening through telemethod.

Language outcomes through face-to-face and telehealth methods revealed no significant

differences in both receptive and expressive domains. Additionally, trained facilitators were

essential in overcoming child-related factors (e.g., poor speech intelligibility, poor audibility of

voice, motivation, interaction with SLP, and need for frequent breaks). Overall, most of the

articles reviewed reported no significant differences in score between telehealth and face-to-face

methods and high satisfaction with feasibility and comfort level of parents and children via

telehealth method.

Patient Satisfaction

Most of the studies did not report client and clinician satisfaction, only three of them reported on

parent satisfaction and child’s comments. Overall, parents reported high satisfaction and comfort

with the telehealth assessments. In addition, the parents reported positive behaviors in the

children. No parents indicated that they felt uncomfortable with the telehealth assessments.

Parents were asked to complete a survey in which the comments from parents reported that their

child had a positive experience and many of the parents commented on their own positive

experience or opinion of the telehealth assessments. However, in (add study here) “Two parents

expressed concern: one parent felt their child lost concentration when he got a question wrong;

the second parent stated their child ‘did like to use the computer and the videoconferencing....

but he would prefer to talk to a person face-to-face’.” Nevertheless, most parents reported that

their children really enjoyed the experience and one parent commented that on the way home the

child stated “it was awesome!” (study)

Discussion

A synthesis of studies examining the feasibility and reliability of conducting language-

based assessments in children ranging in age from 3 to 13 years and with varying abilities

(known/diagnosed disorder v. typical) was conducted to discover what factors (child and

technology- related) may specifically impact assessment administration via telehealth in

comparison to face-to-face administration. This systematic review also looks to investigate

behavioral implications throughout assessment administration and parental satisfaction with

telehealth alternatives.

Of the seven articles, only two were a method comparison study design (2, 3) as opposed

to the other five which followed a non-randomized design(1, 4, 5, 6, 7). Majority of the studies

were carried out in Australia (2, 3, 4, 5), two in the US (1, 6), and one in India (7). All of the

studies were focused on determining the feasibility and/or reliability of administering a

language-based assessment via telehealth. Articles 1, 3, and 7 also examined the behavioral

implications of telehealth administration along with observing technology-related factors (7).

Based on the results of this integrative review, a need for further research has been identified to

determine the feasibility and reliability of conducting language-based assessments particularly in

bilingual children between the ages of 36 to 47 months.

Limitations

Conclusion

Current research suggests that there is no significant difference between remote and face-

to-face assessment administration. The results of the systematic review have demonstrated the

current feasibility of administering assessments online, however, further research is necessary to

determine the feasibility and reliability of conducting language-based assessments online

particularly for bilingual children between the ages of 36 to 47 months.. This service delivery

model has the potential to be used by speech-language pathologists (SLPs) to provide

assessments to children in remote communities and those communities where access to bilingual

SLPs is lacking. Future research should focus on the capacity to provide these telehealth

assessments in schools or clinics.

References

Sirriyeh, R., Lawton, R., Gardner, P., & Armitage, G. (2012). Reviewing studies with

diverse designs: the development and evaluation of a new tool. Journal of Evaluation in

Clinical Practice, 18(4), 746–752. https://doi-org.ezproxy.fiu.edu/10.1111/j.1365-

2753.2011.01662.x