Systematic Review Chart

Vol:.(1234567890)

J Autism Dev Disord (2017) 47:898–904 DOI 10.1007/s10803-016-3002-3

1 3

BRIEF REPORT

Brief Report: Using a Point-of-View Camera to Measure Eye Gaze in Young Children with Autism Spectrum Disorder During Naturalistic Social Interactions: A Pilot Study

Sarah R. Edmunds1 · Agata Rozga2 · Yin Li2 · Elizabeth A. Karp1 · Lisa V. Ibanez1 · James M. Rehg2 · Wendy L. Stone1 

Published online: 9 January 2017 © Springer Science+Business Media New York 2017

and engagement (Jones et  al. 2008; Jones and Klin 2013; Mundy et al. 2009), and studies have found that young chil- dren with ASD engage in less frequent eye contact than do typically developing (TD) children (Barbaro and Dissanay- ake 2012; Jones et  al. 2008; Yoder et  al. 2009; Zwaigen- baum et al. 2005). The conventional protocol for measuring eye contact has been to use one or more video cameras to record children during free play or semi-structured interac- tions with examiners or parents and then have independent raters code the videos (Mundy et  al. 2003; Wetherby and Prizant 2002). A reliance on cameras stationed at a dis- tance from the interaction generally only enables coding of “looks to face” or “looks to examiner” rather than measur- ing eye contact directly (e.g. McDuffie et al. 2006; Mundy et al. 2007; Rozga et al. 2011; Yoder et al. 2009).

More recently, monitor-based eye tracking has enabled more precise measurement of children’s eye gaze to differ- ent parts of the face. Young children with ASD have been found to look less to others’ faces, particularly their eyes, compared to TD children in studies using monitor-based eye tracking methods (Chawarska et  al. 2012; Chawarska and Shic 2009; see Falck-Ytter et  al. 2013 and Guillon et al. 2014 for reviews). While this approach has yielded a wealth of information about how children process faces, its ecological validity is limited by its use of two-dimensional stimuli on a screen rather than engagement in an actual social interaction (Guillon et  al. 2014). Another emerging strategy for measuring eye contact is placing head-mounted cameras or wearable eye trackers on children to record their visual perspective directly during social interactions (Fran- chak et al. 2011; Noris et al. 2012). However, these systems provide challenges for young children with ASD and have been unable to support coding of children’s looks to their social partner’s eyes, beyond codes of looks to her face (Noris et al. 2012). More recent research using eye tracking

Abstract Children with autism spectrum disorder (ASD) show reduced gaze to social partners. Eye contact during live interactions is often measured using stationary cameras that capture various views of the child, but determining a child’s precise gaze target within another’s face is nearly impossible. This study compared eye gaze coding derived from stationary cameras to coding derived from a “point- of-view” (PoV) camera on the social partner. Interobserver agreement for gaze targets was higher using PoV cameras relative to stationary cameras. PoV camera codes, but not stationary cameras codes, revealed a difference between gaze targets of children with ASD and typically develop- ing children. PoV cameras may provide a more sensitive method for measuring eye contact in children with ASD during live interactions.

Keywords Autism · Eye gaze · Behavioral coding · Measurement · Social interaction

Introduction

Impaired eye contact has long been recognized as a clini- cal feature that is central to the diagnosis of autism spec- trum disorder (ASD; APA 2013). Eye contact is consid- ered to indicate the quality or presence of social interest

* Sarah R. Edmunds [email protected]

1 Department of Psychology, University of Washington, Box 351525, Seattle, WA 98195, USA

2 School of Interactive Computing, Georgia Institute of Technology, 85 5th St Nw, Technology Square Research Building, Room 218a, Atlanta, GA 30332, USA

899J Autism Dev Disord (2017) 47:898–904

1 3

during live interactions with an adult shows potential, but thus far has only examined children’s looks to the top and bottom portions of an adult’s face (e.g. Falck-Ytter 2015; Thorup et  al. 2016), rather than to specific facial features such as the eyes.

The ability to assess whether children are making eye contact in the course of naturalistic interactions has the potential to inform both developmental and social motiva- tion accounts of the emergence of ASD (e.g. Chevallier et al. 2012). Examining eye contact in real-life social inter- actions over time in young children may yield new infor- mation as to when atypical eye contact becomes apparent. The relationship between the onset of atypical eye contact and other social behaviors may yield information to sup- port or weaken the social motivation hypothesis, given that the theory posits that a deficit in social orienting (e.g. mak- ing eye contact) early in life leads to more complex down- stream deficits in social communication.

This project examines a novel, unobtrusive method for capturing children’s gaze during naturalistic social interac- tions that may enable coders to be more sensitive in identi- fying eye contact than when using videos recorded by sta- tionary cameras. We instrumented the child’s adult social partner with glasses that contain an outward facing (point of view, PoV) camera in the nose bridge (Fig.  1). The child’s looks toward the adult’s eyes are captured as looks that appear to be directed into the camera. Our objective was to explore the utility of the PoV camera in enabling coders to reliably identify eye gaze targets in children with ASD during a live interaction, compared to coding of vid- eos from stationary cameras.

To evaluate the reliability of this method, we compared interobserver agreement for eye gaze codes from video recordings made from the PoV camera and stationary cam- eras. To evaluate the potential validity of the method, we examined differences by diagnosis (ASD vs. TD) in chil- dren’s looks to the face and eyes of their adult partner. We hypothesized that: (1) interobserver agreement for chil- dren’s looks to the partner’s eyes (vs. the face) would be higher for codes derived from the PoV camera than from

the stationary cameras; (2) the codes derived from both the PoV and the stationary cameras would detect group differ- ences in children’s looks to their partner’s face (given sta- tionary cameras have been used successfully for this pur- pose); and (3) codes derived from the PoV camera, but not the stationary cameras, would detect less frequent looks to the adult’s eyes for children with ASD compared to TD children.

Methods

Participants

Eight children with ASD (M = 33.5 mo; SD = 3.5 mo; six males) and seven TD children (M = 34.6 mo; SD = 6.0 mo; seven males) participated in the study. Children in the ASD group received the Autism Diagnostic Observation Schedule (ADOS; Lord et al. 2000) and a parent interview through participation in other research studies; ASD diag- noses were made by a licensed psychologist. The study was IRB-approved. Parental informed consent was obtained.

Procedure

Each child participated in a 5-minute play interaction with an examiner comprised of three scripted activities designed to orient the child to different parts of the examiner’s face and body. First, the examiner held a small snack item over parts of her face (i.e., her left and right eyes, nose, and mouth) while describing her actions enthusiastically (e.g. “I’m putting it on my eye!”). Next, the examiner sang “Itsy- Bitsy Spider” with traditional accompanying hand motions. Finally, the examiner made four facial expressions: happy, sad, scared, and surprised. Before each expression, she obtained the child’s attention by saying “I’m going to make a ___ face!” and after each expression asked, “Was that silly?” This sequence was repeated three times for three “periods.” In Periods 1 and 3, the examiner did not wear the PoV camera, and videos were recorded only from the

Fig. 1 PoV camera and room arrangement. a “Point of View” (PoV) camera. The eyeglass lenses that appear in this illustration were removed to reduce glare and reduce obstruction of the adult’s eyes. b

Room arrangement. The location of the three stationary cameras (cir- cled in red) are shown with reference to the testing table. The child sat in the yellow chair, directly across from the adult

900 J Autism Dev Disord (2017) 47:898–904

1 3

stationary cameras. In Period 2, the examiner wore the PoV camera, and videos were recorded simultaneously from both the PoV camera and the stationary cameras. The recordings from the stationary cameras were manually syn- chronized using a cue visible in all of the cameras (flash- ing a light). For the purposes of this report, only data from Period 2 were used for the main analyses: codes from the stationary cameras were compared to codes from the PoV camera during Period 2.

Apparatus

Children sat at a small table in a small chair or in their caregiver’s lap. An examiner sat across the table with her face at eye level and approximately 18–24 inches from the child’s face. Three stationary cameras were positioned around the child: two feet behind the child’s left shoulder, capturing the examiner’s face and torso; two feet behind the examiner’s left shoulder, capturing the child’s face and torso; and approximately two feet to the side of the table,

capturing both the examiner and the child in profile (see Fig. 2). The examiner wore Pivothead Kudu glasses, which have a camera embedded in the nose bridge (Fig.  1) and thus capture a video of the child’s face and upper torso from the examiner’s perspective. The lenses were removed to provide an unobstructed view of the examiner’s eyes and prevent glare.

Coding

Research assistants blind to participants’ diagnostic status and study aims coded gaze targets frame-by-frame using ELAN, a free video coding software (http://tla.mpi.nl/tools/ tla-tools/elan/; Lausberg and Sloetjes 2009). Four mutu- ally exclusive and exhaustive gaze target categories were coded: (1) eyes, (2) face (other than eyes), (3) looks away (anywhere other than the examiner’s eyes or face), and (4) “uncodeable” looks (when the child’s eyes were out of frame, or there was excessive shakiness/blur). Coders were

Fig. 2 Sample images for coding, illustrating the camera views, the amount of the frame the child participant occupies in the video from each camera type, and the time-locked cod- ing tier framework of ELAN. a Images from the three stationary cameras. All three views are visible, and coders choose which view will be enlarged. b Image from the PoV camera

901J Autism Dev Disord (2017) 47:898–904

1 3

required to make a decision for every video frame as to which of the gaze target categories it belonged.

Each child’s gaze during Period 2 was coded from two different ELAN files: a file containing the PoV video and a file containing the three videos from the stationary cameras (Fig. 2). In the file with the three stationary camera videos, coders were free to make any video the “primary,” or rela- tively larger video, at any time. Each file was coded twice, by two different coders. File coding order was counterbal- anced by child and camera type to minimize the effect of coders’ experience over time. If a coder coded both the PoV and the stationary camera view for the same participant, at least 2 weeks and several other participants’ files separated the coding of the two files. Before coding began, coders reviewed one file from the stationary cameras and one file from the PoV camera with a supervisor (not included in the current sample). Coders first watched the video once in real-time to become familiar with each child’s gaze behav- ior and then flagged the onset and offset of each gaze event at the frame level.

Analysis Plan

For the purposes of calculating reliability, the identity of “coder 1” and “coder 2” was randomized for each video. Interobserver agreement was calculated in two ways. First, we extracted counts of the number of times each gaze tar- get (eyes, face, away, and “uncodeable”) was used for each participant by each coder and calculated two-way random single measures absolute intraclass correlation coefficients (ICCs) for agreement on total event counts/frequency of each instance of gaze to each target (Shrout and Fleiss 1979). Second, we extracted frame-level agreements and disagreements for each gaze target compared to all other gaze target locations and computed a Kappa coefficient (Cohen 1960; Yoder and Symons 2010). Kappa provides a more stringent way of calculating reliability, as it takes into account chance agreement and requires coders to agree on the frame-by-frame classification, rather than just the over- all frequency, of gaze targets. Multivariate ANOVAs were conducted to assess for differences in the rate per minute (controlling for small variations in period length) of gaze by diagnosis (ASD vs. TD) and camera type (PoV vs. sta- tionary camera).

Results

Preliminary Analyses

A repeated-measures ANOVA (Group × Period) was con- ducted to determine whether looks to eyes, as measured by the stationary cameras, varied by diagnostic status and

across the three periods (i.e., whether or not the exam- iner was wearing the PoV glasses). Results revealed no significant main effect for Period, F (2,26) = 0.261, p = .77, suggesting that wearing the PoV camera did not draw undue attention to the examiner’s eye region. The Group main effect and Group by Period interaction were also non-significant, F (1,13) = 4.59, p = .05, and F (2,26) = 0.70, p = .51, respectively; children’s looks to eyes did not differ by diagnosis across periods.

Reliability of Codes from PoV Versus Stationary Cameras

Interobserver reliability for gaze targets was assessed separately for codes derived from the PoV and station- ary camera recordings. Coding from the PoV recordings produced higher absolute ICCs than those from the sta- tionary cameras for looks to adults’ eyes (0.69 vs. −0.01, respectively) and for uncodeable looks (0.65 vs. 0.33, respectively; see Table  1). In contrast, absolute ICCs were comparable for the PoV camera and the stationary cameras for looks to the face and looks away. Kappas for frame-level agreement are reported for all gaze targets in Table  1. Kappas for looks to eyes compared to all other gaze targets combined (face, away, uncodeable) were k = 0.67 for the PoV camera and k = 0.49 for the station- ary cameras (Table  1; Fig.  3). Rate per minute of looks to each gaze target are reported by camera type and diag- nostic status in Table 2.

Effect of Diagnostic Status on Children’s Gaze Behavior

Multivariate ANOVAs were used to examine the effect of diagnosis on rates of looks to different gaze targets (i.e., eyes, face, and away and uncodeable as correlated dependent variables) for each camera type separately (Table  2). As measured by the PoV camera, children’s

Table 1 Interobserver reliability for eye gaze codes from PoV and stationary cameras

Gaze target Absolute intraclass corre- lation coefficient (ICC)

Cohen’s Kappa

PoV camera Stationary cameras

PoV camera Station- ary cameras

Adult’s eyes 0.69 −0.01 0.67 0.49 Adult’s face 0.71 0.75 0.53 0.43 Away 0.85 0.77 0.76 0.84 Uncodeable 0.65 0.33 0.55 0.25

902 J Autism Dev Disord (2017) 47:898–904

1 3

rate of looks to different gaze targets varied by diagnosis, F(4,10) = 4.55, p = .02, in that children with ASD looked at the examiner’s eyes and face less frequently than did TD children, F(1,14) = 6.11, p = .03 and F(1,14) = 16.44, p = .001. There were no significant group differences for looks away or for uncodeable looks. In contrast, when gaze targets were coded from the stationary cameras, there were no significant differences between the children with ASD and TD for any gaze targets, F(4,10) = 2.38, p = .12 (Fig. 4).

Because previous video coding systems have measured looks to face as a proxy for looks to eyes (e.g. Rozga et al. 2011; Yoder et  al. 2009), it was also of interest to deter- mine whether coding from stationary cameras would yield more reliable codes and group differences when the two gaze targets were combined. Results revealed that looks to

[eyes + face] had higher reliability than that for eyes alone [ICC = 0.41, 95% CI (−0.13, 0.76)]. There were no signif- icant differences in looks to [eyes + face] by diagnosis as

coded by stationary cameras; however, the difference did approach a conventional level of significance, t(13) = 2.07, p = .06.

Discussion

The results of this pilot study provide strong preliminary evidence that a PoV camera worn by a child’s social part- ner can enable coders to produce reliable and conceptu- ally valid estimates of young children’s eye gaze during a live social interaction. First, interobserver agreement was higher for looks to eyes when coded from the PoV record- ings than from the stationary camera recordings. Second, group differences for gaze to the adult’s eyes and gaze to

the adult’s face were found only when coding was based on video recordings from the PoV camera. These results sug- gest that the PoV camera glasses may yield more precise

Fig. 3 Kappas for looks to eyes versus not eyes by camera type. Numbers in the cells denote the percentage of frames that Coder 1 (vertical axis) and Coder 2 (horizontal axis) coded in each category

Table 2 Rates for different gaze targets by group and camera type

Gaze target Mean (SD) F (1, 14) p Partial eta squared (h 2

p) ASD TD

PoV camera  Adult’s eyes 8.35 (4.16) 14.54 (5.53) 6.11 0.03 0.32  Adult’s face 8.16 (3.97) 15.64 (3.03) 16.44 0.001 0.56  Away 11.29 (2.11) 16.16 (7.79) 2.90 0.11 0.18  Uncodeable 3.99 (2.90) 8.15 (7.40) 2.17 0.17 0.14

Stationary camera  Adult’s eyes 5.31 (4.17) 7.48 (5.53) 0.73 0.41 0.05  Adult’s face 7.65 (3.66) 12.03 (5.20) 3.65 0.08 0.22  Away 10.40 (3.86) 12.45 (4.37) 0.95 0.35 0.07  Uncodeable 0.21 (0.59) 1.00 (0.26) 0.20 0.66 0.02

Fig. 4 Rate per minute of children’s looks to the adult’s eyes by cam- era type and diagnostic group. *p < .05. Error bars represent +/− one standard error of the mean

903J Autism Dev Disord (2017) 47:898–904

1 3

information regarding eye contact when compared to sta- tionary cameras.

Agreement between raters is a cornerstone of observa- tional research (Yoder and Symons 2010). ICCs for eye contact (i.e., looking to eyes) were quite high when coded from the PoV recordings, while they were at chance lev- els when coded from the stationary camera recordings. The Kappas yielded somewhat different results. While the Kappa coefficients were higher for the PoV recordings, both were in an acceptable range. ICCs only provide event- level reliability, while Kappas are a more stringent measure in this case because they provide frame-level reliability and better capture the reliability of the timing of the codes.

When looks to eyes and face were combined as coded from stationary cameras, there was no significant group dif- ference in gaze to others’ eyes and face combined, suggest- ing that stationary cameras may not be sufficiently sensitive to detect group differences in eye contact when coders are specifically instructed to code looks to others’ eyes sepa- rately from looks to their face. However, the fact that the group difference trended towards significance does suggest that previous studies may have found group differences in eye contact because they may have instructed coders to consider a wider “region of interest” around others’ eyes as eye contact.

To our knowledge, this is one of the first studies to rep- licate findings of decreased looking specifically to a social partner’s eyes (vs. face) in children with ASD during a real- life social interaction. The importance of looking specifi- cally at eye contact has been highlighted by recent studies indicating that early developmental patterns of eye contact may be predictive of later diagnosis (e.g. Jones and Klin 2013). These studies have used two-dimensional stimuli (videos), but live interactions may provide more meaning- ful results due to their increased ecological validity. The PoV camera may also be used to study more complex eye gaze behaviors such as joint attention and gaze following. Studies using monitor-based eye tracking have found that young children at risk for ASD may make inferences based on head turn or direction, rather than eye gaze alone (Tho- rup et al. 2016).

One limitation of this study is that we cannot be cer- tain that children were looking directly at the eyes of the examiner. Although the PoV camera enabled coders to make reliable distinctions between looks to the eyes versus other parts of the face, there still remains a “zone of ambiguity,” as we cannot differentiate looks to the center of the eyes from looks to the eyebrows or bridge of the nose. However, we do believe that the PoV camera improves upon the current “gold standard” measurement of eye contact, for which looks to the face are accepted as a proxy for eye contact. Although the PoV camera clearly narrows this “zone,” some ambiguity remains regarding

the exact focus of the child’s gaze. Notably, even with commercial eye trackers there is an error ellipse that defines a zone of ambiguity within the region of interest (e.g., eyes).

The findings reported here are preliminary and require replication with larger samples and more natural settings. For example, it would be useful to examine whether the PoV camera generates codeable videos of free-play, out- door, or floor interactions. Additional empirical work is needed to determine the optimal conditions (e.g. distance and angle at which the pair interact, movement level) under which the PoV camera can yield codeable images. The PoV camera represents one of many innovative ways of capturing eye contact, and it may be that different combinations of technology allow for the most accurate coding of naturalistic interactions.

The PoV camera is unobtrusive and cost-effective com- pared to traditional stationary cameras. The glasses used in this study cost $199.00 at retail, which is less costly than a set of stationary cameras with video-mixing soft- ware. Overall, instrumenting adult social partners with a simple wearable camera may be an important avenue for studying gaze during naturalistic social interactions.

Acknowledgments We offer our sincere thanks to the families who have participated in this research. We would also like to thank Katherine Ragsdale, Courtney Froehlig, Ghina Haidar, and Jasmine Yip for their assistance in coding. This research was supported in part by Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) Grant R01 HD057284.

Author contributions SE conceived of the study, participated in data collection, performed statistical analyses, participated in the interpretation of the data, and drafted the manuscript. AR helped conceive of the study, performed statistical analyses, participated in the interpretation of the data, and helped to draft the manuscript. YL performed statistical analyses and assisted in the interpretation of the data. EK participated in data collection, participated in the interpreta- tion of the data, and helped to draft the manuscript. LI helped con- ceive of the study, assisted in the interpretation of the data, and helped to draft the manuscript. JR helped conceive of the study and assisted in the interpretation of the data. WS helped conceive of the study, assisted in the interpretation of the data, and helped to draft the manu- script. All authors read and approved the final manuscript.

Compliance with Ethical Standards

Conflict of interest The authors declare that they have no conflict of interest.

Ethical Approval All procedures performed in studies involving human participations were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent Informed consent was obtained from all indi- vidual participants included in the study.

904 J Autism Dev Disord (2017) 47:898–904

1 3

References

American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders-V. Washington, DC: APA.

Barbaro, J., & Dissanayake, C. (2012). Early markers of autism spec- trum disorders in infants and toddlers prospectively identified in the Social Attention and Communication Study. Autism, 17, 64–86. doi:10.1177/1362361312442597.

Chawarska, K., Macari, S., & Shic, F. (2012). Context modulates attention to social scenes in toddlers with autism. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 53(8), 903–913. doi:10.1111/j.1469-7610.2012.02538.x.

Chawarska, K., & Shic, F. (2009). Looking but not seeing: Atypi- cal visual scanning and recognition of faces in 2 and 4-year-old children with autism spectrum disorder. Journal of Autism and Developmental Disorders, 39(12), 1663–1672. doi:10.1007/ s10803-009-0803-7.

Chevallier, C., Kohls, G., Troiani, V., Brodkin, E. S., & Schultz, R. T. (2012). The social motivation theory of autism. Trends in Cog- nitive Sciences, 16(4), 231–239. doi:10.1016/j.tics.2012.02.007.

Cohen, J. (1960). A coefficient of agreement for nominal scales. Edu- cational and Psychological Measurement, 20, 37–46.

Falck-Ytter, T., Bölte, S., & Gredebäck, G. (2013). Eye tracking in early autism research. Journal of Neurodevelopmental Disorders, 5(1), 28. doi:10.1186/1866-1955-5-28.

Falck-Ytter, T., Carlström, C., & Johansson, M. (2015). Eye con- tact modulates cognitive processing differently in children with autism. Child Development, 86(1), 37–47. doi:10.1111/ cdev.12273.

Franchak, J. M., Kretch, K. S., Soska, K. C., & Adolph, K. E. (2011). Head-mounted eye tracking: A new method to describe infant looking. Child Development, 82(6), 1738–1750. doi:10.1111/j.1467-8624.2011.01670.x.

Guillon, Q., Hadjikhani, N., Baduel, S., & Rogé, B. (2014). Visual social attention in autism spectrum disorder: Insights from eye tracking studies. Neuroscience and Biobehavioral Reviews, 42, 279–297. doi:10.1016/j.neubiorev.2014.03.013.

Jones, W., Carr, K., & Klin, A. (2008). Absence of preferential look- ing to the eyes of approaching adults predicts level of social disability in 2-year-old toddlers with autism spectrum disorder. Archives of General Psychiatry, 65(8), 946–954. doi:10.1001/ archpsyc.65.8.946.

Jones, W., & Klin, A. (2013). Attention to eyes is present but in decline in 2–6-month-old infants later diagnosed with autism. Nature. doi:10.1038/nature12715.

Lausberg, H., & Sloetjes, H. (2009). Coding gestural behavior with the NEUROGES-ELAN system. Behavior Research Meth- ods, Instruments, & Computers, 41(3), 841–849. doi:10.3758/ BRM.41.3.591.

Lord, C., Risi, S., Lambrecht, L., Cook, E. H., Leventhal, B. L., DiLavore, P. C., Pickles, A., & Rutter, M. (2000). The autism

diagnostic observation schedule–generic: A standard measure of social and communication deficits associated with the spec- trum of autism. Journal of Autism and Developmental Disorders, 30(3), 205–223.

McDuffie, A. S., Yoder, P. J., & Stone, W. L. (2006). Labels increase attention to novel objects in children with autism and compre- hension-matched children with typical development. Autism, 10(3), 288–301. doi:10.1177/1362361306063287.

Mundy, P., Fox, N., & Card, J. (2003). EEG coherence, joint attention and language development in the second year. Developmental Science, 6(1), 48–54. doi:10.1111/1467-7687.00253.

Mundy, P., Block, J., Delgado, C., Pomares, Y., Van Hecke, A. V., & Parlade, M. V. (2007). Individual differences and the devel- opment of joint attention in infancy. Child Development, 78(3), 938–954.

Mundy, P., Sullivan, L., & Mastergeorge, A. M. (2009). A parallel and distributed-processing model of joint attention, social cognition and autism. Autism Research, 2, 2–21. doi:10.1002/aur.61.

Noris, B., Nadel, J., Barker, M., Hadjikhani, N., & Billard, A. (2012). Investigating gaze of children with ASD in naturalistic settings. PloS One, 7(9), e44144. doi:10.1371/journal.pone.0044144.

Rozga, A., Hutman, T., Young, G. S., Rogers, S. J., Ozonoff, S., Dapretto, M., & Sigman, M. (2011). Behavioral profiles of affected and unaffected siblings of children with autism: Con- tribution of measures of mother-infant interaction and nonverbal communication. Journal of Autism and Developmental Disor- ders, 41(3), 287–301. doi:10.1007/s10803-010-1051-6.

Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Use in assessing rater reliability. Psychological Bulletin, 86(2), 420–428.

Thorup, E., Nyström, P., Gredebäck, G., Bölte, S., Falck-Ytter, T., & Team, T. E. (2016). Altered gaze following during live interac- tion in infants at risk for autism: An eye tracking study. Molecu- lar Autism.doi:10.1186/s13229-016-0069-9.

Wetherby, A., & Prizant, B. (2002). Communication and symbolic behavior scales developmental profile—First Normed edition. Baltimore, MD: Paul H. Brookes.

Yoder, P., Stone, W. L., Walden, T., & Malesa, E. (2009). Predict- ing social impairment and ASD diagnosis in younger siblings of children with autism spectrum disorder. Journal of Autism and Developmental Disorders, 39(10), 1381–1391. doi:10.1007/ s10803-009-0753-0.

Yoder, P., & Symons, F. (2010). Observational measurement of behavior. New York: Springer.

Zwaigenbaum, L., Bryson, S., Rogers, T., Roberts, W., Brian, J., & Szatmari, P. (2005). Behavioral manifestations of autism in the first year of life. International Journal of Developmental Neuro- science, 23(2–3), 143–152. doi:10.1016/j.ijdevneu.2004.05.001.

Journal of Autism & Developmental Disorders is a copyright of Springer, 2017. All Rights Reserved.