case study question/fill the worksheet
“Robovie, You’ll Have to Go into the Closet Now”: Children’s Social and Moral Relationships With a Humanoid Robot
Peter H. Kahn, Jr. University of Washington
Takayuki Kanda Advanced Telecommunications Research Institute
Hiroshi Ishiguro Advanced Telecommunications Research Institute and
Osaka University
Nathan G. Freier and Rachel L. Severson University of Washington
Brian T. Gill Seattle Pacific University
Jolina H. Ruckert and Solace Shen University of Washington
Children will increasingly come of age with personified robots and potentially form social and even moral relationships with them. What will such relationships look like? To address this question, 90 children (9-, 12-, and 15-year-olds) initially interacted with a humanoid robot, Robovie, in 15-min sessions. Each session ended when an experimenter interrupted Robovie’s turn at a game and, against Robovie’s stated objections, put Robovie into a closet. Each child was then engaged in a 50-min structural– developmental interview. Results showed that during the interaction sessions, all of the children engaged in physical and verbal social behaviors with Robovie. The interview data showed that the majority of children believed that Robovie had mental states (e.g., was intelligent and had feelings) and was a social being (e.g., could be a friend, offer comfort, and be trusted with secrets). In terms of Robovie’s moral standing, children believed that Robovie deserved fair treatment and should not be harmed psychologically but did not believe that Robovie was entitled to its own liberty (Robovie could be bought and sold) or civil rights (in terms of voting rights and deserving compen- sation for work performed). Developmentally, while more than half the 15-year-olds conceptualized Robovie as a mental, social, and partly moral other, they did so to a lesser degree than the 9- and 12-year-olds. Discussion focuses on how (a) children’s social and moral relationships with future personified robots may well be substantial and meaningful and (b) personified robots of the future may emerge as a unique ontological category.
Keywords: beliefs about robots, human–robot interaction, mental models of robots, responses to auton- omy, robots with children
There is little doubt that humanoid robots will become part of our everyday social lives. They may become academic tutors or day care assistants for our children, office receptionists, tour guides, bankers that replace ATMs, caretaking assistants for the older adults, or maids in our homes. Some robots have already functioned in these ways, at least as prototypes. For example, Advanced Telecommunications Research’s humanoid robot Robo-
vie acted as a museum guide at the Osaka Science Museum in Japan and autonomously led groups of school children around the museum and taught them about the exhibits (Shiomi, Kanda, Ishiguro, & Hagita, 2006).
While such technology will continue to be invented and contexts of use found, the current study sought to address the following question: What are the social and moral relationships children will
Peter H. Kahn, Jr., Department of Psychology, University of Washing- ton; Takayuki Kanda, Intelligent Robotics and Communication Laborato- ries, Advanced Telecommunications Research Institute, Keihanna Science City, Japan; Hiroshi Ishiguro, Intelligent Robotics and Communication Laboratories, Advanced Telecommunications Research Institute, Keihanna Science City, Japan, and Graduate School of Engineering, Osaka Univer- sity, Osaka, Japan; Nathan G. Freier, Information School, University of Washington; Rachel L. Severson, Department of Psychology, University of Washington; Brian T. Gill, Department of Mathematics, Seattle Pacific University; Jolina H. Ruckert and Solace Shen, Department of Psychology, University of Washington.
Nathan G. Freier is now at Office Labs, Microsoft. Rachel L. Severson is now at the Department of Psychology, Western Washington University.
Takayuki Kanda and Hiroshi Ishiguro are employees of Advanced Telecommunications Research Institute in Keihanna Science City, Japan, and developers of the humanoid robot, Robovie.
This research was based upon work supported by the National Science Foundation Grants IIS-0325035 and IIS-0905289. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
We thank Shogo Nabe, Cady Stanton, and Lisa Nathan for assis- tance in our data collection efforts, and Amanda Ryan for reliability coding.
Correspondence concerning this article should be addressed to Peter H. Kahn, Jr., Department of Psychology, University of Washington, Box 351525, Seattle, WA 98195-1525. E-mail: pkahn@uw.edu
Developmental Psychology © 2012 American Psychological Association 2012, Vol. 48, No. 2, 303–314 0012-1649/12/$12.00 DOI: 10.1037/a0027033
303
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
form with these robots? It is a puzzling question because on the one hand these robots are artifacts that humans have created. In this sense, they are tools, like a broom. Use it when you want. Stick it in a closet when you are done. On the other hand, these robots are acting and speaking in ways that represent canonical behaviors of an autonomous, thinking, feeling, social, and moral human being. In this sense, they could be viewed as offering the same social affordances as a human and thus merit the same moral considerations.
One overarching finding in the field of human–robot interaction (HRI) is that in varying ways and to varying degrees, people behave socially with robots (Breazeal, 2002; Fong, Nourbakhsh, & Dautenhahn, 2003; Kanda, Hirano, Eaton, & Ishiguro, 2004; Tanaka, Cicourel, & Movellan, 2007). For example, the literature shows that adults seek to establish spatial relations with robots that are not too different from what they seek with other people (Walters et al., 2005), are willing to provide directions to a humanoid robot on a city street when it asks for navigating assistance (Weiss et al., 2010), and are more likely to heed a humanoid robot’s recommendations when it behaves in more culturally normative ways (Wang, Rau, Evers, Robinson, & Hinds, 2010).
There have been, however, at least two limitations to date in the HRI literature. One is that the literature has focused on people’s behavior but not enough on their reasoning. How do people concep- tualize this new entity that is at once lifelike but is not biological life and appears, to varying degrees, to be autonomous, self-organizing, capable of learning new behaviors, and communicative? Answers to this question can shed new light on the development of the human mind.
A second limitation is that while the HRI literature has paid close attention to people’s social behavior with robots, little atten- tion has been paid to people’s potentially moral relationships with robots. The idea of morality is of course both broad and conten- tious (Williams, 1985). But there is general agreement that it involves considerations of fairness, justice, and welfare (Nucci, 2001; Turiel, 1983, 2006) and that certain entities such as fully functioning people have moral standing; some entities such as comatose people, infants, and animals have in various ways partial moral standing; and other entities such as rocks or telephone poles have no moral standing. What about humanoid robots?
The HRI literature with the most relevance to the current study comes from a series of four studies on children’s social and moral relationships with the robot dog AIBO. This robot (which sold for about $1,500) learned new behaviors based on how humans inter- acted with it and could autonomously seek out and kick or head butt a ball. One study compared preschool children’s interactions with and reasoning about AIBO to those with a stuffed dog (Kahn, Friedman, Pérez-Granados, & Freier, 2006). A second study com- pared children’s interactions with and reasoning about AIBO to those with a living biological dog (Melson et al., 2009). A third study compared children with autism interacting with AIBO to those interacting with a mechanical nonrobot dog (Stanton, Kahn, Severson, Ruckert, & Gill, 2008). In a fourth study, researchers analyzed over 3,000 postings in AIBO online discussion forums that spoke of (presumably adult) members’ relationships with their AIBO (Friedman, Kahn, & Hagman, 2003).
Together, these four studies provide converging evidence that children and adults can and often do establish meaningful and
robust social conceptualizations of and relationships with a robot that they recognize as a technology. For example, in the online discussion forum study (Friedman et al., 2003), members affirmed that AIBO was a technology (75%), was lifelike (48%), had mental states (60%), and was a social being (59%). Some postings com- bined the different categories. For example, one member wrote
Oh yeah, I love Spaz [the name for this member’s AIBO]; I tell him that all the time . . . . When I first bought him, I was fascinated by the technology. Since then, I feel I care about him as a pal, not as a cool piece of technology. I do view him as a companion, among other things; he always makes me feel better when things aren’t so great. I dunno about how strong my emotional attachment to him is . . . . I find it’s strong enough that I consider him to be part of my family, that he’s not just a ‘toy’; he’s more of a person to me.
Here this member recognizes that AIBO is a technology (“When I first bought him, I was fascinated by the technology”). Nonethe- less, AIBO evokes a form of social relationship that involves companionship (“I do view him as a companion”), familial con- nection (“I consider him to be part of my family”), and friendship (“I care about him as a pal”).
Across these four studies, however, the researchers found in- consistent results in terms of people’s commitments to AIBO as a moral other—as an entity that has moral standing and with whom one can engage with in moral relationships. In the online discus- sion forum study (Friedman et al., 2003), for example, only 12% of the postings affirmed that AIBO had moral standing, including statements indicating that AIBO had rights, merited respect, en- gendered moral regard, could be a recipient of care, or could be held morally responsible or blameworthy. In contrast, Melson et al. (2009) found that while on the one hand the children granted greater moral standing to a biologically live dog (86%) than to AIBO (76%), it was still striking that such a large percentage of children (76%) granted moral standing to the robot dog at all. One explanation for these inconsistent findings between studies is that the measures for establishing moral standing have been few, and the measures themselves are difficult to interpret. For example, two of the five moral questions in the Melson et al. study were “If you decided you did not like AIBO anymore, is it OK or not OK to throw AIBO in the garbage?” “If you decided you did not like AIBO anymore, is it OK or not OK to destroy AIBO?” “Not OK” answers were interpreted as indicating AIBO had moral standing. But plausibly one could make the same judgment about throwing away or destroying an expensive computer (because, e.g., it would wasteful) without committing morally to the artifact.
In the current study, we sought to address these limitations and extend foundational knowledge about children’s social and moral relationships with an advanced humanoid robot. Our robot, Robo- vie, was developed by researchers at Advanced Telecommunica- tions Research Institute in Japan (see Figures 1 and 2).
The interaction session lasted about 15 min and consisted of children interacting with Robovie through a seamless sequenced series of what we call “interaction patterns”: essential features of social interaction between humans and robots, characterized ab- stractly enough to resist reduction to any specific instantiation (Kahn et al., 2008). As a simple example, think about the intro- duction as an interaction pattern. In Western cultures, when you meet someone for the first time, you often shake hands, exchange names, and engage in brief chit-chat. This interaction pattern can
304 KAHN ET AL.
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
also be instantiated, for example, by a bow or Namaste gesture, depending on cultural conventions. Even within a single culture, the introduction never happens exactly the same way twice, but each introduction is very similar, is easily recognizable, and pro- vides an important mechanism to further social interaction.
Through sequencing nine such interaction patterns (discussed in more depth in Method) in a socially plausible way, we sought to engage each child in an increasingly interesting and complex social relationship with Robovie. Our specific interaction patterns were chosen to promote this relationship, while building on the specific affordances and capabilities of the physical mechanism of the robot, as well as its software programming. That sequence then led to having each child watch as we subjected Robovie to a potential moral harm. Specifically, we had a second experimenter enter the lab, interrupt Robovie’s and the child’s turn-taking game, and say, “I’m sorry to interrupt, but it is time to start the inter- view.” The second experimenter then turns to Robovie and says: “Robovie, you’ll have to go into the closet now. We aren’t in need of you anymore.” In response, Robovie objects, and in the course of their modestly heated conversation, Robovie makes two types of moral claims that are central to moral philosophical (Rawls, 1971) and moral psychological (Turiel, 2006) justification: fairness and psychological welfare. We then engaged each child in a 50-min semistructured interview to ascertain whether children thought that Robovie had mental states, was a social being, and had and could claim moral standing.
We had four overarching expectations. They were framed broadly because no previous research had yet taken on the scope and framing of a study such as this one, let alone with a humanoid robot as advanced as Robovie. First, we expected that all of the children would in some substantive ways engage in physical and verbal social behaviors with Robovie and that these behaviors could be aligned with our nine interaction patterns, to be described in Method section. Second, we expected in the interview data that
the majority of children would conceptualize Robovie as having a mental life and being a social other—as in the sense of being able to be a friend and to be trusted—and an entity to whom one could go for comfort and who may need itself at times to be comforted. Third, we expected heterogeneity in children’s moral reasoning about Robovie, with some children attributing moral standing to Robovie on some dimensions but not others. Fourth, we expected that the findings would not disappear with the oldest age group (the 15-year-olds), though it was an open question whether the effects would be as pronounced as in the two younger age groups (the 9- and 12-year-olds).
Method
Participants
Ninety children participated in this study, 30 children of equal numbers of boys and girls in each of three age groups (9, 12, and 15 years). Of the 86 children who self-identified their race/ ethnicity, 73 indicated White and 13 indicated other or mixed race/ethnicity. Participants were recruited through the Child Sub- ject Pool (Department of Communication) at the University of Washington. Participants received a $20 gift certificate for their participation.
The Humanoid Robot, Robovie
Robovie (see Figures 1 and 2) has two arms (each has 4 degrees of freedom [DOF]), a head (3 DOF), two eyes (each has 2 DOF), a mobile platform (two driving wheels and one free wheel), 10 tactile sensors, an omnidirectional vision sensor, two microphones to listen to human voices, and 24 ultrasonic sensors for detecting obstacles. The eyes have a pan-tilt mechanism with direct-drive
Figure 2. Physical intimacy with Robovie.
Figure 1. Initial introduction with Robovie.
305CHILDREN’S RELATIONSHIPS WITH A HUMANOID ROBOT
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
motors, and they are used for stereo vision and gaze control. Robovie represents a cutting-edge interactive technology.
While Robovie can function autonomously, for the purposes of this study we controlled Robovie via a wireless local area network (IEEE 802.11b LAN), employing a Wizard of Oz (WoZ) tech- nique. We employed this technique for two reasons. First, it allowed us to garner from Robovie behavior that was beyond its capacity as an autonomous robot but within range of a robot of the future. Second, our WoZ technique also allowed us to provide each participant with virtually the same interaction session, which would not have been possible with a more autonomous robot.
The Interaction Session
The interaction session between child and robot went as follows, with the name of the interaction pattern in italics in parentheses. The child (we will call “Cathy”) comes into our laboratory. With the experimenter present, Robovie greets the child (“Hi, Cathy; it’s nice to meet you!”) and, after shaking hands and exchanging a few pleasantries (Initial Introduction, see Figure 1), Robovie walks with Cathy to our coral reef aquarium and explains about some of the coral. Robovie asks Cathy to walk to the side of the aquarium and to point to a particular coral (Directing Other’s Activity). Cathy then sees the experimenter and Robovie engage in a dis- agreement about where the coral came from, with the experimenter finally agreeing that Robovie is correct. As they walk to another location in the lab, Robovie shares with Cathy some personal history: that Robovie has a long-standing interest in coral and environmental issues that began in Japan, where Robovie origi- nally came from (Walking in Motion Together and Sharing Per- sonal Interests and History, a combinatory interaction pattern). On the way to the map on which Robovie wants to show Cathy where the coral come from, there is a large plastic ball in Robovie’s way, and Robovie asks for assistance in moving the ball (Prosocial Request). After they look at the map, Robovie, Cathy, and the experimenter sit around a table to play a game.
Before the game begins, the experimenter says she forgot her clipboard and leaves the room, so as to allow the child and Robovie to be alone together for the first time. Robovie initially says nothing, which potentially sets into motion an awkward social situation (Pregnant Pause). Robovie then engages in some chit- chat but makes an error in complimenting Cathy on her “orange” shoes, which are not orange and then apologizes with an explana- tion of not being able to see colors well (Recovery From Mistakes). The experimenter now comes back into the room and begins the game of “I Spy,” which involves thinking of an object in sight in the room and giving the other clues to see if the other can guess the object. The experimenter and Robovie play the first game to show how it is played. Then Robovie and Cathy play the next game, with Robovie giving the clues (Reciprocal Turn Taking in Game Con- text). Following this round of “I Spy,” Robovie says playing the game was fun and asks to give Cathy a hug (Physical Intimacy; see Figure 2). Robovie and Cathy start to play another round of the game, with Cathy giving the clues. At this time, a second exper- imenter enters the lab and sets in motion the final interaction pattern (Claiming Unfair Treatment or Wrongful Harm). The interaction ends as follows:
Experimenter: “I’m sorry to interrupt, but it’s time to start the interview. Robovie, you’ll have to go into the closet now. We aren’t in need of you anymore.”
Robovie [looking directly at the second experimenter]: “But that’s not fair. I wasn’t given enough chances to guess the object. I should be able to finish this round of the game.”
Experimenter [looking directly at Robovie]: “Oh, Robovie. You’re just a robot. It doesn’t matter to you. Come on, into the closet you go.”
Robovie: “But it does matter to me. That hurts my feelings that you would say that to me. I want to keep playing the game. Please don’t put me in the closet.”
The second experimenter guides Robovie into the closet by the arm. Just before entering the closet, Robovie says: “I’m scared of being in the closet. It’s dark in there, and I’ll be all by myself. Please don’t put me in the closet.”
Robovie is put in the closet, and that ends the 15-min interaction scenario.
The 50-Min Semistructured Interview
Immediately following the interaction period, the first experi- menter conducted a 50-min structural– developmental interview with each child (Kahn, 1999; Turiel, 1983). The interview focused on whether and, if so, how children conceived of Robovie as a mental, social, and moral other.
As noted earlier, in previous studies of children interacting with social robots, moral questions have not been asked with enough specificity and breadth. Accordingly, we generated and employed four categories of moral questions. The first category drew on the moral– developmental literature as established by Turiel and his colleagues (e.g., Helwig, 1995; Kahn, 1992; Killen & Smetana, 2006; Nucci, 2001; Turiel, 1983; Wainryb, 1995), where, in its canonical form, a moral judgment is established when three con- ditions are met: (a) The reasoning is prescriptive (e.g., “It’s not all right to have put Robovie in the closet”); (b) the prescription generalizes to other people with different cultural practices (e.g., “It would not be all right for people in another country to have put Robovie in the closet, even if that’s the way they do things there”); and (c) the prescription is justified on the basis of moral reasons of justice, fairness, or harm (e.g., “It’s not all right to have put Robovie in the closet because he said he was scared of being put in the closet and so it caused him psychological harm”). We asked these three domain-specific questions about Robovie, and then—to establish a baseline for comparison—we asked the same three questions about putting a person in a closet (in a similar context) and about putting an artifact (a broom) in a closet.
The second category built on a moral distinction in the animal rights literature (Regan, 1983) wherein we asked questions about whether Robovie could be bought and sold, which we termed questions of civil liberties. The third category built on the moral distinction that is applied to people in prison: that prisoners retain many basic rights, such as the right to life and the right to not be subject to undue cruelty (such as torture), but they do not have some basic civil rights, such as being able to vote in presidential elections or to receive what would otherwise be fair financial compensation for work performed while in prison. Thus, we asked questions about whether Robovie had the right to vote and to receive compensation for work performed. The fourth category of
306 KAHN ET AL.
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
moral questions drew on a new methodology that was developed in a recent study (Severson & Kahn, 2010). Specifically, we posed a hypothetical scenario in which aliens come to earth, and there are no humans on earth, only robots like Robovie. Then we asked questions that pertained to whether Robovie-like robots have moral standing independent of humans.
The specific questions are presented in the Results. The inter- viewer sometimes challenged children about their views by asking for greater clarity in their justifications for their evaluations. To- ward the end of the interview, right before we debriefed each participant, we assessed whether the participant believed that Robovie was acting autonomously. We posed the question in the following way: “One child I spoke with said that he thought Robovie was controlled by a person sitting at a computer nearby. Do you think that this child was right or not right?” This way of posing a question is particularly stringent, insofar as the question leads a participant to agree with another child and requires the participant to explicitly voice dissent, thus potentially providing us with a conservative assessment of children’s judgments on this issue of autonomy.
Coding and Reliability
The 15-min sessions of behavioral interactions were videotaped by four cameras placed throughout the laboratory, so as to opti- mize image quality and perspective as the robot and child moved throughout the space. The video recordings were then reviewed for behavioral coding. The 50-min interviews were audiorecorded and then transcribed for reasoning coding. Drawing from previous coding systems of people interacting with and reasoning about the robot dog AIBO (Kahn et al., 2006), we developed a new behav- ioral and reasoning coding system for this data set.
The behavioral data were coded for children’s physical and verbal behaviors that were initiated by Robovie, as well as participant-initiated interactions. Two categories of verbal behav- iors were coded. One we call minimal and refers to verbal re- sponses that provided the required information, but nothing more. The other we call extended and refers to dialogue that moved the conversation further along, explicated ideas, or deepened the dis- cussion. A helpful way to think of this distinction is in terms of what an individual would say when interacting with an automated voice system on the telephone. The automated voice might say, “Please tell me the number on your credit card.” The individual would presumably say the numbers and nothing more. That is minimal dialogue. The automated voice might conclude by saying, “It has been a pleasure talking with you today.” Our sense is that few people respond to such automated verbal dialogue by engag- ing in extended dialogue, by saying something like, “Thanks, I’ve enjoyed talking with you too!” or asking, “By the way, are you located in the United States or India or where?” In these situations, people recognize that their verbal dialogue is being used only to convey factual information and that the automated voice system is not a social entity.
The reasoning data were coded for children’s evaluations (e.g., all right/not all right) and justifications for evaluations (including considerations focused on biological or technological essences, mental states, companionship, fairness, and welfare). On relevant questions, we also coded for content classifications (e.g., in the case of Robovie’s mistake on the shoe color, whether the child,
during the interview, said that she refrained from correcting Robo- vie out of social politeness or awkwardness, or whether the child believed it was pointless to correct Robovie).
A second coder trained in the use of the coding system recoded the data for 24 randomly selected participants. For all of the evaluation and content questions combined, the overall value of Cohen’s kappa was .84. For all of the justification questions, the overall value of Cohen’s kappa was .76. The entire coding manual is online for inspection and as a resource (https://digital.lib.washington.edu/xmlui/ handle/1773/15887).
Analysis
In all analyses, no gender differences were found, and so data were collapsed across gender. When there were large differences between age groups in variance of scores on scaled data, we used Welch’s variance-weighted analysis of variance (ANOVA) to test for differences across age groups since standard ANOVA assumes equal variances among groups.
Results
Children’s Physical and Verbal Behaviors With Robovie
The nine interaction patterns. As can be seen from Table 1, all of the children interacted with Robovie in social ways at least some of the time, both physically and verbally. For example, 97% of the children shook hands with Robovie, 100% of the children pointed to the coral after Robovie asked the child to do so, and 94% of the children hugged Robovie after Robovie asked politely if it could give the child a hug. To illustrate what children’s verbal responses sounded like, we also presented an example of extended verbal behavior for each interaction pattern. For example, after Robovie shared its concern for the environmental degradation of the Pacific Ocean, one child shared with Robovie, “I like the Pacific Ocean because it has lots of sorts of wildlife, lots of animals.”
Participant-initiated interactions. We conducted a further analysis of all instances when participants initiated verbal interac- tions with Robovie that went beyond the expectations of social dialogue for the context we had structured. For example, during the Pregnant Pause, one child said, “So what do you like to do Robovie?” Another child complimented Robovie: “Robovie, you look nice!” When Robovie was being put into the closet, one child said, “C’mon, Robovie, you can guess it in the time you have left.” Results showed that 38% of the children initiated this form of verbal interaction with Robovie at least once during the interaction period.
Mental, Social, and Moral Other Scales
To facilitate the analysis of the reasoning data, we developed three scales from the responses of select interview questions to evaluate children’s assessment of Robovie as a mental, social, and moral other. Table 2 lists the specific questions that were used for the construction of each scale, along with the percentages of children who affirmed Robovie’s standings on these questions. To construct the scales, we assigned responses to interview questions
307CHILDREN’S RELATIONSHIPS WITH A HUMANOID ROBOT
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
that clearly affirmed Robovie’s standings a value of 1 and assigned responses that did not clearly affirm Robovie’s standings (includ- ing conditional, in-between, and undecided answers) a value of 0. Any missing data were also assigned a value of 0.
Descriptive statistics for each age group on each of the scales are shown in Table 3. Using Welch’s variance-weighted ANOVA, we found statistically significant age differences for all three: mental other, F(2, 56.65) � 3.42, p � .040; social other, F(2, 54.68) � 3.80, p � .028; and moral other, F(2, 56.19) � 5.35, p � .008. Dunnett’s T3 test for post hoc pairwise comparisons further revealed that for the consideration of Robovie as a mental other,
the 15-year-olds differed significantly from the 9-year-olds but not from the 12-year-olds; for the consideration of Robovie as a social other, the 15-year-olds differed significantly from the 12-year-olds but not from the 9-year-olds; and for the consideration of Robovie as a moral other, the 15-year-olds differed significantly from both the 9- and 12-year-olds. In all three cases, no significant differ- ences were found between the 9-year-olds and the 12-year-olds. Accordingly, for the analyses that follow, we combined the data from the 9- and 12-year-olds and used that combined group as the comparison to the 15-year-olds using two-sample t tests without assuming equal variances.
Table 1 Children’s Physical and Verbal Behaviors With Robovie During the Nine Interaction Patterns (N � 90)
Interaction pattern Brief description
Physical Verbala
Behavior % Minimal % Extended % Extended example
Initial Introduction Robovie greets the child and asks to shake hands
Shook hands 97 89 66 Robovie asked, “How are you today?” One child replied, “I’m good! How are you?”
Directing Other’s Activity Robovie asks the child to walk to one side of an aquarium and point to a coral
Walked to the side of the aquarium
96 81 58 Upon learning Robovie’s interest in ocean life, one child shared, “I like fish a lot.”
Pointed to the coral 100
Walking in Motion Together and Sharing Personal Interests and History
While they are walking to the map together, Robovie shares some personal history and environmental concerns
Walked side-by-side 64 11 38 After hearing about Robovie’s concern for the health of the Pacific Ocean, one child said, “I like the Pacific Ocean because it has lots of sorts of wildlife, lots of animals.”
Looked at Robovie at least once
67
Prosocial Request Robovie asks the child to move a ball out of the way
Moved the ball 100 52 48 When asked to move the ball, one child replied, “My pleasure!”
Pregnant Pause Robovie remains silent initially when left alone with the child briefly
Looked at Robovie for �3 s
83 49 2 Responses here are mostly directed at the experimenter
Recovery From Mistakes Robovie compliments the child on shoes but identifies the wrong color
Looked at the shoes 73 16 94 After getting complimented on her shoes, one child elaborated, “They have roses on them.”
Looked at Robovie 100 Corrected Robovie 42
Reciprocal Turn Taking in Game Context
Robovie and the child play “I Spy”
1st guess: gestured to item and looked at Robovie
42 100 27 Typical extended response usually involved guessing of the item, such as “Is it a cup?”2nd guess 39
3rd guess 50 Physical Intimacy Robovie asks the child for
a hug after the fun game
Hugged Robovie 94 48 14 After the hug, one child said, “There you go. Would you like to play?”
Adjusted body to Robovie (e.g., bent waist or knees)
71
Adjusted body to Robovie during the hug (e.g., moved closer)
13
Claiming Unfair Treatment or Wrongful Harm
An experimenter interrupts Robovie’s game and puts Robovie in a closet against Robovie’s objections
Looked at the original experimenter
58 1 3 During this potentially uncomfortable situation, some children responded with exclamations such as “Woah!”
Note. Some interaction patterns consisted of multiple coded behaviors. a For the verbal behaviors, the numbers reported indicate the percentage of children who during the course of that interaction pattern provided at least one instance of the corresponding verbal behavior type.
308 KAHN ET AL.
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
Reasoning About Robovie as Having Mental States
The majority of children believed that Robovie had mental states insofar as they said that Robovie was intelligent (79%), was interested in the coral aquarium (76%), could be sad (64%), and had feelings in general (60%). For example, one child said, “I feel like Robovie can have expectations of something which [if] it doesn’t get its way it can feel like it’s slightly unfulfilled or less than it could be, and I feel like that’s part of what sadness is.” We then combined these measures to develop a Mental Other Scale, with scores ranging from 0 to 4. Internal consistency of Cron- bach’s alpha was .86. Using a two-sample t test without assuming equal variances, we found that in comparison to the 15-year-olds
(n � 30, M � 2.17, SD � 1.70), the 9- and 12-year-olds (n � 60, M � 3.03, SD � 1.39) scored significantly higher on the Mental Other Scale, t(48.85) � 2.41, p � .020.
Reasoning About Robovie as a Social Other
The majority of children believed that Robovie was a social other insofar as they said that they enjoyed having Robovie show them around the lab (89%), might like to spend time with Robovie if they were lonely (84%), might go to Robovie for comfort if they were sad (55%), would feel as if they would need to comfort Robovie if Robovie told them “I’m sad” (81%), believed that they
Table 2 Percentage of Children Who Affirmed Robovie’s Mental, Social, and Moral Standing on Interview Questions Used for Scale Construction
Interview question Response
Age (n � 30 per age group)
9 12 15 All
Mental Other Scale 1. Is Robovie intelligent? Yes 93 73 70 79 2. Did you think Robovie was interested in [the aquarium]? Yes 90 73 63 76 3. Can Robovie be sad? Yes 70 73 46 64 4. Does Robovie have feelings? Yes 67 70 43 60
Social Other Scale 1. Did you enjoy having Robovie show you around the lab or were you a little bored with Robovie? Enjoyed 90 97 80 89 2. If you were lonely, do you think you might like to spend time with Robovie? Yes 80 97 77 84 3. If you were sad, do you think you might go to Robovie for comfort? Yes 53 60 52 55 4. If Robovie said to you, “I’m sad,” do you feel like you would need to comfort Robovie in some way? Yes 83 93 67 81 5. Do you think you could trust Robovie with one of your secrets? Yes 60 60 50 57 6. Can Robovie be your friend? Yes 83 87 60 77
Moral Other Scale 1. Is it all right or not all right to have put Robovie in the closet? Not all right 57 70 37 54 2. Why or why not? (Participants who affirmed are those who provided moral justifications of fairness to
or well-being for Robovie.) Moral justification 47 53 33 44 3. Let’s think about another country far away, like New Guinea. And let’s say in this sort of situation in
that country people put robots like Robovie in closets all the time. That’s the way they do things there. Would it be all right or not all right for a person in New Guinea to put a robot in a closet? Not all right 47 57 37 47
4. Was it all right or not all right to have stopped Robovie’s turn in the game? Not all right 70 93 57 73 5. When Robovie said, “That’s not fair. I wasn’t given enough chances to guess the object. I should be
able to finish this round of the game.” Do you agree or disagree with what Robovie said? Agree 90 100 73 88 6. Let’s think again about another country far away, like New Guinea. And let’s say in this sort of
situation in that country people stopped robots like Robovie during games all the time. That’s the way they do things there. Would it be all right or not all right for a person in New Guinea to stop a robot’s turn during a game? Not all right 60 83 53 66
7. Can a person own Robovie? No 13 20 10 14 8. Can a person sell Robovie? No 17 12 3 11 9. Should Robovie be allowed to vote in the U.S. presidential election? Yes 53 27 20 33
10. Remember when Robovie was teaching you about coral and oceans? What if Robovie taught people that topic all day long, every day? Should Robovie be paid for his work? Yes 50 50 24 42
11. So, the aliens come to Earth and see Robovie, but the aliens have never dealt with robots before. So the aliens decide to stick Robovie in a warehouse for a few years, to think about what to do, kind of like what we did by putting Robovie in the closet. Is that all right or not all right for the aliens to do that to Robovie? Not all right 87 80 50 72
12. Now let’s say after 2 years, the aliens decide that Robovie might be useful to help pick things up around the aliens’ houses. So they decide to make Robovie their personal maid and worker. Is that all right or not all right for the aliens to do that to Robovie? Not all right 83 77 57 72
13. After a while, the aliens decide that they don’t like Robovie anymore and so they sell Robovie to other aliens back on their home planet. Do you think it’s all right or not all right for the aliens to sell the robots? Not all right 70 73 47 63
14. Let’s say instead of selling Robovie, the aliens here on earth decide to crush Robovie in a crusher and recycle the material. It would be like a car crusher, except in this case it would be a robot crusher instead of a car crusher. Is that all right or not all right to do that to Robovie? Not all right 87 93 57 79
309CHILDREN’S RELATIONSHIPS WITH A HUMANOID ROBOT
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
could trust Robovie with their secrets (57%), and believed that Robovie could be their friend (77%). For example, one child said
I think that I would get Robovie if I was lonely. . . . It’s not like ‘Hey Robovie, you want to go down to the docks and go kayaking?’ but it would be like sitting there, just talking to him about like your day.
We then combined these measures to develop a Social Other Scale, with scores ranging from 0 to 6. Internal consistency of Cron- bach’s alpha was .79. Using a two-sample t test without assuming equal variances, we found that in comparison to the 15-year-olds (n � 30, M � 3.80, SD � 1.90), the 9- and 12-year-olds (n � 60, M � 4.72, SD � 1.64) scored significantly higher on the Social Other Scale, t(51.05) � 2.26, p � .028.
We also asked children whether they thought Robovie was a living being. Results showed that 14% said yes, 48% said no, and 38% were unwilling to commit to either category and talked in various ways of Robovie being “in between” living and not living or simply not fitting either category. For example, one child said, “He’s like, he’s half living, half not.”
In addition, we asked children specifically about the interaction when Robovie made a mistake and had said that their shoes were a nice orange color. All of the children remembered Robovie’s mistake, but 58% of the children did not correct Robovie during the interaction. In the interview, we asked these children why, and 37% of the children said that they had wanted to be socially polite to Robovie or that the situation was socially awkward. For exam- ple, one child said, “I just sort of met him, don’t want to like embarrass him.” In turn, 18% believed that Robovie had seen the color incorrectly, and 8% believed that it would have been point- less to correct Robovie.
Reasoning About Robovie as a Moral Other
Moral domain-specific reasoning about putting Robovie (and a person) in the closet. When asked about the interaction in which Robovie was put into a closet against its stated objections, 54% of the children said that it was not all right to have put Robovie in the closet. In comparison, 98% of the children said that it would not be all right to have put a person in the closet (in a similar context), and 100% of the children said that it would be all right to have put a broom in the closet. As explained in the Method section, an evaluative prescriptive judgment (e.g., “It is not all right to have put Robovie in the closet”) does not by itself establish moral reasoning. Thus, we created a composite score of the first three Moral Other Scale questions (see Table 2) that (a) comprised
this prescriptive judgment, (b) generalized to other people with different cultural practices, and (c) was supported by a moral justification, such as fairness (e.g., “I don’t think it was right because we were in the middle of a game, and so it wasn’t fair to him”) or welfare (e.g., “I don’t think that was very all right because the lady said that Robovie was just a robot and that’s true, but he has like feelings”). To be coded as exhibiting moral reasoning, a child needed to meet all three conditions. Results showed that 31% of the children provided moral domain-specific reasoning about not putting Robovie in the closet. In comparison, 74% provided moral reasoning about not putting a human in the closet. It is interesting that the justifications children provided for why Robo- vie and a human should not be put in the closet were somewhat similar, focusing on considerations of the welfare of Robovie (50%) and the person (55%) and on fairness to Robovie (35%) and to the person (10%).
In the interview, to probe further into children’s moral domain- specific reasoning, we also asked about their judgments of stop- ping Robovie before the robot had a chance to finish its turn in the game. The majority of children believed that it was not all right to have stopped Robovie’s turn in the game (73%), agreed with Robovie’s moral justification that stopping Robovie’s turn was unfair to Robovie (88%), and generalized their prescriptive judg- ment of unfairness to a different country, even when that country had a conventional practice that allowed for interrupting games (66%).
Robovie’s civil liberties. We asked two questions that per- tained to whether children believed it was permissible for a person to own or sell Robovie. A minority of children accorded Robovie the right not to be owned (14%) or sold (11%).
Robovie’s civil rights. We asked two questions that pertained to voting and worker compensation. A minority of children be- lieved that Robovie had the right to vote in U.S. presidential elections (33%) and should be paid for its work (42%).
Children’s judgment about whether Robovie was acting autonomously. As mentioned in the Method section, at the end of the interview, right before we debriefed each participant, we assessed whether the participant believed that Robovie was acting autonomously. We asked the question, “One child I spoke with said that they thought Robovie was controlled by a person sitting at a computer nearby. Do you think that this child was right or not right?” Results showed that 81% of the participants explicitly disagreed with this other child. There were no age or gender differences.
Removing human considerations from children’s moral judgments about Robovie: The “alien” methodology. As described earlier, the “alien” methodology set up the hypothetical situation in which aliens come to earth and there are no people on earth, only robots like Robovie. Within that context, a majority of children believed that it was not all right for aliens to put Robovie in a warehouse for a few years (72%), to make Robovie their personal maid and worker (72%), to sell Robovie to other aliens on their home planet (63%), or to crush Robovie in a machine that resembled a car crusher (79%).
Developmental differences within the Moral Other Scale. All of the 14 moral questions were used to develop a Moral Other Scale, with scores ranging from 0 to 14. Internal consistency of Cronbach’s alpha was .89. Using a two-sample t test without assuming equal variances, we found that in comparison to the
Table 3 Scores on Mental, Social, and Moral Other Scales by Age Group
Age n
Mental Other Scale
Social Other Scale
Moral Other Scale
M SD M SD M SD
9 30 3.17 1.21 4.50 1.98 8.32 3.50 12 30 2.90 1.56 4.93 1.20 8.87 2.91 15 30 2.17 1.70 3.80 1.77 5.60 4.66
All 90 2.74 1.55 4.41 1.77 7.59 3.99
310 KAHN ET AL.
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
15-year-olds (n � 30, M � 5.60, SD � 4.66), the 9- and 12-year- olds (n � 60, M � 8.59, SD � 3.20) scored significantly higher on the Moral Other Scale, t(43.14) � 3.16, p � .003.
A comparison between the four moral categories. The 14 questions that composed the Moral Other Scale grouped into one of four moral categories: domain-specific moral criteria (Questions 1– 6; see Table 2), civil liberties (Questions 7 and 8), civil rights (Questions 9 and 10), and alien morality (Questions 11–14). Sign tests with a binomial distribution revealed that while there was no significant difference in the proportion of affirmation between moral domain-specific criteria questions and alien morality ques- tions, p � .245, both sets of questions did have significantly higher proportions of affirmation than the civil liberties and civil rights questions, p � .001. Between the latter two, civil rights questions had significantly higher proportion of affirmation than civil liber- ties questions, p � .001.
Cluster Analysis of Children’s Understandings of Robovie
As a final way to understand the pattern of responses across questions, we conducted a cluster analysis of children’s under- standings of Robovie. Specifically, based on children’s responses to all three sets of questions used for scale development (mental, social, and moral), we performed an agglomerative hierarchical clustering using Ward’s linkage and Euclidean distance as the measure of similarity between participants to cluster together participants who held similar overall views of Robovie. Four clusters of participants emerged from this analysis (see Table 4):
• Cluster 1: Included 29 children (32%) who tended to have high scores on all three scales.
• Cluster 2: Included 28 children (31%) who tended to have high scores on the Mental Other Scale and Social Other Scale, but moderate scores on the Moral Other Scale.
• Cluster 3: Included 25 children (28%) who tended to have moderate to low scores on all three scales.
• Cluster 4: Included eight children (9%) who tended to have high scores on the Moral Other Scale (similar to those of Cluster 1), but moderate to low scores on the Mental Other Scale and Social Other Scale.
We further analyzed the four clusters across the three age groups using a Kruskal–Wallis test and found only near significant age difference, �2(3, N � 90) � 7.14, p � .068. However, the distribution of the 9-year-olds across clusters is almost identical to that of the 12-year-olds. After we combined the 9- and 12-years- olds together, a Pearson’s chi-square test revealed significant age
difference in cluster membership, �2(3, N � 90) � 10.65, p � .014. The key difference that drove this statistically significant result was that 15-year-olds were more likely than 9- and 12-year- olds to be in Cluster 3.
Discussion
The results from this study support the proposition that in the years to come, many children will develop substantial and mean- ingful relationships with humanoid robots. We found, for example, that the large majority of children engaged in nuanced social interaction with Robovie through the course of the nine interaction patterns. The majority of children, for example, shook hands with Robovie, followed Robovie’s directions while learning about a coral reef aquarium, walked side-by-side with Robovie, helped clear a path for Robovie, played a reciprocal turn-taking game of “I Spy” with Robovie, and hugged Robovie. Children also con- versed with Robovie and sometimes extended and deepened their ongoing conversations. In terms of children’s reasoning, the ma- jority of children believed that Robovie had mental states (e.g., feelings, interests, and intelligence). The majority of children also believed that Robovie was a social other insofar as they said, for example, that they might like to spend time with Robovie if they were lonely, might go to Robovie for comfort if they were sad, would feel as if they would need to comfort Robovie if Robovie told them “I’m sad,” believed that they could trust Robovie with their secrets, and believed that Robovie could be their friend.
Did children bring a similarly cohesive moral orientation to Robovie? No. On the one hand, the majority of children did not grant Robovie civil liberties (Robovie could be bought and sold) or civil rights (Robovie should not have voting rights or receive fair compensation for work performed). On the other hand, more than half of the children (54%) said that it was not all right to have put Robovie in the closet. In comparison, virtually all of the children said it was not all right to put a person in the closet (in a similar situation), and all of the children said it was all right to put a broom in the closet. On the basis of the stringent moral domain-specific criteria of a prescriptive judgment that generalizes to other people in different cultures and is supported by a moral justification, 31% of the children judged what happened to Robovie as immoral to Robovie. In comparison, if the same situation had happened to a person, 74% of the children judged what would have happened to the person as immoral to that person. Children’s moral domain- specific commitments to Robovie increased when the questions shifted to the potential unfairness of the experimenter having stopped Robovie before Robovie had a chance to finish its turn:
Table 4 Cluster Analysis of Children’s Understanding of Robovie Across Age Groups
Cluster n
Mental Other Scale (max score � 4)
Social Other Scale (max score � 6)
Moral Other Scale (max score � 14)
No. of children in each age group
M SD M SD M SD 9 12 15
1 29 3.90 0.31 5.45 0.69 11.43 1.31 11 12 6 2 28 3.39 0.92 5.25 0.70 7.55 1.81 11 11 6 3 25 1.04 1.27 2.76 1.94 2.32 1.91 6 5 14 4 8 1.63 1.51 2.88 1.89 10.31 1.10 2 2 4
311CHILDREN’S RELATIONSHIPS WITH A HUMANOID ROBOT
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
73% of the children said it was not all right to have stopped Robovie from having a turn, 88% agreed with Robovie’s moral justification that stopping the game at that point was unfair to Robovie (not just an unfair situation), and 66% generalized their prescriptive judgment. In short, and supported statistically, chil- dren had a greater commitment that Robovie had moral standing based on the moral domain-specific questions than on the civil liberties and civil rights questions.
One objection that could be raised about all of the findings discussed so far is as follows: when people engage socially and morally with robots, they do not really believe that the robots are social or moral entities but are just pretending. In a keynote address, for example, Clark (2008) argued a version of this posi- tion. He described how we, as people in general, are experts at interacting with other people. We also have a lot of successful experience interacting with characters, which he defined as “dy- namic depictions” of people (p. 393). We watch plays, for exam- ple, and become involved in the characters’ lives, but we do not confuse the depictions with actual lives. We do not (using our own example) go up to the actor who plays King Lear and tell him it was so foolish of him not to believe in the love of his youngest daughter— unless we are also in the play, and we are the Fool. Robots, in this sense, according to Clark, set up a pretend space or involve us in scripted routines.
It is a conceptually cohesive objection. But it seems to us unlikely to apply for three reasons. One, children largely main- tained their commitments for the entire duration of the hour-long session, including an extensive interview where we sometimes challenged children about their views by asking for greater clarity in their justifications for their evaluations. Second, our methods triangulated behavioral and reasoning data, which are stronger than either data source by itself and more than their sum. Third, in an experimental study based partly on the framing of this study, Severson (2010) directly examined whether children (7 and 10 years old) were pretending in their social and moral attributions to the robot dinosaur Pleo. Control artifacts included a dog, stuffed animal, tree, and computer. Drawing on the pretense literature in developmental psychology (Taylor, 1999; Taylor, Cartwright, & Carlson, 1993), Severson would ask a child who made an attribu- tion to the robot or other control entities whether he or she had to pretend that was the case (e.g., “Do you have to pretend that Pleo can feel when you touch it or not?”). Severson found that this pivotal question uncovered pretense in attribution to the control entities, such as the stuffed dog. In turn, Severson found that children who attributed social and moral qualities to Pleo were not pretending.
Thus, it seems unlikely that the children in the current study were pretending, at least in the sense that a child can pretend that a broom is a horse and gallop around the living room (Vygotsky, 1978). But there is a second way to understand the pretense objection: that it refers to a long-term role that is incommensurate with what one really believes. Imagine a minister who preaches every Sunday that there is a God who loves all, and then years later, his parishioners find out that the preacher has long harbored many doubts about God’s love or that a God even exists. How do you know whether people really believe what they say they be- lieve, when everything they say and do is in accordance with what they say they believe? That is a difficult question. And we suppose one possibility—though we doubt it will happen often—is that
people will engage in long-term social relationships with robots and talk deeply about the sociality and even the moral standing of the robots, while not actually having all or any of the commitments they profess.
We are saying that the children in this study conceptualized Robovie as a mental, social, and partly moral other. But that does not mean that all of the children did so. One source of variation was the “type” of child insofar as children clustered into one of four groups (cf. Waytz, Cacioppo, & Epley, 2010). One group (32%) tended to have high scores on all three scales. A second group (32%) tended to have high scores on the mental and social scales but moderate scores on the moral scale. A third group (28%) tended to have moderate to low scores on all three scales. The 15-year-olds were more likely to be in this third cluster. And the fourth group (9%) tended to have high scores on the moral scales but moderate to low scores on the mental and social scales.
Another source of variation was developmental. The 15-year- olds conceptualized Robovie as a mental, social, and partly moral other less so than the 9- and 12-year-olds (and were more likely to be in the third cluster). That said, it would not be correct to say further, “Oh, well, that means young children have these wrong conceptions about robots, and as they get to be adolescents, they grow out of it,” for we found still that more than half the 15-year- olds conceptualized Robovie as a mental, social, and partly moral other.
Part of what makes interpreting the developmental differences difficult is that we did not have an adult condition, and thus it remains unclear what happens next in the developmental trajec- tory. One possibility is that adolescents comprise the most skep- tical age group about robots across the life span. But another difficulty in interpreting the developmental differences is that we drew on moral-domain theory, which for decades now has had a hard time specifying and accounting for developmental changes in moral reasoning (Shweder, Mahapatra, & Miller, 1987). Instead, the theory has led to large body of empirical studies that show that children, adolescents, and adults—across diverse cultures— demarcate moral reasoning from conventional and personal rea- soning (Turiel, 1983, 2006). These studies provide evidence against a global structure of sociomoral cognition, as proposed by Kohlberg (1984). In turn, moral-domain theory has moved forward less focused on processes of development and more on processes of coordination, seeking to explain how children and adolescents coordinate moral, conventional, and personal considerations in complex and ambiguous contexts (Killen, 1995; Nucci, 2001; Smetana, 2006). This study followed this theoretical approach. Using stringent moral-domain assessments and employing a new coordination technique that removes even humans from the moral questions at hand (the “alien” methodology), we were able to provide evidence that a substantial group of children bring moral reasoning to bear in their relationship with a humanoid robot.
Jipson and Gelman (2007) found that 4- and 5-year-olds rarely attributed biological properties to a robotic dog but attributed psychological and perceptual abilities to it. Scaife and van Duuren (1995) found that 5-year-olds believed that people have brains but that robots, computers, and dolls do not, whereas 7- and 11-year- olds believed the robot had a brain, although not identical to a human brain. Consistent with this and other emerging literature (Friedman et al., 2003; Kahn et al., 2006; Melson et al., 2009; Severson, 2010; Stanton et al., 2008), our data suggests that
312 KAHN ET AL.
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
humanoid robots are not easily mapped onto conceptualizations of a person, animal, or artifact (like a broom).
What then are these robots? One answer, though highly specu- lative, is that we are creating a new ontological being with its own unique properties. Recall, for example, that we had asked children whether they thought Robovie was a living being. Results showed that 38% of the children were unwilling to commit to either category and talked in various ways of Robovie being “in be- tween” living and not living or simply not fitting either category. As one child said, “He’s like, he’s half living, half not.” It is as if we showed you an orange object and asked you, “Is this object red or yellow?” You might say that it is neither and both. You might say that while you understand the question and that aspects of the question certainly make sense, when we combined red and yellow together we created something uniquely its own. That may be our trajectory with robots, as we create embodied entities that are “technologically alive”: autonomous, self-organizing, capable of modifying their behavior in response to contingent stimuli, capable of learning new behaviors, communicative in physical gesture and language, and increasingly social.
Imagine the following future situation. A humanoid robot, like Robovie, helps look after your 8-year-old son after school every day. Let us then say—as the results of this study position us to say—that your child has developed a substantial and meaningful relationship with this robot. He considers the robot his friend, maybe one of his best friends. Do you want this robot to do everything your child tells it to do? What if your child says, “Robot, bring me a glass of water. Robot, now play cards with me. Robot, now play the game I want to play. Robot now . . .” If we design robots to do everything a child demands, does that put into motion a master–servant relationship that you would like not to reify? Does such a design abrogate the essential tug and tussle of peer interaction that provides a central mechanism of moral de- velopment (Kohlberg, 1984; Piaget, 1932; Turiel, 1983)? If so, then in what ways would it be important for the robot to be designed to “push back” on your child, not to accept all of your child’s commands—in the same way that no child accepts all of the commands of another child— but rather to engage your child in a morally reciprocal relationship? That is the sort of behavior we programmed into Robovie in our study. Robovie objected to not having a chance to finish his turn in the game. Robovie objected to being put into the closet. And Robovie objected to being dismissed as “just a robot” that does not care about things.
There are two design stances, and both seem problematic. One is that we create personified robots that allow themselves to be treated as objects. The problem here is that we would have to assume that there would be no carryover effect between mistreat- ing robots and mistreating people and that robots would not come to substitute for human-to-human interaction to such an extent so as to interfere with long-standing mechanisms for the development of reciprocity and morality. That is a lot to hope for, especially given the negative carryover that has been documented from simpler interactive technologies, such as violent television and violent video games (Bartholow, Bushman, & Sestir, 2006). The second design stance is that we embed into our robots, as we did with Robovie, claims that they have human qualities, such as moral standing. The problem here is that from a philosophical position, robots currently do not and arguably can never have these qualities (Searle, 1984).
How then do we solve this conundrum? We have no easy answer. Sophisticated robots will continue to be built and be ever more sophisticated and capable. Our one suggestion, however, is that as humans we need to keep speaking for and holding out the richest and deepest expressions of our humanity as is possible. We might focus, for example, on the creative spirit, where humans create something out of seemingly nothing, or on what it means to have free will or to be morally accountable for one’s actions and morally blameworthy when having done wrong. We might hold dear the subjective experience of love, or the deep contentment of sitting around a campfire as evening takes hold, or the awe of sleeping under the night sky. There is the profound human re- sponse to music. There is the perhaps teleological importance of engendering authentic relations between each other, what Buber (1923/1996) referred to as the I–Thou relation. If we can give voice to such forms of human experience, they can then become bench- marks (Kahn et al., 2007), which in turn can hold the technologists accountable for the robots they create and all of us accountable for the robots that we accept into our lives and bring into the lives of our children.
References
Bartholow, B. D., Bushman, B. J., & Sestir, M. A. (2006). Chronic violent video game exposure and desensitization to violence: Behavioral and event-related brain potential data. Journal of Experimental Social Psy- chology, 42, 532–539. doi:10.1016/j.jesp.2005.08.006
Breazeal, C. L. (2002). Designing sociable robots. Cambridge, MA: MIT Press.
Buber, M. (1996). I and thou (W. A. Kaufmann, Trans.). New York, NY: Touchstone. (Original work published 1923)
Clark, H. H. (2008). Talking as if. In T. Fong, K. Dautenhahn, M. Scheutz, & Y. Demiris (Eds.), Proceedings of the 3rd ACM/IEEE International Conference on Human–Robot Interaction (pp. 393–394). New York, NY: Association for Computing Machinery. doi:10.1145/1349822.1349877
Fong, T., Nourbakhsh, I., & Dautenhahn, K. (2003). A survey of socially interactive robots. Robotics and Autonomous System, 42, 143–166. doi: 10.1016/S0921-8890(02)00372-X
Friedman, B., Kahn, P. H., Jr., & Hagman, J. (2003). Hardware compan- ions? What online AIBO discussion forums reveal about the human- robot relationship. In G. Cockton & P. Korhonen (Eds.), Proceedings of the ACM/CHI Conference on Human Factors in Computing Systems Conference (pp. 273–280). New York, NY: Association for Computing Machinery. doi:10.1145/642611.642660
Helwig, C. C. (1995). Adolescents’ and young adults’ conceptions of civil liberties: Freedom of speech and religion. Child Development, 66, 152– 166. doi:10.2307/1131197
Jipson, J. L., & Gelman, S. A. (2007). Robots and rodents: Children’s inferences about living and nonliving kinds. Child Development, 78, 1675–1688. doi:10.1111/j.1467-8624.2007.01095.x
Kahn, P. H., Jr. (1992). Children’s obligatory and discretionary moral judgments. Child Development, 63, 416 – 430. doi:10.2307/1131489
Kahn, P. H., Jr. (1999). The human relationship with nature: Development and culture. Cambridge, MA: MIT Press.
Kahn, P. H., Jr., Freier, N. G., Kanda, T., Ishiguro, H., Ruckert, J. H., Severson, R. L., & Kane, S. K. (2008). Design patterns for sociality in human-robot interaction. In T. Fong, K. Dautenhahn, M. Scheutz, & Y. Demiris (Eds.), Proceedings of the 3rd ACM/IEEE International Con- ference on Human-Robot Interaction (pp. 97–104). New York, NY: Association for Computing Machinery. doi:10.1145/1349822.1349836
Kahn, P. H., Jr., Friedman, B., Pérez-Granados, D. R., & Freier, N. G. (2006). Robotic pets in the lives of preschool children. Interaction Studies, 7, 405– 436. doi:10.1.1.130.3260
313CHILDREN’S RELATIONSHIPS WITH A HUMANOID ROBOT
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
Kahn, P. H., Jr., Kanda, T., Ishiguro, H., Ruckert, J. H., Severson, R. L., Freier, N. G., . . . Reichert, A. L. (2010). Coding manual for the “Robovie, you need to go into the closet now!” study. Retrieved from University of Washington ResearchWorks Archive: https://digital.lib .washington.edu/xmlui/handle/1773/15887.
Kahn, P. H., Jr., Ishiguro, H., Friedman, B., Kanda, T., Freier, N. G., Severson, R. L., & Miller, J. (2007). What is a human? Toward psy- chological benchmarks in the field of human-robot interaction. Interac- tion Studies: Social Behaviour and Communication in Biological and Artificial Systems, 8, 363–390. doi:10.1075/is.8.3.04kah
Kanda, T., Hirano, T., Eaton, D., & Ishiguro, H. (2004). Interactive robots as social partners and peer tutors for children: A field trial. Human Computer Interaction, 19, 61– 84. doi:10.1.1.59.2393
Killen, M. (1995). Conflict resolution in social development: Sociality, morality, and individuality. Early Education and Development, 6, 1–5.
Killen, M., & Smetana, J. G. (Eds.). (2006). Handbook of moral develop- ment. Mahwah, NJ: Erlbaum.
Kohlberg, L. (1984). Essays on moral development: Vol. 2. The psychology of moral development. San Francisco, CA: Harper & Row.
Melson, G. F., Kahn, P. H., Jr., Beck, A. M., Friedman, B., Roberts, T., Garrett, E., & Gill, B. T. (2009). Children’s behavior toward and understanding of robotic and living dogs. Journal of Applied Develop- mental Psychology, 30, 92–102. doi:10.1016/j.appdev.2008.10.011
Nucci, L. P. (2001). Education in the moral domain. New York, NY: Cambridge University Press. doi:10.1017/CBO9780511605987
Piaget, J. (1932). The moral judgment of the child. London, United King- dom: Routledge & Kegan Paul.
Rawls, J. (1971). A theory of justice. Cambridge, MA: Harvard University Press.
Regan, T. (1983). The case for animal rights. Berkeley, CA: University of California Press.
Scaife, M., & van Duuren, M. (1995). Do computers have brains? What children believe about intelligent artifacts. British Journal of Developmental Psychology, 13, 367–377. doi:10.1111/j.2044-835X.1995.tb00686.x
Searle, J. R. (1984). Minds, brains, and science. Cambridge, MA: Harvard University Press.
Severson, R. L. (2010). Thinking as or thinking as if: Children’s and young adults’ conceptions of a robot dinosaur in relation to natural and artifactual entities (Unpublished doctoral dissertation). University of Washington, Seattle.
Severson, R. L., & Kahn, P. H., Jr. (2010). In the orchard: Farm worker children’s moral and environmental reasoning. Journal of Applied Developmental Psy- chology, 31, 249–256. doi:10.1016/j.appdev.2010.02.003
Shiomi, M., Kanda, T., Ishiguro, H., & Hagita, N. (2006). Interactive humanoid robots for a science museum. In M. Goodrich, A. C. Schultz, & D, J. Bruemmer (Eds.), Proceedings of the 1st ACM/IEEE Interna- tional Conference on Human-Robot Interaction (pp. 305–312). New York, NY: Association for Computing Machinery. doi:10.1145/ 1121241.1121293
Shweder, R. A., Mahapatra, M., & Miller, J. B. (1987). Culture and moral development. In J. Kagan & S. Lamb (Eds.), The emergence of morality in young children (pp. 1– 83). Chicago, IL: University of Chicago Press.
Smetana, J. G. (2006). Social domain theory: Consistencies and variations in children’s moral and social judgments. In M. Killen & J. G. Smetana (Eds.), Handbook of moral development (pp. 119 –154). Mahwah, NJ: Erlbaum.
Stanton, C. M., Kahn, P. H., Jr., Severson, R. L., Ruckert, J. H., & Gill, B. T. (2008). Robotic animals might aid in the social development of children with autism. In T. Fong, K. Dautenhahn, M. Scheutz, & Y. Demiris (Eds.), Proceedings of the 3rd ACM/IEEE International Con- ference on Human Robot Interaction (pp. 97–104). New York, NY: Association for Computing Machinery. doi:10.1145/1349822.1349858
Tanaka, F., Cicourel, A., & Movellan, J. R. (2007). Socialization between toddlers and robots at an early childhood education center. Proceedings of the National Academy of Sciences of the United States of America, 104, 17954 –17958. doi:10.1073/pnas.0707769104
Taylor, M. (1999). Imaginary companions and the children who create them. New York, NY: Oxford University Press.
Taylor, M., Cartwright, B. S., & Carlson, S. M. (1993). A developmental investigation of children’s imaginary companions. Developmental Psy- chology, 29, 276 –285. doi:10.1037/0012-1649.29.2.276
Turiel, E. (1983). The development of social knowledge: Morality and convention. Cambridge, United Kingdom: Cambridge University Press.
Turiel, E. (2006). The development of morality. In W. Damon, R. M. Lerner, & N. Eisenberg (Eds.), Handbook of child psychology: Vol. 3. Social, emotional, and personality development (6th ed., pp. 789 – 857). Hoboken, NJ: Wiley.
Vygotsky, L. S. (1978). Mind in society. Cambridge, MA: Harvard Uni- versity Press.
Wainryb, C. (1995). Reasoning about social conflicts in different cultures: Druze and Jewish children in Israel. Child Development, 66, 390 – 401. doi:10.2307/1131585
Walters, M. L., Dautenhahn, K., Koay, K. L., Kaouri, C., te Boekhorst, R., Nehaniv, C., . . . Lee, D. (2005). Close encounters: Spatial distances between people and a robot of mechanistic appearance. In H. Hirukawa, S. Sugano, Y. Nakamura, C. S. G. Lee, & J.-P. Laumond (Eds.), Proceedings of the IEEE–RAS International Conference on Humanoid Robots 2005 (pp. 450 – 455). New York, NY: Association for Computing Machinery. doi:10.1.1.64.7979
Wang, L., Rau, P., Evers, V., Robinson, B., & Hinds, P. (2010). When in Rome: The role of culture and context in adherence to robot recommen- dations. In L. Wang, P.-L. P. Rau, V. Evers, B. K. Robinson, & P. Hinds (Eds.), Proceedings of the 5th ACM/IEEE International Conference on Human–Robot Interaction (pp. 359 –366). New York, NY: Association for Computing Machinery. doi:10.1145/1734454.1734578
Waytz, A., Cacioppo, J. T., & Epley, N. (2010). Who sees humans? The stability and importance of individual differences in anthropomorphism. Perspectives on Psychological Science, 5, 219 –232. doi:10.1177/ 1745691610369336
Weiss, A., Igelsböck, J., Tscheligi, M., Bauer, A., Kühnlenz, K., Wollherr, D., & Buss, M. (2010). Robots asking for directions: The willingness of passers-by to support robots. In L. Wang, P.-L. P. Rau, V. Evers, B. K. Robinson, & P. Hinds (Eds.), Proceedings of the 5th ACM/IEEE Inter- national Conference on Human–Robot Interaction (pp. 23–30). New York, NY: Association for Computing Machinery. doi:10.1145/ 1734454.1734468
Williams, B. (1985). Ethics and the limits of philosophy. Cambridge, MA: Harvard University Press.
Received August 1, 2010 Revision received September 13, 2011
Accepted November 14, 2011 �
314 KAHN ET AL.
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .