Two paragraphs about Caregivers Robots in family.

407

TELEMEDICINE AND e-HEALTH Volume 13, Number 4, 2007 © Mary Ann Liebert, Inc. DOI: 10.1089/tmj.2006.0068

Original Research

Developing a Telepresence Robot for Interpersonal Communication with the Elderly in a

Home Environment

TZUNG-CHENG TSAI, Ph.D., YEH-LIANG HSU, Ph.D., An-I. MA, M.S., TREVOR KING, M.S., and CHANG-HUEI WU, Ph.D.

ABSTRACT

“Telepresence” is an interesting field that includes virtual reality implementations with hu- man–system interfaces, communication technologies, and robotics. This paper describes the development of a telepresence robot called Telepresence Robot for Interpersonal Communica- tion (TRIC) for the purpose of interpersonal communication with the elderly in a home en- vironment. The main aim behind TRIC’s development is to allow elderly populations to re- main in their home environments, while loved ones and caregivers are able to maintain a higher level of communication and monitoring than via traditional methods. TRIC aims to be a low-cost, lightweight robot, which can be easily implemented in the home environment. Under this goal, decisions on the design elements included are discussed. In particular, the implementation of key autonomous behaviors in TRIC to increase the user’s capability of pro- jection of self and operation of the telepresence robot, in addition to increasing the interac- tive capability of the participant as a dialogist are emphasized. The technical development and integration of the modules in TRIC, as well as human factors considerations are then de- scribed. Preliminary functional tests show that new users were able to effectively navigate TRIC and easily locate visual targets. Finally the future developments of TRIC, especially the possibility of using TRIC for home tele-health monitoring and tele-homecare visits are dis- cussed.

Department of Mechanical Engineering, Yuan Ze University, Chung-Li, Taiwan, Republic of China.

INTRODUCTION

TELEPRESENCE IS AN INTERESTING FIELD thatincludes virtual reality implementations with human–system interfaces, communica- tion technologies, and robotics. The earliest re- search in telepresence dates back to the 1960s. Goertz, in 1965, and Chatten, in 1972, showed that when a video display is fixed relative to

the operator’s head and the head’s own pan- and-tilt drive the camera’s pan-and-tilt func- tions, the operator feels as if she were physi- cally present at the location of the camera, however remote it is.1

Sheridan defines telepresence as: “ . . . visual, kinesthetic, tactile or other sensory feedback from the teleoperator to the human operator that is sufficient and properly displayed such

that the human feels that he is present at the remote site, and that the teleoperator is an ex- tension of his own body.”1,2 Whereas Akin et al.3 describe telepresence such that, “At the worksite, the manipulators have the dexterity to allow the operator to perform normal human functions. At the control station, the operator receives sufficient quantity and quality of sen- sory feedback to provide a feeling of actual presence at the worksite.” These two defini- tions emphasize control and sensory feedback between the human operator and the “teleop- erator” or the “manipulator at the worksite.”

Draper4 discerns three definitions of tele- presence: the simple, the cybernetic, and the ex- periential. In the simple definition, telepres- ence refers to the ability to operate in a computer-mediated environment. In the cyber- netic definition, telepresence is an index of the quality of the human–machine interface. In the experiential definition, telepresence is a mental state in which a user feels physically present within the computer-mediated environment.

Schloerb defines telepresence from the point of view of an “observer,” “a person is objec- tively present in a real environment that is physically separated from the person in space.”5 He uses three types of specifications to make the definitions more precise: (1) a set of tasks, (2) a transformation imposed on the human operator’s control output and sensory input, and (3) a transformation of the region of presence. The degree of objective telepresence is equal to the probability of successfully com- pleting a specified task. Schloerb also proposed

that perfect telepresence occurs when the op- erator cannot discriminate virtual from reality.

To summarize, telepresence provides a con- nection between a user (or the “operator” as defined by Sheridan and Akin) and a distant participant or an environment (real world or computer-generated world), to perform social interactions (user–participant interactions) or specific tasks (user–environment interactions). In this research, we are interested in the appli- cation of a telepresence robot for communicat- ing and interacting between the “user” and the “participant” in a remote site. In such applica- tions, the remote participant is not only an “ob- server” as in Schloerb’s definition, but also a “dialogist” in the interpersonal communication.

Following this framework, there are two views in telepresence application for interper- sonal communication: the user’s view and the participant’s view, as depicted in Figure 1. From the user’s view, telepresence enables the user to project herself/himself to another place by controlling the telepresence robot or system. In the meantime, the user perceives immersion from the sensory feedback from the remote en- vironment created by telepresence.

As discussed earlier, the “participant” may have two roles in telepresence application in in- terpersonal communication: as an observer and a dialogist. From the participant’s view as an observer, telepresence provides necessary ele- ments to the user and the telepresence robot, so that the participant recognizes the telepres- ence robot as a representation of the user. From the participant’s view as a dialogist, telepres-

TSAI ET AL.408

FIG. 1. The framework of projection-immersion and observer-dialogist.

ence also enables dialogue between the partic- ipant and the user by transmitting audio, video, gestures, physical movements, and other envi- ronmental information between the participant and the user, which are helpful for effective communication.

For interpersonal communication, telepres- ence differs from traditional video conferenc- ing by establishing a true sense of shared space among geographically remote persons. By duplicating the three-dimensional human ex- perience via actual face-to-face encounters, telepresence is a stunningly different way to telecommunicate.

Applying robotic technology for homecare of the elderly

Many developed countries are facing the problems associated with an increasing elderly population. The need of healthcare for the el- derly, both physiologically and psychologi- cally, is an urgent issue. In particular, many el- derly people desire to stay in their own private residences for as long as possible, and thus methods are needed to allow them to do so safely and at a reasonable cost. Development of service robots to assist elderly people in ac- tivities of daily living (ADL) or to improve their quality of life (QOL) in a home environment is an important research field.

Prassler et al.6 developed the robotic wheel- chair Mobility Aid for Elderly and Disabled People (MAid) to support and transport elderly people with limited motion skills and to pro- vide them with a certain amount of autonomy and independence. Kiguchi et al.7 developed robotic exoskeletons to assist the mobility of physically weak persons such as the elderly, the disabled, and the injured. The control of the robotic exoskeleton is based on electromyogra- phy (EMG) signals, as EMG signals are imper- ative signals to understand how the user in- tends to move.

Robots are not only used for physiological assistance but also can provide psychological consolations for elderly people. Wada et al.8 tested a seal-type animal robot “Paro” at a day- care service center to investigate its effects on the elderly. After interaction with Paro, the face scale scores of elderly people were classified as

“good,” and their vigor scores in the question- naires were considered high, confirming that Paro improved the mood of elderly people and made them more vigorous. Kanamori et al.9 also examined the usefulness of pet-type robots named AIBO among elderly patients and hand- icapped persons in nursing homes or in home environments via biochemical markers, self-as- sessment, or health-related QOL question- naires.

In slight contrast, Pollack et al.10 developed “Autominder,” an intelligent cognitive orthotic system intended to help older adults adapt to cognitive decline and continue satisfactory per- formance of routine activities, thereby poten- tially enabling them to remain in their own homes longer. Lytle11 presented a robotic care bear whose purpose was to watch over the el- derly residents in a hi-tech retirement center. The Teddy-like bear could record how long the residents spent performing various tasks and assisted in monitoring the residents’ response times to spoken questions, before relaying con- clusions to staff or altering them of unexpected changes.

In telepresence research, there have been few examples in the field of medical care, empha- sizing the use and application of telepresence robots as a tool for interpersonal communica- tion. “Physician-Robot” is a telepresence robot developed by Intouch-health Company in co- operation with Johns Hopkins University for physicians to easily and more frequently visit with hospitalized patients.12 Physicians oper- ate the robot with a swiveling video camera and computer screen mounted on a mechani- cal base by guiding it through patients’ rooms via a remote-control joystick. By communicat- ing through the hospital’s virtual private net- work (VPN) and an 802.11b wireless network, doctors at home or out of town can teleconfer- ence with and examine patients. Physician-Ro- bot is not expected to replace visits from real physicians, but is to act as an extension of physician–patient contact. In the evaluation at the Johns Hopkins University Hospital, pa- tients are more satisfied because physicians spend more time with them.

Providing Education By Bringing Learning Environments to Students (PEBBLES), a tele- presence system, is developed to unite med-

DEVELOPING A TELEPRESENCE ROBOT FOR INTERPERSONAL COMMUNICATION 409

ically fragile children who are hospitalized for prolonged periods with their regular school site. PEBBLES was developed in Canada by a private company called Telbotics, in coopera- tion with Ryerson University and the Univer- sity of Toronto. A PEBBLES system consists of two child-sized robots capable of transmitting video, audio, and documents to each other. One unit is placed with the hospitalized child and the other unit is located in the child’s reg- ular classroom. The units are connected via a high-speed communications link. The class- room unit has a swiveling monitor that dupli- cates human head movement and a hand that serves as an attention-getting device. PEBBLES creates a virtual presence for the remote child in the classroom. The presence is so real that in the many evaluation cases teachers and fellow students come to react to the school unit as if it were the hospitalized child.13

Telepresence robot for interpersonal communication

Aging is associated with an increased risk for isolation. However, social interaction can delay the deterioration and associated health prob- lems of elderly people.14 This paper describes the development of a telepresence robot called Telepresence Robot for Interpersonal Communica- tion (TRIC) for interpersonal communication with the elderly in a home environment. The main aim behind TRIC’s development is to al- low elderly populations to remain in their home environments, while loved ones and caregivers are able to maintain a higher level of communication and monitoring via tradi- tional methods.

TRIC is positioned as a low-cost, lightweight robot, which can be easily implemented in the

home environment. Actually the TRIC robot is intended to be used as often and as easily as a home appliance. It is expected that TRIC not only provides a convenient interactive com- munication device for family members and caregivers to communicate with and express care to elderly people, but also a tool for tele- homecare visits by doctors or nurses, and a tool for home tele-health monitoring tasks such as measuring vital signs (blood pressure, glucose, etc.), and monitoring ADL.

Section 2 of this paper reviews the recent and relevant research literature in the field of telep- resence, with an emphasis on the various de- sign elements used throughout. Decisions on the design elements included are discussed in Section 3. In particular, the implementation of key autonomous behaviors in TRIC to increase the user’s capability of projection of self and operation of the telepresence robot, in addition to increasing the interactive capability of the participant as a dialogist are emphasized. Sec- tion 4 describes TRIC’s hardware design. Focus then shifts to human factors considerations and interface design in Section 5. Section 6 describes preliminary evaluations of TRIC, followed by the concluding remarks in Section 7. Future de- velopments of TRIC, especially the possibility of using TRIC for home telehealth monitoring and tele-homecare visits are discussed.

Design elements in telepresence systems

This section surveys the application-oriented telepresence literature which describes the de- velopment of a telepresence system. The design elements emphasized in these studies are ex- tracted and summarized in Table 1. A discus- sion of these design elements as they fit into the framework of projection-immersion and

TSAI ET AL.410

TABLE 1. DESIGN ELEMENTS AND RELATED TECHNOLOGICAL KEYWORDS FOR TELEPRESENCE

Design elements Related technological keywords

Data transmission RF and Internet transmission, time-delay improved algorithm Teleoperation Simultaneous operation, robotic design Supersensory Dexterous mechanism Anthropomorphic elements Humanoid mechanism and expression Stereoscopic elements Binocular and panoramic vision, image processing Stereophonic elements Head-related transfer function, stereo audio Eye contact Camera and screen with specific placement Autonomous behaviors Environmental map establishment, self-maintenance capability

observer-dialogist illustrated in Figure 1 in the previous section is given below.

Data transmission. Data transmission, the transmission of control commands and sensory feedback, is a basic design element for the connection between the user and the remote telepresence robot or system. Wireless radio frequency and Internet are used in most tele- presence applications, and dedicated lines are used in specific applications (such as operation in space and deep sea).

From the user’s view, timing of data trans- mission is important. Time delays would de- grade the telepresence performance in both projection and immersion of the user. From the participant’s view, the time delays also affect the participant’s impression as an observer and interactive capability as a dialogist. Therefore, past telepresence research in data transmission focused on the development of a control scheme to deal with time delays for promoting performance.15,16

Teleoperation. Many studies in telepresence emphasize on enabling the user to modify the remote environment17–19 that is, projecting the user to the teleoperator. A teleoperator is a ma- chine that extends the user’s sensing and/or manipulating capability to a location remote from that user. Teleoperation refers to direct and continuous human control of the teleoper- ator.

NASA’s Full-Immersion Telepresence Test- bed (FITT), which combines a wearable inter- face integrating human perception, cognition and eye–hand coordination skills with a robot’s physical abilities, is a recent example of re- search in teleoperation.20 The teleoperated master–slave system “Robonaut” allows an in- tuitive, one-to-one mapping between master and slave motions. The operator uses the FITT wearable interface to remotely control the Robonuat to follow the operator’s motion fully in simultaneous operation to perform complex tasks in the International Space Station.

Supersensory. Supersensory refers to an ad- vanced capability to modify the remote envi- ronment provided by a dexterous robot or a precise telepresence system. From the user’s

view, the user’s manipulative efficiency for special tasks is enhanced when projecting onto a telepresence robot with supersensory. Green et al.21 developed a telepresence surgery sys- tem integrating vision, hearing, and manipula- tion. It consists of two main modules, a sur- geon’s console, and a remote surgical unit located at the surgical table. The romote unit provides scaled motion, force reflection, and minimized friction for the surgeon to carry out complex tasks with quick, precise motions. Sa- tava,22 Schurr et al.,23 and Ballantyne24 have also applied supersensory in telepresence sur- gery.

Supersensory elements can also provide the user with a novel immersion feeling in a remote environment. For example, the user can control the zoom function of the camera on a telepres- ence robot to observe the small details of the remote environment, which the user does not normally see with the naked eye.

Anthropomorphic elements. In telepresence ap- plications, nonanthropomorphic telepresence robots are usually designed to perform specific tasks. Many researches added anthropomor- phic elements to their telepresence robots in or- der to improve the interaction between users and participants.25–28 From the participant’s view, anthropomorphic elements enhance the impression of the telepresence robot as a true representation of the remote user. The friendly interface and characteristics of the anthropo- morphic telepresence robot also increase the in- teractive capability of the participant as a dial- ogist.

From the participant’s view, the basic re- quirement for interpersonal communication using telepresence is that the participants must realize whom the telepresence robot repre- sents. Therefore, the primary anthropomorphic element incorporated is the user’s face dis- played on an LCD screen mounted on the tele- presence robot. Both Dr. Robot and the tele- presence system PEBBLES described in the first section of the paper use an LCD screen to dis- play the user’s face.

In addition to using the LCD screen, creat- ing mechanical facial expressions is another in- teresting field in robotic research.29 Mechanical facial expressions can also be used to increase

DEVELOPING A TELEPRESENCE ROBOT FOR INTERPERSONAL COMMUNICATION 411

the humanoid characteristics of the telepres- ence robot to further encourage people to in- teract and communicate with the user.

Stereoscopic and stereophonic elements. In tele- presence research, stereoscopic and stereo- phonic design elements are often emphasized to create a telepresence illusion of the remote environment or people aiming to increase the feeling of immersion for the user. For example, the user can identify the distance between an object and the telepresence robot by binocular vision;30 the head-related transfer function (HRTF) for stereophonic effect enables the user to identify the location and direction of a sound.31

Telepresence videoconferencing is an impor- tant application using stereoscopic and stereo- phonic elements.32–34 Telepresence videocon- ferencing enables the users and the participants to communicate more efficiently. In other words, the interactive capability of the partici- pant as a dialogist is enhanced. Lei et al.35 pro- posed a representation and reconstruction module for an image-based telepresence sys- tem, using a viewpoint-adaptation scheme and an image-based rendering technique. This sys- tem provides life-size views and 3D perception of participants and viewers in videoconferenc- ing. The purpose of this research is to provide the feeling of a virtual-reality presence, in which realistic 3D views of the user should be perceived by the participant in real time and with the correct perspective.

Eye contact. In telepresence applications, eye contact can increase the immersion feeling of the user and the interactive capability of the participant as a dialogist. The only means to provide eye contact during interpersonal com- munication between the user and the partici- pant through a telepresence robot is when the face of the user is displayed on an LCD screen. However, the placement of the camera on a telepresence robot is usually on top of the LCD screen, which hinders direct eye contact be- tween the user and the participant through the telepresence robot.

Hopf36 proposed an implementation of an autostereoscopic desktop display suitable for computer and communication applications.

The goal of this research is to develop a system combining a collimation optic with an au- tostereoscopic display unit to provide natural face-to-face and eye contact communication without causing eye strain.

Autonomous behaviors. In principle, a tele- presence robot is operated by a remote user, and does not possess autonomous behaviors. However, the telepresence robot should be able to deal with possible hazardous situations au- tonomously when the remote user is not aware of the hazardous situation, cannot control the telepresence robot properly, or the data trans- mission is lost. From the user’s view, au- tonomous behavior increases the user’s capabil- ity of projection to operate the telepresence robot safely and reliably in a dynamic environment. From the participant’s view, autonomous be- havior also increases the interactive capability of the participant as a dialogist. For example, a telepresence robot with the autonomous behav- ior of identifying the direction of the participant who is speaking can assist the remote user to re- spond more quickly and properly.

An interactive museum tour-guide robot was developed by two research projects TOURBOT and WebFAIR, funded by the European Union.25–28 Thousands of users around the world controlled this robot through the Web to visit a museum. They developed a modular and distributed software architecture which in- tegrates localization, mapping, collision avoid- ance, planning, and various modules con- cerned with user interaction and Web-based telepresence. With these autonomous features, the user can operate the robot to move quickly and safely in a museum crowded with visitors.

Basic data transmission structure and design elements of TRIC

As mentioned earlier, the telepresence robot TRIC developed in this research aims to be a low-cost, lightweight robot, which can be easily implemented in the home environment. There- fore the primary decision was whether to use the Asymmetric Digital Subscriber Line (ADSL) and the Wireless Local Area Network (WLAN), which are commonly found in the home envi- ronment, as the channel of data transmission.

TSAI ET AL.412

Two-way audio and one-way video communi- cation can be transmitted through a network In- ternet Protocol (IP) camera, which is also a com- mon tool for home monitoring.

Instead of using a PC, a Mobile Data Server (MDS) was developed as the core of TRIC. Fig- ure 2 shows a picture of the laboratory proto- type of the MDS, which consists of a PIC server mounted on a peripheral application board. The PIC server integrates a PIC microcontroller (PIC18F6722, Microchip, Chandler, AZ), EEP- ROM (24LC1025, Microchip) and a networking IC (RTL8019AS, Realtek, Taiwan, ROC). It pro- vides networking capability and can be used as a Web server. The peripheral application board as well as the program in the PIC microcon- troller can be easily customized to adapt to dif- ferent sensors and applications. The dimen- sions of the MDS prototype are 40 � 85 � 15 mm. Internet and serial interface (RS-232) are the primary communication interfaces of the MDS with client PCs and other devices. The MDS also receives external signals (e.g., sensor signals) through specific analogue or digital I/O ports, and provides interintegrated circuit (I2C) communications to allow connections with external modules. A Multi-Media Card (MMC) in the MDS can be used to store data in FAT16 file format. Compared to a PC, the MDS is low cost, has smaller dimensions, con- sumes less energy (thus, can be powered by batteries), is not affected by viruses, and is safer and more reliable.

Figure 3 shows the basic data transmission structure of TRIC. The user projects herself/ himself to TRIC in the remote environment by sending control commands to TRIC through the Internet gateway. The user is able to im- merse in the remote environment from the sen- sory feedback transmitted through the Internet gateway. TRIC uses a WLAN the the connec- tor by connecting to the WLAN in the home environment. MDS takes charge of receiving commands from the user and sending com- mands to specific modules which coordinate with each other to perform specific tasks. Fi- nally, the user can have physical interaction and verbal communication with the participant by controlling TRIC as his/her physical exten- sion in the remote environment.

Under this basic structure, Table 2 lists the design elements currently planned for the de- sign of TRIC. The implementation of teleoper- ation in TRIC is quite fundamental. Teleopera- tion allows the user to move TRIC through the environment while controlling the pan and tilt of the IP camera from a remote client PC. Su- persensory ability is reflected in the zooming capability of the IP cam and the sensing ca- pability of the various sensors installed for en- vironment detection. With the limited process- ing ability of the MDS, the user’s face cannot be displayed (so eye contact is also not possi- ble on this telepresence robot). Instead, me- chanical facial expressions are incorporated as the anthropomorphic element. Sophisticated

DEVELOPING A TELEPRESENCE ROBOT FOR INTERPERSONAL COMMUNICATION 413

FIG. 2. A picture of the laboratory prototype of the MDS.

stereoscopic and stereophonic elements have been omitted to keep TRIC a low-cost, afford- able homecare robot.

Autonomous behavior is the design element that received the most attention during the planning of TRIC. In principle, a telepresence robot is operated by a remote user who pos- sesses complete control authority. However, a major emphasis of this research is to implement key autonomous behaviors in TRIC in order to increase the user’s operating capability and re-

duce the user’s workload during operation. By doing so, the aim was to also increase the in- teractive capability of elderly people as recip- rocal communicators.

Adding autonomous behaviors implies that the control authorities of the telepresence ro- bot are shared with the participant or the en- vironment it is interacting with. Several pos- sible features for sharing control authority with the remote participants are discussed below.

TSAI ET AL.414

FIG. 3. The data transmission structure of TRIC.

“Look at that!” Participants engaged in a face- to-face conversation often share the same view by pointing to an object in discussion. How- ever, it will be difficult for the user to either point to a certain object or to find the object the remote participant is pointing at through the telepresence robot. A 2 degree-of-freedom ro- bot arm equipped with a laser pointer is used as a joint attention device to realize the “look at that!” function. The remote participant can direct the view of the telepresence robot by pointing the laser pointer to the object in ques- tion.

“Where is the speaker?” It is not easy for the user to locate the source of sound in 3D space through the telepresence robot. When interact- ing with the remote participant, “Where is the speaker?” enables the telepresence robot to au- tomatically locate and track speakers without control from the user. With this feature, the par- ticipant controls the telepresence robot by us- ing her/his own voice.

“Come here!” and “Follow me!” In “Where is the speaker?” the telepresence robot can locate the source of the sound. Therefore the “Come here!” feature allows the user to command the telepresence robot to go to the source of the sound. “Follow me!” is another interactive be- havior which is common in interpersonal com- munication. The passive infrared motion sen- sors combined with ultrasonic range-finding sensors are used to perform the low-cost and reliable function of “Follow me!” where TRIC continuously follows the intended participant.

Several possible modes in sharing control au- thority with the remote environment are dis- cussed below.

Obstacles avoidance. It is difficult for the user to identify environmental information from the robot’s limited viewing angles. Therefore, au- tomatic obstacle avoidance is necessary. When an obstacle is detected within a specific dis- tance from the robot, the obstacle avoidance al- gorithm is activated, and the robot deviates from the movement direction controlled by the user in order to avoid this obstacle.

Self-maintenance. The most fundamental self- maintenance function is the ability of TRIC to automatically recharge its battery when needed. This includes the ability to detect en- ergy capacity, self-positioning to locate and move to the charging station, and automatic parking control to dock the robot in the charg- ing station.

Hardware design of TRIC

Figure 4 shows a picture of TRIC, and Fig- ure 5 shows its spatial configuration. TRIC integrates an MDS, an IP camera with pan/ tilt/zoom functions and two-way audio com- munications, a wireless LAN adaptor pair, a speaker and microphone, and a 12-V LiFePO4 battery (10 Ah) with power management sys- tem, all within a mobile platform. Table 3 sum- marizes the basic specifications of TRIC.

In Section 3, the basic data transmission structure has been defined (Fig. 3). Following this structure, Figure 6 shows the control struc- ture for TRIC. TRIC has its own IP address and is connected to the world via a wireless Inter- net link. The WLAN adaptor pair provides a standard wireless network for the MDS and IP camera. The MDS is the core of TRIC for data transmission, which receives commands from

DEVELOPING A TELEPRESENCE ROBOT FOR INTERPERSONAL COMMUNICATION 415

TABLE 2. DESIGN ELEMENTS INCLUDED IN TRIC

Design elements Corresponding technological strategies

Data transmission Use MDS for the core of system Teleoperation Design of mobility platform Supersensory Provide zoom of IP cam, implement various sensors

for environment detection Anthropomorphic elements Design of mechanical facial expression Stereoscopic elements Not included Stereophonic elements Not included Eye contact Not available Autonomous behaviors Share control authority to participate and environment

the user via the Internet and sends commands to the various modules using the I2C bus.

As shown in Figure 6, currently three mod- ules have been implementd: the IP camera with facial expression control module, the environ- mental perception module, and the DC motor driver (DMD) module. Each module has one slave microprocessor for control and data pro- cessing, but the master MDS has higher prior- ity for taking corrective actions from the user’s decisions. The design of each module is de- scribed in details below.

IP camera with facial expressions control module. The controls of pan/tilt/zoom functions for IP camera, controls of facial expressions, and a passive 2 degrees-of-freedom arm are inte- grated into a module. The IP camera provides RS232 interface for controlling its pan/tilt/ zoom functions. The slave microcontroller re-

ceives pan/tilt/zoom control commands from master MDS then sends them to the IP camera via RS232 interface. Facial expressions are cre- ated via the various positioning of eyebrows and an LED mouth. Motions of eyebrows are driven by two servomotors. A matrix of light- emitting diodes (LEDs) is implemented to dis- play happiness, sorrow, and normality. Mo- tions generated by servomotors and LEDs symbolize emotions of the user.

The joint attention “Look at that!” function described in Section 3 is achieved by the pas- sive 2 degrees-of-freedom arm. The participant manipulates the passive arm equipped with a laser pointer to point to a specific direction, which is then measured and transformed into viewing coordinates for the IP camera. Then the user activates the “Look at that!” function from a remote client PC, and the IP camera ro- tates to the viewing angle so that the user shares the same view as the participant.

DC Motor Driver (DMD) module. TRIC has two differential driving wheels on its middle axis. Its mobility is controlled by the DMD module that includes a slave microcontroller driving two motor controllers with H-bridge DC motor control circuits. Each motor is equipped with an incremental encoder count- ing 128 clocks per rotation. Velocity and posi- tion data are available from two optical en- coders for precise closed loop control. The slave microcontroller in DMD module performs four basis tasks: (1) communication with the master MDS via I2C bus, (2) generation of Pulse Width Modulation (PWM) duty cycle, (3) reading counts from each encoder, (4) reading 4-bit bi- nary code from an environmental perception module.

The DMD module receives commands from the user through the Internet, so that the user can navigate TRIC to move around freely in the remote environment. The DMD module also cooperates with other modules to achieve the autonomous behaviors described in the previ- ous section.

Environmental perception module. The envi- ronmental perception module is designed for the user to acquire information about the envi- ronment around TRIC. This module uses six ul-

TSAI ET AL.416

FIG. 4. A picture of TRIC.

trasonic sensors, a digital compass, and a tem- perature sensor for environment perception, collision detection, and the “follow me” func- tion. Six ultrasonic sensors, with frequency of 40 kHz, an effective range of roughly 3 cm to 4 m, and approximately 20 to 35 degrees of measurement cone, are arranged around TRIC as shown in Figure 7. The slave microcontroller in this environmental perception module man- ages measurements from the ultrasonic sensors to generate information such as whether there are objects around TRIC. Urgent stop or back- ward commands are then sent to the master MDS when objects are detected in the “close” region in front of TRIC.

Human factor and interface design of TRIC

As mentioned earlier, the TRIC robot is in- tended to be used as often and as easily as a home appliance. Therefore, human factors con- siderations are very important. The current height of the robot is 75 cm, with the camera positioned at approximately 70 cm tall. At this height, optimal viewing distances were deter-

mined to fulfill two human factors considera- tions: limiting neck flexion to less than 20–30°37 during prolonged interaction, and maintaining an acceptable viewing distance for the el- derly.38 Anthropometric data on Taiwanese el- derly was used to determine appropriate hori- zontal positioning distances for taller and shorter participants between themselves and the robot.39 When the participant is standing, the optimal viewing distance lies between 110 to 154 cm, depending on the height of the par- ticipant.

The weight of TRIC is approximately 8.7 kg when the battery pack is installed. This is the lowest achievable weight of TRIC while main- taining all of its current functions. Two com- mon ergonomic tools were used to assess the weight of the robot for lifting purposes from a height of 12� to a height of 32,� with a horizontal distance of 12,� and a frequency of 1/8 hours or once a day. Both the revised National Insti- tute for Occupational Safety and Health (NIOSH) lifting equation40 and Mital et al.’s41 tables for lifting/lowering limits resulted in a lifting limit of 15 kg for 90% of the female pop-

DEVELOPING A TELEPRESENCE ROBOT FOR INTERPERSONAL COMMUNICATION 417

FIG. 5. Spatial configuration of TRIC.

ulation. However, since TRIC was designed to communicate with elderly participants, a cor- rection factor for age was needed to account for decreases in strength. From the graph depict- ing a decrease in total body strength with age by Voorbig and Steenbekkers,42 at 80 years and above, strength falls to 57% of peak body strength among women. Therefore, the 15 kg limit found via the revised NIOSH lifting equa- tion and Mital’s Lifting/Lowering tables was multiplied by 0.57 to result in a corrected lift- ing limit of 8.55 kg. This new limit accounts for a larger majority of the general population than set by typical ergonomic standards, and ac-

counts for the majority of elderly populations as well. The weight of TRIC falls within 0.15 kg of the new corrected lifting limit. Thus, we can assume a low risk of injury among a large ma- jority of the general and elderly population for lifting and lowering TRIC once a day from the locations mentioned above.

It is also suggested to maintain TRIC ’s speed below typical walking speed—1.2 m/second for pedestrian crossings, but higher than the lowest walking speed found among elderly people undergoing rehabilitation—0–0.17m/ second, to prevent collisions from occurring due to differing speeds between the robot and

TSAI ET AL.418

FIG. 6. Hardware architecture of the control system.

TABLE 3. SPECIFICATIONS OF TRIC

L � W � H 400 � 410 � 750 mm Battery type LiFePO4 Li-ion

Weight 8.7 kg Battery capacity 12.8 V 10Ah Speed 20–25 cm/seconds Battery operation time 120–150 minutes Circumrotation radius 34.5 cm Battery recharge time 90 minutes (6A)

the participant, while maintaining a reasonable speed of transportation for the user.43 There- fore the current speed of the robot is set to 23 cm/second. In addition, the robot is equipped with obstacle detection sensors that cease the robot’s motion when a certain minimum dis- tance is breached. This will also prevent the ro- bot from colliding with its participants and the participant’s environment.

A primary consideration for communication between the user and the participant via TRIC is sound quality for projection and receiving. It has been identified that as we age we prefer louder sounds for hearing speech sounds ade-

quately.44 For this reason speakers capable of 85 dB of sound, with volume control, are mounted to TRIC so that elderly participants would be able to hear speech sounds produced by remote users clearly. In addition, the closest point for communication is on TRIC ’s “head” (the high- est point on the robot). Therefore, a microphone is placed as close as feasibly possible to the head of TRIC in an attempt to optimize voice capture from the participant to the user. It has also been suggested that a remote control with various functions for TRIC be created in the future that would include a microphone to further improve voice capture from the participant.

DEVELOPING A TELEPRESENCE ROBOT FOR INTERPERSONAL COMMUNICATION 419

FIG. 7. A specific arrangement of six ultrasonic sensors.

Figure 8 shows the user control interface for TRIC containing five sizeable frames. Each frame provides some panels to handle differ- ent tasks. The user first inputs the IP address for the MDS and the IP camera in the upper- right corner. The MDS and the IP camera can share the one physical IP using different ports. An authorization process using passwords can also be implemented.

The large frame on the left-hand side of the screen contains a video image from TRIC. The lower left-hand frame provides buttons for controlling the movement of TRIC, as well as the button for detecting environmental situa- tions. The frame for environmental situations allows users to adequately assess the distance of objects around TRIC by processing the mea- surements from the environment perception module. Other environmental information such as temperature is also displayed in this frame. The lower right-hand frame provides the control buttons of the IP camera for pan,

tilt, and zoom. Users can express emotions by pushing the corresponding buttons in above the IP camera control frame. Basic autonomous behaviors can be activated by pushing the con- trol buttons in the middle right-hand frame.

A preliminary evaluation for usability for the remote user

Experimental data is necessary to assess TRIC’s success as a telepresence communication device. TRIC’s evaluations will be extensive, as there are two realms of concern. Objective mea- sures of functionality are required from a tech- nical perspective to ensure that TRIC is capable of performing each intended command. Objec- tive measures of usability are also required to de- termine if TRIC proves to be a realistic, useful communication tool. Last, subjective measures from users and participants are needed to deter- mine how accepted TRIC is in a domestic envi- ronment for its intended purpose.

TSAI ET AL.420

FIG. 8. Control interface for TRIC.

This section describes a preliminary evalua- tion for assessing usability and functionality of TRIC. The purpose of the evaluation was to assess how efficient new users were at navi- gating TRIC through a fixed obstacle course (similar to that of a furnished, domestic envi- ronment), to assess the efficiency of new users using the multidirectional viewing camera to locate predetermined targets, and to assess whether there was a significant improvement in time across users with practice for both nav- igation and visual objectives.

In this test, seven users, unfamiliar with TRIC ’s navigation interface, were asked to nav- igate TRIC through a predetermined route to a specified target (Task A). Once the target was reached, users were then asked to locate two separate target objects with TRIC’s adjustable camera in sequential order (Tasks B and C). The test was then repeated two more times by each user (Trial 1 to Trial 3). Users were located in another room such that they had no visual con- tact with TRIC nor the remote environment other than the first-person visual display trans- mitted to the PC operating interface. In addi- tion, users were familiar with the experimental space and the location of the target objects be- fore testing commenced. This was done as it was assumed that potential users would be fa- miliar with the environment of the participants.

Due to the small number of subjects and sub- sequent samples, statistical analysis was not conducted on the raw data scores from these preliminary evaluations. Table 4 shows the av- erage time of the seven users to complete each task from Trial 1 to Trial 3. For each task, an “optimal time” achieved by a skilled user after many practices was used as a reference for cal-

culating the “efficiency” of each user on this task. From the efficiency scores, the navigation task (Task A) seems to be the most difficult task for the new user. However, a trend of an in- crease in percent efficiency from Trial 1 to Trial 3 is clear, which implies that the users’ perfor- mance improves quickly after practice. The ef- ficiency scores of Task B and C for locating ob- jects in the remote environment through TRIC maintained at 80 to 90%, which shows that the users had no difficulty with these tasks.

Overall, it can be assumed that users are able to effectively achieve navigational and visual target goals, while improving with as little as three attempts. Extensive tests are planned in an environment similar to that of the intended use (a domestic environment) with both users and the intended participants (older adults), to determine objectively and subjectively TRIC’s practicality as an effective tele-health commu- nication device.

CONCLUSIONS AND DISCUSSIONS

This paper presents the development of a teleprepresence robot TRIC. The main aim be- hind TRIC’s development is to allow elderly populations to remain in their home environ- ments, while loved ones and caregivers are able to maintain a higher level of communication by establishing a true sense of shared space among geographically remote persons.

TRIC allows users to project a sense of self via telepresence. By controlling TRIC’s IP cam- era, navigational actions, real-time voice com- munication, and limited nonverbal cues the user is able to communicate and monitor the elderly participant in a manner above video– audio communication. Human factors consid- erations have also been implemented to allow for optimal communication and use by both sets of users (local and remote), and to lessen any risk of potential injury.

With a simplified design in place, safety fac- tors accounted for, and widely accessible via the Internet and a personal computer, it is be- lieved that TRIC would be a cost-feasible op- portunity to provide an enriched communica- tion and monitoring experience for the elderly, their loved ones, and their caregivers. Prelimi-

DEVELOPING A TELEPRESENCE ROBOT FOR INTERPERSONAL COMMUNICATION 421

TABLE 4. DATA FOR THE PRELIMINARY EVALUATION FOR USABILITY FOR THE REMOTE USER

Trial 1 Trial 2 Trial 3

Task A Average time (seconds) 96.43 80.00 69.57 Efficiency 51.85% 62.50% 71.87%

Task B Average time (seconds) 22.43 17.71 17.14 Efficiency 71.34% 90.32% 93.33%

Task C Average time (seconds) 10.57 10.29 9.57 Efficiency 85.14% 87.50% 94.03%

nary functional tests have shown that new users were able to effectively navigate TRIC and easily locate visual targets. Their efficiency in doing so increased significantly in as little as three trials. Further tests in domestic environ- ments with elderly participants are needed to further assess objective functions and to cap- ture subjective ratings on TRIC’s potential as an effective communication device.

Besides interpersonal communication, there is also a potential of using TRIC for home tele- health monitoring and tele-homecare visits. Figure 9 shows the structure of the Portable Telehomecare Monitoring System (PTMS) de- veloped by the authors.45 The PTMS is a de- centralized system. Instead of using the cen- tralized database structure that gathers data from many households, a single household is the fundamental unit for sensing, data trans- mission, storage, and analysis in the PTMS. The monitoring data is stored in the Distributed Data Server (DDS), which is exactly the same as the MDS used in TRIC inside a household.

As shown in Figure 9, sensing data from sen- sors embedded in the home environment are

transmitted to the DDS, which is the core com- ponent of the PTMS. Sensing signals are then processed and stored in the DDS. Authorized remote users can request data from the DDS using an Internet Web browser (through an ap- plication server) or a Visual Basic (VB) program (direct access to the DDS). Event-driven mes- sages (mobile phone short messages or e-mails) can be sent to specified caregivers when an ur- gent situation is detected.

Using the PTMS structure described above, an “Activity of Daily Living (ADL)” monitor- ing system is under developed using the MDS of TRIC as the core to monitor the change in pattern of daily physical activities, to recognize the transition of a senior from a healthy, inde- pendent state into a state of incapacity and de- pendency, and to remind the caregivers to take early actions. Special-designed sensors embed- ded in the home environment detect activities such as eating, bathing, using the toilet, lying in bed, and watching TV. Sensor signals are transmitted to the MDS through a battery-pow- ered RF transmitter. Caregivers can read real- time sensor data, download historical data,

TSAI ET AL.422

FIG. 9. The structure of the PTMS.

perform various analyses, or set up event-dri- ven messages (mobile phone short messages or e-mail messages) once any sensor is activated.

TRIC can also become a station for vital sign parameter (VSP) measuring. Commercial VSP measuring devices can easily be integrated with the MDS. Integration of an electronic sphygmomanometer and a glucose monitor with the MDS has been successfully demon- strated. The data acquired from these two mea- suring devices can be transmitted to the MDS through RS-232 serial interface for data pro- cessing, storage, and further analyses.

One obvious advantage of using TRIC in home tele-health monitoring is its mobility and capability of interpersonal communication. Family members and caregivers can actively approach the elderly participants through TRIC to express care if abnormal signals are de- tected. Doctors and nurses can also use TRIC as a tool for tele-homecare visits. Therefore, fu- ture development of TRIC will emphasize on enhancing its functions in home tele-health monitoring.

REFERENCES

1. Sheridan TB. Telerobotics, automation and human super- visory control. Cambridge, MA: MIT Press, 1992.

2. Sheridan TB. Musing on telepresence and virtual presence. Presence: Teleoperatiors and Virtual Environ- ment, 1992:1:120–126.

3. Akin DL, Minsky ML, Thiel ED, Kurtzman CR. Space applications of automation, robotics and machine in- telligence systems (ARAMIS)—Phase II. Vols. 1–3 NASA Contractor Reports no. 3734-6, for Contract NAS8-34381; 1983.

4. Draper JV. Teleoperators for advanced manufactur- ing: Applications and human factors challenges. Int J Hum Factors Manufact 1995;5:53–85.

5. Schloerb DW. A quantitative measure of telepresence. Presence: Teleoperators and Virtual Environments 1995;4: 64–80.

6. Prassler E, Scholz J, Fiorini P. A robotics wheelchair for crowded public environment. Robotics Automa Magazine, IEEE 2001;8:38–45.

7. Kiguchi K, Tanaka T, Fukuda T. Neuro-fuzzy control of a robotic exoskeleton with EMG signals. Fuzzy Sys- tems, IEEE Trans 2004;12:481–490.

8. Wada K, Shibata T, Saito T, Tanie K. Effects of robot- assisted activity for elderly people and nurses at a day service center. Proc IEEE, 2004;92:1780–1788.

9. Kanamori M, Suzuki M, Oshiro H, Tanaka M, In- oguchi T, Takasugi H, Saito Y, Yokoyama T. Pilot

study on improvement of quality of life among el- derly using a pet-type robot. Computational Intelli- gence in Robotics and Automation, 2003. Proc 2003 IEEE Int Symp 2003;1:107–112.

10. Pollack ME, Brown L, Colbry D, et al. Autominder: An intelligent cognitive orthotic system for people with memory impairment. Robotics Auton Syst 2003; 44:273–282.

11. Lytle JM. Robot care bears for the elderly. BBC News, 21 February, 2002, 08:51.

12. Thacker PD. Physician–robot makes the rounds. JAMA 2005;293:150.

13. Fels DI, Weiss PL. Video-mediated communication in the classroom to support sick children: A case study. Int J Indust Ergonom 2001;28:251–263.

14. Roy N, Baltus G., Fox D, Gemperle F, Goetz J, Hirsch T, Margaritis D, Montemerlo M, Pineau J, Schulte J, Thrun S. Towards personal service robots for the el- derly,” Workshop on Interactive Robots and Entertain- ment (WIRE 2000), Pittsburgh, PA. 2000.

15. Tzafestas SG, Prokopiou PA. Compensation of tele- operator modeling uncertainties with a sliding mode controller. Robotics Comput Integrated Manufact 1997; 13:9–20.

16. Daniel RW, McAree PR. Fundamental limits of per- formance for force reflecting teleoperation. Int J Ro- botics Res 1998;17:811–830.

17. Stoker CR, Burch DR, Hine BP III, Barry J. Antarctic undersea exploration using a robotic submarine with a telepresence user interface. Expert, IEEE, 1995;10: 14–23.

18. Engelberger G. NASA’s robonaut. Indust Robot 2001; 28:35–39.

19. Spudis PD. The case for renewed human exploration of the Moon. Earth Moon Planets 2001;87:159–169.

20. Rehnmark F, Bluethmann W, Mehling J, Ambrose RO, Diftler M, Chu M, Necessary R. Robonaut: The “short list” of technology hurdles. Computer 2005;38:28–37.

21. Green PS, Hill JW, Jensen JF, Shah A. Telepresence surgery. Eng Med Biol Magazine, IEEE 1995;14:324–329.

22. Satava RM. Emerging technologies for surgery in the 21st century. Arch Surg 1999;134:1197–1202.

23. Schurr MO, Buess G., Neisius B, Voges U. Robotics and telemanipulation technologies for endoscopic surgery—A review of the ARTEMIS project. Surg En- dosc Other Intervent Tech 2000;14:375–381.

24. Ballantyne G.H. Robotic surgery, telerobotic surgery, telepresence, and telementoring—Review of early clinical results. Surg Endosc Other Intervent Tech 2002;16:1389–1402.

25. Burgard W, Cremers AB, Fox D, Hahnel D, Lake- meyer G., Schulz D, Steiner W, Thrun S. Experiences with an interactive museum tour-guide robot. Artific Intell 1999;114:3–55.

26. Schulz D, Burgard W, Fox D, Thrun S, Cremers AB. Web interfaces for mobile robots in public places. IEEE Robotics Automat Magazine 2000;7:48–56.

27. Tachi S, Komoriya K, Sawada K, Nishiyama T, Itoko T, Kobayashi M, Inoue K. Telexistence cockpit for hu- manoid robot control. Adv Robotics 2003;17:199–217.

DEVELOPING A TELEPRESENCE ROBOT FOR INTERPERSONAL COMMUNICATION 423

28. Trahanias P, Burgard W, Argyros A, Hahnel D, Baltzakis H, Pfaff P, Stachniss C. “TOURBOT and WebFAIR: Web-operated mobile robots for tele-pres- ence in populated exhibitions. Robotics Automat Mag- azine, IEEE 2005;12:77–89.

29. Fong T, Nourbakhsh I, Dautenhahn K. A survey of socially interactive robots. Robotics Auton Syst 2003;42: 143–166.

30. Brooker JP, Sharkey PM, Wann JP, Plooy AM. A hel- met mounted display system with active gaze control for visual telepresence. Mechatronics 1999;9:703–716.

31. Hawksford MOJ. Scalable multichannel coding with HRTF enhancement for DVD and virtual sound sys- tems. J Audio Eng Soc 2002;50:894–913.

32. Izquierdo E. Stereo matching for enhanced telepres- ence in three-dimensional videocommunications. Cir- cuits Syst Video Technol, IEEE Tran 1997;7:629–643.

33. Ohm JR, Gruneberg K, Hendriks E, Izquierdo ME, Kalivas D, Karl M, Papadimatos D, Redert A. A real- time hardware system for stereoscopic videoconfer- encing with viewpoint adaptation. Signal Process-Im- age Commun 1998;14:147–171.

34. Xu LQ, Loffler A, Sheppard PJ, Machin D. True-view videoconferencing system through 3-D impression of telepresence. BT Technol J 1999;17:59–68.

35. Lei BJ, Chang C, Hendriks EA. An efficient image- based telepresence system for videoconferencing. Cir- cuits Syst Video Technol IEEE Tran 2004;14:335–347.

36. Hopf K. An autostereoscopic display providing com- fortable viewing conditions and a high degree of telepresence. Circuits Syst Video Technol IEEE Tran 2000;10:359–365.

37. Chaffin D, Andersson G., Martin BJ. Occupational bio- mechanics, 3rd ed. New York: John Wiley & Sons, 1999.

38. Council on Aging of Ottawa. 2006, Senior Acces- sible Checklist. http://www.coaottawa.ca/library/ publications/SAHCchecklist05.pdf

39. Chiu HC, Chang HY, Mau LW, Lee Ti Kai, Liu HW. Height, Weight, and body mass index of elderly per- sons in Taiwan. J Gerentol 2000;55A:684–690.

40. Waters TR, Putz-Anderson V, Garg A, Fine LJ. Re- vised NIOSH equation for the design and evaluation of manual lifting tasks. Ergonomics 1993;36:749–776.

41. Mital A, Nicholson AS, Ayoub AA. A guide to manual materials handling. London: Taylor and Francis, 1993.

42. Voorbig AIM, Steenbekkers LPA. The composition of a graph on the decline of total body strength with age based on pushing, pulling, twisting, and gripping force. App Ergonom 2001;32:287–292.

43. Purser JL, Weinberger M, Cohen HJ, Pieper CF, Morey MC, Li T, Williams GR, Lapuerta P. Walking speed predicts health status and hospital costs for frail elderly male veterans. J Rehabil Res Dev 2005;42: 535–546.

44. Coren S. Most comfortable listening level as a func- tion of age. Ergonomics 1994;37:1269–1274.

45. Hsu YL, Yang CC, Tsai TC, Cheng CM, Wu CH. 2006. Development of a decentralized home telehealth monitoring system. Telemed e-Health. (in press).

Address reprint requests to: Yeh-Liang Hsu, Ph.D.

Department of Mechanical Engineering Yuan Ze University

135, Yuan-Tung Rd., Chung-Li Taiwan, ROC

E-mail: [email protected]

TSAI ET AL.424

This article has been cited by:

1. Jun-Ming Lu, Yeh-Liang HsuTelepresence Robots for Medical and Homecare Applications 725-735. [CrossRef] 2. Wendy Moyle, Cindy Jones, Marie Cooke, Siobhan O¿Dwyer, Billy Sung, Suzie Drummond. 2014. Connecting the person with

dementia and family: a feasibility study of a telepresence robot. BMC Geriatrics 14, 7. [CrossRef]