Psychology Expert Needed

EBSCO Publishing Citation Format: APA (American Psychological Assoc.):

NOTE: Review the instructions at http://support.ebsco.com.library.capella.edu/help/?int=ehost&lang=&

feature_id=APA and make any necessary corrections before using. Pay special attention to personal names,

capitalization, and dates. Always consult your library resources for the exact formatting and punctuation guidelines.

References

Rizzo, A. “Skip,” & Koenig, S. T. (2017). Is clinical virtual reality ready for primetime? Neuropsychology,

31(8), 877–899. https://doi-org.library.capella.edu/10.1037/neu0000405

<!--Additional Information:

Persistent link to this record (Permalink): http://library.capella.edu/login?url=http://search.ebscohost.com

/login.aspx?direct=true&db=pdh&AN=2018-03502-004&site=ehost-live&scope=site

End of citation-->

Is Clinical Virtual Reality Ready for Primetime?

By: Albert “Skip” Rizzo

Medical Virtual Reality Lab, University of Southern California Institute for Creative Technologies;

Sebastian Thomas Koenig

Director, Human Interface Technology Engineer, Katana Simulations Pty Ltd., Adelaide, Australia

Acknowledgement: The author Sebastian T. Koenig has a financial interest in two applications

mentioned in this article. As director of Katana Simulations Pty Ltd., he has ownership of the C3A

Virtual Kitchen as referenced by Wall and colleagues (2017), and the Virtual Classroom as

referenced by Blume and colleagues (2017).

For access to a large library of online videos demonstrating many of the applications discussed in

this article, go to https://www.youtube.com/user/albertskiprizzo.

Virtual reality (VR) technology offers new opportunities for clinical research, assessment, and

intervention. Since the mid-1990s, VR-based testing, training, and treatment approaches have been

developed by clinicians and researchers that would be difficult, if not impossible, to deliver using

traditional methods. During this time, a large (but still maturing) scientific literature has evolved

regarding the outcomes and effects from the use of what we now refer to as clinical VR applications

targeting cognitive, psychological, motor, and functional impairments across a wide range of clinical

health conditions. Moreover, continuing advances in the underlying enabling technologies for creating

and delivering VR applications have resulted in its widespread availability as a consumer product,

sometimes at a very low cost. So, when one studies the scientific literature, examines the evolving

state of the technology, and observes the growing enthusiasm for VR in the popular culture, a big

question emerges for psychology, neuropsychology, and rehabilitation: “Is clinical VR ready for

primetime?” Although many well-thought-out VR-based research prototypes have generated a

provocative scientific literature and a fair share of excitement, how far are we away from mainstream

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

1 of 47 2/23/2019, 4:23 PM

availability, adoption, and implementation? To address this question, the current article will briefly

describe VR technology, discuss the trajectory of clinical VR over the last 20 years, and summarize

the assets that VR offers for creating clinical applications. The discussion section addresses the

question of readiness based on an assessment of the theoretical basis for VR relevant to clinical

applications, the science to date in specific areas of use, the pragmatic factors regarding availability,

usability, and costs of clinical VR content/systems, and the ethical issues for the safe use of VR with

clinical populations. Some of the discussion in the current article includes topics that have been

discussed in previous articles, which may be consulted for additional reading (Lange et al., 2012;

Rizzo, Buckwalter, & Neumann, 1997; Rizzo, Schultheis, Kerns, & Mateer, 2004).

What Is Virtual Reality?

The concept and definition of VR has been subject to debate by scientists and clinicians over the

years. VR has been very generally defined as a way for humans to visualize, manipulate, and

interact with computers and extremely complex data (Aukstakalnis & Blatner, 1992). From this

baseline perspective, VR can be seen as an advanced form of human-computer interaction (Rizzo et

al., 1997) that allows a user to more naturally interact with computers beyond what is typically

afforded with standard mouse and keyboard interface devices. Moreover, some VR formats enable

users to become immersed within synthetic computer-generated virtual environments. However, VR

is not defined or limited by any one technological approach or hardware set up. The creation of an

engaged VR user experience can be accomplished using combinations of a wide variety of

interaction devices, sensory display systems, and content presented in the virtual environment. Thus,

there are three common variations for how VR can be created and used.

Nonimmersive VR is the most basic format and is similar to the experience of someone playing a

modern computer or console videogame. Content is delivered on a standard flat-screen computer

monitor or TV with no occlusion of the outside world. Users interact with three-dimensional (3D)

computer graphics using a gamepad, a joystick, specialized interface devices (from a treadmill to a

handheld Nintendo Wii remote), as well as basic mouse or keyboard. Modern computer games that

support user interaction and navigation within such 3D worlds, even though presented on a flat-

screen display, can technically be referred to as VR environments.

Immersive VR can be produced by the integration of computers, head-mounted displays (HMDs),

body-tracking sensors, specialized interface devices, and 3D graphics. These set-ups allow users to

operate in a computer-generated simulated world that changes in a natural or intuitive way with head

and body motion. Using an HMD that occludes the user’s view of the outside world, an engaged

immersive virtual experience employs head and body-tracking technology that senses the user’s

position and movement and sends that information to a computing system that can update the

sensory stimuli presented to the user in near real-time, contingent on user activity. This serves to

create the illusion of being immersed “in” a virtual space, within which users can interact. When

immersed within computer-generated visual imagery and sounds of a simulated virtual scene, user

interaction produces an experience that corresponds to what the individual would see and hear if the

scene were real. Another less common method for producing immersive VR experiences uses

stereoscopic projection screens arrayed around a user in various configurations. Sometimes six-

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

2 of 47 2/23/2019, 4:23 PM

walled projection rooms known as cave automatic virtual environments (CAVEs; Cruz-Neira et al.,

1993; DeFanti et al., 2011) are used that allow for interaction in a less encumbered, wide field of view

simulated environment for multiple concurrent users. However, such CAVE systems are costlier and

more complex and are typically beyond the practical resources of most clinical service providers

and/or basic researchers.

Regardless of the technical approach, the key aim of these immersive systems is to perceptually

replace the outside world with the virtual world to psychologically engage users with simulated digital

content designed to create a specific user experience. Immersive VR (most commonly delivered in

an HMD) is typically the choice for applications where a controlled stimulus environment is desirable

for constraining a user’s perceptual experience within a specific synthetic world. This format has

been often used in clinical VR applications for anxiety disorder exposure therapy, analgesic

distraction for patients undergoing acutely painful medical procedures, and in the cognitive

assessment of users to measure performance under a range of systematically delivered challenges

and distractions.

A Very Brief History of Clinical VR

VR has recently captured the public’s imagination as the next big thing in media. However, the

technology for creating VR experiences and its clinical use has existed for at least two decades.

During the 1990s the growing availability and rapid evolution of personal computing drove the global

adoption of innovative digital technologies for the purposes of productivity enhancement,

communication, and social interaction. At the same time, the advances in modern computing power

required to automate processes and store/analyze vast quantities of data did not go unnoticed by

clinical researchers and providers, who imagined and prototyped novel behavioral health care

applications. Primordial efforts from this period can be seen in developments that aimed to use

personal computers to enhance productivity in patient documentation and record-keeping, automate

the administration and scoring of psychometric tests, and in the computer-delivery of cognitive

training/rehabilitation activities (Robertson, 1990). As well, with the rapid improvements in Internet

connectivity seen during the 1990s, the idea of enhancing access to care via Internet-based

teletherapy (Cuijpers, van Straten, & Andersson, 2008; Putrino, 2014; Rizzo, Strickland, & Bouchard,

2004; Stamm, 1998) and self-help cognitive-behavioral programs (Carlbring et al., 2001; Spek et al.,

2007) was explored. Since that time, the impact of computer and information technology on society

has grown dramatically. This can be seen in the current adoption and growing demand for mobile

devices, high speed network access, smart televisions, social media sites, photorealistic digital

games, wearable behavioral sensing devices, and now, the second coming of virtual reality. Such

consumer-driven technologies, thought of as visionary just 10 years ago, have now become

increasingly common and essential fixtures in the digital landscape of a global society.

The idea of using VR for clinical purposes was first recognized in the early to mid-90s with initial

efforts to design VR simulations to deliver exposure therapy for specific phobias (e.g., fear of heights,

flying, spiders, and public speaking; Lamson, 1994; Rothbaum et al., 1995) and for cognitive

rehabilitation (Brown et al., 1998; Cromby et al., 1996; Pugnetti et al., 1995; Rizzo, 1994). The

compelling feature that drove this innovation was that VR could leverage computing beyond its

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

3 of 47 2/23/2019, 4:23 PM

cardinal purpose—the automation of processes—to instead use computers to produce and deliver

sensory stimuli for the creation of embodied, interactive, and immersive user experiences. This was

recognized early on in the visionary article “The Experience Society” by VR pioneer, Myron Krueger

(1993), in his prophetic statement, “. . . Virtual Reality arrives at a moment when computer

technology in general is moving from automating the paradigms of the past, to creating new ones for

the future.” (p. 163). Viewed from this perspective, VR afforded the opportunity to create highly

realistic, interactive, and systematically controllable stimulus environments that users could be

immersed in, and interact with, for human performance measurement and training, as well as clinical

assessment and intervention. Clinicians and scientists who were drawn to the idea of VR during this

time were often guided by the belief that its core features and assets could support the development

of innovative clinical approaches that were not possible with existing traditional methodologies.

The added value for such VR systems can be seen in the technology’s capacity to create systematic

human testing, training, teaching, and treatment environments that allow for the precise control of

complex, multisensory, dynamic 3D stimulus presentations. Within such simulations, sophisticated

behavioral interaction is possible and such physical activity can be precisely tracked, recorded, and

analyzed to study human performance and behavior. Much like an aircraft simulator serves to test

and train piloting ability under a wide variety of controlled conditions, VR can be used to create

relevant simulated environments where the assessment and treatment of cognitive, emotional, and

sensorimotor processes can take place under stimulus conditions that are not easily deliverable and

controllable in the physical world. When combining VR’s stimulus control features with the ability to

immerse users in functional and ecologically relevant virtual environments, early clinical VR scientists

envisioned a fundamental advancement in how human assessment and intervention could be

addressed. It could be conjectured that this “Ultimate Skinner Box” perspective was what human

experimental researchers and clinicians had always strived for, but were limited by the constraints

imposed by the laws of physics that govern physical reality. This “vision” drove the enthusiasm and

innovative efforts seen in the fledgling area of clinical VR in the 1990s.

Unfortunately, many technical challenges needed to be overcome before this vision of clinical VR

could be achieved. When discussion of the potential use of VR for human research and clinical

intervention first emerged in the 1990s, the technology needed to deliver on this vision was not

sufficiently mature. Consequently, during these early years VR suffered from a somewhat imbalanced

“expectation-to-delivery” ratio, as most who explored VR systems during that time will attest.

Computers were too slow, 3D graphics were primitive, and user interface devices were awkward,

requiring more effort than users were willing to expend to learn how to operate them effectively.

Moreover, VR HMDs were costly, bulky, and had limited tracking speed, resolution, and field of view.

As a consequence, VR commenced its “nuclear winter” period in 1995 as the public became

disenchanted with the quality of a typical VR experience and generally viewed it as a failed

technology. Thus, VR languished for many years in what Gartner Inc. has termed “the trough of

disillusionment,” the stage in technology adoption that often follows the “peak of inflated

expectations” period described in their regularly updated Hype Cycle for Emerging Technologies

(Gartner Inc., 2016).

Despite these technical challenges, the core vision of clinical VR was sound and VR “enthusiasts”

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

4 of 47 2/23/2019, 4:23 PM

continued to pursue the research and development needed to advance the technology and document

its added clinical value. And, over the last 22 years, the technology for creating VR systems gradually

caught up with the vision of creating compelling, usable, and effective clinical VR applications. This

has been possible in large part due to the gradual, but continuous, advances in the underlying VR-

enabling technologies and methods (e.g., computational speed, computer graphics, panoramic video,

audio/visual/haptic displays, natural user interfaces, tracking sensors, speech and language

processing, artificial intelligence, virtual human agents, authoring software, etc.). Such advances

have resulted in the technical capability needed to support the creation of low-cost, yet sophisticated,

immersive, and interactive VR systems, capable of running on commodity-level computing devices.

In part driven by the digital gaming and entertainment sectors, and a near insatiable global demand

for mobile and interactive networked consumer products, these advances in technological “prowess”

and accessibility have provided the hardware and software platforms needed to produce more

adaptable and high-fidelity clinical VR scenarios. This has created a state of affairs where clinical VR

applications can now usefully leverage the interactive and immersive assets that VR affords as the

technology continues to get faster, better, and cheaper moving forward toward the third decade of the

21st century. Moreover, since the 1990s, a significant scientific literature has evolved, almost under

the radar, reporting many positive outcomes across a range of clinical applications that have

leveraged the assets provided by VR (Botella et al., 2015; Dascal et al., 2017; Freeman et al., 2017;

Hoffman et al., 2011; Howard, 2017; Maples-Keller et al., 2017; Morina et al., 2015; Rizzo et al.,

2015a, 2015b; Rose et al., 2005; Slater & Sanchez-Vives, 2016).

A short list of the areas where clinical VR has been usefully applied includes fear reduction in

persons with specific phobias (Morina et al., 2015; Opris et al., 2012; Parsons & Rizzo, 2008a;

Powers & Emmelkamp, 2008), treatment for posttraumatic stress disorder (PTSD), depression, and

paranoid delusions (Beidel et al., 2017; Botella et al., 2015; Difede et al., 2007, 2013; Falconer et al.,

2016; Freeman et al., 2016; Rizzo et al., 2010, 2013, 2015a; Rothbaum et al., 2001, 2014),

discomfort reduction in cancer patients undergoing chemotherapy (Chirico et al., 2016; Schneider,

Kisby, & Flint, 2011), acute pain reduction during wound care and physical therapy with burn patients

(Hoffman et al., 2011) and in other painful procedures (Gold et al., 2006; Mosadeghi et al., 2016;

Tashjian et al., 2017; Trost et al., 2015), body image disturbances in patients with eating disorders

(Riva, 2011), navigation and spatial training in children and adults with motor impairments (John et

al., 2017; Rizzo et al., 2004; Stanton et al., 1998), functional skill training and motor rehabilitation in

patients with central nervous system (CNS) dysfunction (e.g., stroke, traumatic brain injury [TBI],

spinal cord injury, cerebral palsy, multiple sclerosis; Deutsch & Westcott McCoy, 2017; Holden, 2005;

Howard, 2017; Klamroth-Marganska et al., 2014; Lange et al., 2012; Merians et al., 2002, 2010), and

for the assessment and rehabilitation of attention, memory, spatial skills, and other cognitive

functions in both clinical and unimpaired populations (Bogdanova, Yee, Ho, & Cicerone, 2016;

Brooks et al., 1999; Brown et al., 1998; Matheis et al., 2007; Ogourtsova, Silva, Archambault, &

Lamontagne, 2015; Parsons, Rizzo, Rogers, & York, 2009; Passig, Tzuriel, & Eshel-Kedmi, 2016;

Pugnetti et al., 1995; Rizzo, 1994; Rizzo et al., 2006; Rose et al., 2005; Valladares-Rodriguez et al.,

2016). To do this, clinical VR scientists have constructed virtual airplanes, skyscrapers, spiders,

battlefields, social settings, beaches, fantasy worlds, and the mundane (but highly relevant)

functional environments of the schoolroom, office, home, street, and supermarket. In essence, VR

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

5 of 47 2/23/2019, 4:23 PM

environments mimicking real or imagined worlds can be applied to engage users in simulations that

support the aims and mechanics of a specific clinical assessment or therapeutic approach. As a

result, there is a growing consensus that VR has now emerged as a promising tool in many domains

of research (Bohil et al., 2011; Larson, Feigon, Gagliardo, & Dvorkin, 2014) and clinical care

(Freeman et al., 2016; Goldman-Sachs, 2016; Lange et al., 2012; Norcross et al., 2013).

Analysis of Clinical VR Assets

What makes clinical VR so distinctively innovative is that it represents more than a simple linear

extension of existing computer technology for human use. By way of VRs capacity to immerse a user

within an interactive computer-generated simulation, new possibilities exist that can go beyond the

simple automation of previous clinical assessment and intervention approaches. Nevertheless, in

deciding as to whether clinical VR is ready for primetime, one needs to consider what features VR

offers that may make it especially suited for clinical and research usage.

On a very general level VR can be seen to foster core processes that are relevant across a variety of

clinical domains. These processes can be briefly summarized as expose (e.g., exposure therapy for

anxiety disorders, PTSD, or addiction treatment), distract (e.g., distracting attention away from painful

medical procedures to reduce pain perception or promote discomfort reduction), motivate (e.g.,

motivating clients in cognitive/physical rehabilitation to perform repetitive and sometimes boring tasks

by embedding them within game-like contexts), measure (e.g., measuring performance on

physical/cognitive assessment activities), and engage (e.g., generally seen as the captivation of

attention/action that is useful for engaging participation with clinical applications). To effectively drive

these processes in a thoughtful fashion, it is helpful to be aware of the features and assets that are

available for clinical use of VR technology. These assets have been specifically detailed as they

relate to the predecessor field of aviation simulation technology (Jentsch & Curtis, 2017) and an

earlier detailing of these assets for neuropsychology appeared in Rizzo et al. (2004). However, in

view of the rapidly advancing state of VR technology, a revisiting of its current status is warranted,

especially as it pertains to general clinical applications.

Ecological Relevance

Clinical VR scenarios can be modeled after relevant contexts that exist in everyday life. Within such

simulated environments, it is possible to create activities that mimic challenges faced by clinical

populations, and implement them as part of assessment and intervention strategies. This has been a

guiding feature in clinical VR development since the 1990s, leading to the creation of many standard

archetypic testing and treatment spaces (e.g., homes, offices, classrooms, stores, tall buildings, cars,

battlefields, hospital settings, social gatherings, public speaking auditoriums, etc.). The primary driver

for these efforts is the view that we can better predict or enhance human functioning (e.g., behavioral

outcomes, emotional coping, cognitive/motor task performance) in the real world by providing

systematic and highly controllable assessments and interventions within functionally similar virtual

worlds.

This is particularly relevant in view of the underlying concepts of generalization and transfer of

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

6 of 47 2/23/2019, 4:23 PM

training that have been “big” issues across all domains of psychology and rehabilitation. For

example, traditional neuropsychological assessment and rehabilitation has been criticized by some

authors (Parsons, Carlew, Magtoto, & Stonecipher, 2015; Rizzo, Buckwalter, & Neumann, 1997;

Sbordone, & Long, 1996; Wilson, 1997) as limited in the area of ecological validity, that is, the degree

of relevance or similarity that a test or training system has relative to the real world (Neisser, 1978). A

number of examples illustrate efforts to enhance the ecological validity of assessment and

rehabilitation by designing VR scenarios that are “replicas” of relevant archetypic functional

environments. This has included the creation of virtual cities (Brown et al., 1998; Costas, Carvalho, &

de Aragon, 2000; Gamito et al., 2016), supermarkets (Cromby et al., 1996; Josman et al., 2014; Levy

et al., 2015), homes (Koenig, 2012; Rose et al., 2001), kitchens (Christiansen et al., 1998; Davies et

al., 1998; Foloppe et al., 2015; Wall et al., 2017), school environments (Rizzo et al., 2000, 2006;

Stanton et al., 1998), workspaces/offices (Koenig et al., 2012; Krch et al., 2013; Matheis et al., 2007;

McGeorge et al., 2001), rehabilitation wards (Brooks et al., 1999), and even a virtual beach (Elkind et

al., 2001). From these efforts, recent reviews have provided support for the impact of ecologically

relevant clinical VR applications on real-world treatment outcomes in both clinical psychology (Morina

et al., 2015) and in rehabilitation (Howard, 2017).

Although early attempts at the creation of these environments varied significantly in their level of

pictorial or graphical realism, this fidelity factor may be secondary in importance, relative to the actual

activities that are carried out in the environment for determining their value from an ecological

relevance standpoint (cf. Parsons, 2015; Rizzo et al., 2006). Interestingly, when in a virtual

environment, humans often times display a high capacity to “suspend disbelief” and respond as if the

scenario was real. It could be conjectured that the “ecological value” of a VR task that needs to be

performed may be well supported despite limited graphical realism. In essence, as long as the VR

scenario “resembles” the real world, possesses design elements that replicate key real-life

challenges, and the system responds well to user interaction, then the graphical realism can be less

important for activating behavior and emotion. This has especially been observed by clinicians using

VR to conduct exposure-based therapies for anxiety disorders, PTSD, and addiction (Bordnick et al.,

2013). Clients commonly report significant emotional activation despite the “cartoonish” nature of the

visual content seen in some VR scenarios. Thus, while a number of the successful VR scenarios

designed for exposure-based therapy of specific phobias would never be mistaken for the real world,

clients experiencing these VR worlds still manifest physiological responses and report subjective

units of discomfort levels that suggest they are responding as if they are in the presence of the

feared stimuli (Costanzo et al., 2014; Norrholm et al., 2016; Wiederhold & Wiederhold, 1998).

The recent advances in computer graphics as seen in modern computer games have now made the

“fidelity” issue less of a concern. As well, the growing popularity of panoramic 360-degree cameras

and photogrammetry has provided an affordable means to create photorealistic content for VR

applications. Although expectations of computer graphics have also increased steadily, especially

with a younger generation that has grown up with computer and console games and may be put off

by low-quality graphics, perceptually convincing VR scenarios are now more the norm than the

exception in current VR development. Although it is yet to be documented that increased realism has

had an impact on improving clinical outcomes, the ability to create more compelling visual VR

content may, at the very least, improve face validity and increase user buy-in from patients and

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

7 of 47 2/23/2019, 4:23 PM

clinical end-users.

Systematic Delivery and Control of Sensory Stimuli

One of the cardinal assets of any advanced form of simulation technology involves the capacity for

systematic delivery and control of stimuli. This asset provides significant opportunities for developing

clinical VR methods. In fact, one could conjecture that the systematic delivery and control of stimuli in

a testing or treatment environment provides the basic foundation of all human research and clinical

methodologies along with the subsequent capture and analysis of the behavior that occurs in

response to those conditions. In this regard, an ideal match exists between the stimulus delivery

assets of VR simulation systems and the requirements of any clinical assessment and intervention

procedure. This can be seen as a core asset whether one is testing construct-specific cognitive

processes (e.g., selective attention performance contingent on varying levels of stimulus intensity

and distraction; Rizzo et al., 2006; Mühlberger et al., 2016), to the complex targeting of more molar

functional behaviors (e.g., planning and initiating the steps to function within a complex office or

home setting; Keefe et al., 2016; Krch et al., 2013), to the precise titration of anxiety activating

content in the service of pacing exposure therapy for the treatment of phobias or PTSD (Rizzo et al.,

2015a; Rothbaum et al., 1995, 1999).

Moreover, the precise control over multiple concurrent tasks and presentation of realistic distractions

during these tasks presents a unique opportunity to simulate complex, lifelike scenarios that is only

starting to receive attention in clinical VR research and development. This approach stands in stark

contrast to the traditional single-construct exposure to cognitive tasks in distraction-free

environments such as a clinician’s office.

This capacity for systematic stimulus control within the context of ecologically relevant simulations of

everyday life (i.e., The Ultimate Skinner Box) for assessment and intervention purposes is one of the

key areas that differentiate clinical VR from all previous methodologies. Thus, VR’s stimulus delivery

capability has been recognized as a significant asset for supporting the integration of VR with brain

imaging and psychophysiology studies (Bohil et al., 2011; Chou et al., 2012; Costanzo et al., 2014;

Norrholm et al., 2016; Tarr & Warren, 2002). This is especially relevant for the field of

neuropsychology which has been increasingly integrating advanced neural imaging technologies in

its quest for a better accounting of the structure and processes underlying brain/behavior

relationships. In fact, the use of VR in imaging studies has nearly as long a history as the direct use

of the technology for clinical interventions (Astur, Ortiz, & Sutherland, 1998).

For example, a VR simulation of the Morris Water Maze test of spatial navigation and place learning,

commonly used with rodents, has generated significant research examining the role of the

hippocampus in human learning (Astur et al., 1998, 2002, 2004). In this elegant and well-matched

use of VR, a human user must navigate a space to find a hidden platform using visual cues in the

surrounding environment. Used in conjunction with fMRI, the test has been applied to assess place

learning performance while concurrently measuring hippocampal activity. Research with this VR

system has reported poorer performance and decreased activation in health conditions where the

hippocampus is implicated such as with Alzheimer’s disease (Shipman & Astur, 2008), PTSD (Astur

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

8 of 47 2/23/2019, 4:23 PM

et al., 2006), and schizophrenia (Folley et al., 2010). Other researchers have similarly integrated VR

and brain imaging and have reported, reduced activation of pain-related regions of interest using VR

as a distractor from experimentally induced pain (Hoffman et al., 2006, 2011), changes in brain

activation (i.e., amygdala and 3 frontal areas) to VR stimuli following exposure therapy for PTSD

(Roy et al., 2010, 2014), neural predictors of change in emotion recognition in persons on the autism

spectrum using VR social cognition training (Yang et al., 2017), and cortical reorganization and

associated locomotor recovery in chronic stroke patients with VR game-based rehabilitation (You et

al., 2005). In a recent effort to combine a virtual classroom scenario with near-infrared spectroscopy

(NIRS), Blume et al. (2017), in collaboration with Katana Simulations, created an immersive virtual

classroom neurofeedback training to treat deficits associated with attention-deficit/hyperactivity

disorder (ADHD; Blume et al., 2017). A clinical trial is currently evaluating the efficacy of the training,

that utilizes the NIRS signal to control the classroom’s lighting intensity as a feedback mechanism. It

is hypothesized that a training protocol of 15 sessions, containing activation and deactivation trials,

will facilitate self-regulation skills, and improve ADHD symptoms and motor activity in the

participating 90 children with ADHD. Participants are randomly assigned to the NIRS-based training

in the VR classroom, a NIRS-based training in a 2D classroom, or an electromyogram-based training

in VR. This clinical trial is ongoing.

Although head movement is restricted in most brain imaging systems (excluding NIRS), specialized

“magnet-friendly” interaction devices and displays can still allow users to engage with dynamic virtual

content, albeit the experience is different than a typical unrestricted VR application. With that

limitation acknowledged, the integration of VR as a tool for delivering complex, interactive stimuli with

advanced brain imaging techniques may support neuropsychology in reaching its stated purpose,

that of determining unequivocal brain-behavior relationships, in addition to advancing the state of the

science in other clinical disciplines.

Delivery of Strategic Real-Time Performance Feedback

VR simulations can be designed to provide users with feedback as to the state of their performance

during task practice (knowledge of performance) and after task completion (knowledge of results;

Levin, Weiss, & Keshner, 2015). A primary aim is to promote behavioral calibration of the clients’

actions using clear signals that indicate their status toward achieving performance outcomes.

However, careful consideration needs to be placed on the use of positive and negative feedback

during and after correct and incorrect performance to balance short-term and long-term goals as they

relate to user motivation and task performance (Burgers et al., 2015). Delivery of feedback stimuli

can appear in graded (degree) or absolute (correct/incorrect) forms and can be presented via

auditory, visual, or tactile sensory modalities depending on the goals of the application and the needs

of the user. Moreover, feedback can be inherent to the task and the way the user’s actions are

represented in the VR environment. For example, representing the user’s hands through virtual

models is also a form of feedback, providing real-time information about the user’s movements. This

feedback can be modulated, such as exaggerating, dampening, slowing down, speeding up, or even

mirroring movement (e.g., Regenbrecht et al., 2014), depending on the user’s therapeutic goals.

Feedback delivery is an intuitively essential component for rehabilitation efforts as it is generally

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

9 of 47 2/23/2019, 4:23 PM

accepted to be necessary for most forms of learning or skill acquisition (Levin, Weiss, & Keshner,

2015; Sohlberg & Mateer, 1989, 2001). Although VR-based feedback can be presented to signal

performance status in a form that wouldn’t naturally occur in the real world (e.g., a soft tone indicating

a correct response), more relevant or naturalistic sounds can also be creatively applied to enhance

both ecological relevance and the believability of the scenario. For example, in a VR application

designed to help children with learning disabilities practice escape from a house fire (Strickland,

2001), the sound of a smoke detector alarm raises in volume as the child gets near to the fire’s

location. As the child successfully navigates to safety, the alarm fades contingent on her choosing

the correct escape route. More recently, Jin et al. (2016) have implemented a biofeedback

methodology for users aiming to reduce chronic pain via treadmill interaction within a virtual forest

walking task. As users lower their level of skin conductance level (as part of an effort to teach

relaxation and mindfulness strategies), the “fog” within the forest gradually lifts to reveal an engaging

and idyllic wilderness setting. Physical rehabilitation applications have also leveraged the strategic

delivery of performance feedback to enhance relearning of upper extremity abilities following stroke

or traumatic brain injury (Adamovich et al., 2009; Badia et al., 2016; Deutsch, Latonio, Burdea, &

Boian, 2001; Jack et al., 2001; Klamroth-Marganska et al., 2014).

Delivery of Cueing Stimuli to Guide Successful Performance and Impact Behavior

The capacity for dynamic stimulus delivery and control within a virtual environment also supports the

presentation of cueing stimuli that can be used to guide user performance. This is especially relevant

for cognitive rehabilitation applications that implement “error-free” learning strategies. Error-free

training in contrast to trial-and-error learning has been shown to be successful in a number of non-

VR investigations with diverse test populations including persons with developmental disabilities,

schizophrenia, and CNS disorders (Fish et al., 2015; Wilson et al., 1994, 1996). This asset

underscores the idea that for some clinical approaches it may not be desirable for VR to simply

mimic reality with all its opportunities for error. Instead, cueing stimuli features that are not easily

deliverable in the real world can be presented in the virtual world to help guide and train successful

performance. In this special case of stimulus delivery, cues are given to the user prior to a response

in order to help guide successful error-free performance.

Although the use of cueing to support errorless learning is compelling and can now be easily

programmed as a feature within VR simulations, it has rarely been applied and tested in VR contexts.

In the only VR-based head-to-head comparison of this type, Connor, Wing, Humphreys, Bracewell,

and Harvey (2002), reported on a series of case studies of users with TBI operating a haptic joystick-

mediated “Trails B” type training task. In the error-free condition, the haptic joystick restricted

movement on the nonimmersive Trails task such that the user was not allowed to make navigation

errors. Mixed findings were reported, but error-free training resulted in significant response speed

improvements compared with trial-and-error training in some cases. In a case report, Brooks et al.

(1999) used error-free training for wayfinding in a rehabilitation ward as one component in a VR

training system that produced positive transfer to the real ward. Harrison et al. (2002) also reported

the use of cueing stimuli in a VR system designed to train maneuverability and route-finding in novice

motorized wheelchair users. Arrows were presented to trainees with the caption “Go this way” to

guide successful route navigation whenever the user would stray into areas where invisible “collision

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

10 of 47 2/23/2019, 4:23 PM

boxes” were programmed in the environment. Two patients with severe memory impairments took

part in route finding training over the course of seven days with the patients successfully learning two

subsections of the test routes while failing to eradicate errors on two further subsections of the

routes. Cueing was also incorporated into a VR system designed for executive function assessment

and training in the context of a series of food preparation tasks within a virtual kitchen scenario

(Christiansen et al., 1998). This VR scenario assessed the ability to perform 30 discrete steps

required to prepare a can of soup and make a sandwich using both visual and auditory cues to

prompt successful performance. However, the specific effect of this cueing was not isolated, nor was

a system in place to prevent errors from actually occurring. Finally, a more recent case report has

shown positive gains in a user with Alzheimer’s disease using a similar virtual kitchen (Foloppe et al.,

2015). Generally, it appears that the use of cueing stimuli to support error-free VR rehabilitation is

promising in concept but there is currently only limited research support with its use in VR. However,

while empirical support is still lacking, the ease for programming these components within VR make it

an appealing feature to test more rigorously in future research.

Beyond errorless learning for cognitive rehabilitation, perhaps the use of verbal cueing could be

applied for cognitive-behavioral approaches that address self-talk within provocative VR settings. For

example, if key prompting statements could be specified in advance, users could prerecord

supportive self-talk cues in their own voice. These cues could then be played back to the user in a

modulated “dreamlike” vocal tone during strategic points within a socially stressful VR scenario

designed to help users deal with anger management, social phobia, or shyness issues. This form of

natural “inner voice” guidance might be useful for self-talk methods within virtual social scenarios with

the aim to improve generalization of the user’s self-generated sub vocal cognitions that could

facilitate more optimal social interaction in the real world.

A dramatic extension of this type of proposed self-cueing feature worth mentioning involves recent

innovative VR efforts to address the cognitive distortions of persons with depression (Falconer et al.,

2016). With the goal of improving “self-compassion”, clinically depressed users were invited to enter

a virtual world for 8 min where they were requested to “console” a distressed virtual child using

tactics on which they had received prior coaching. After a short period of time, the user was switched

into the role (and virtual body) of the child and presented with a replay of their own attempts at

consoling the child. The replay was delivered by an adult virtual representation of themselves that

expressed their own consoling words back to them in their own voice captured from their previous

verbalization and behavioral activity with the virtual child. In a small initial trial (n = 15), after three

repetitions of this body-swapping scenario, significant reductions were measured in depression

severity and self-criticism, along with a significant increase in self-compassion, from baseline to

4-week follow-up. Four patients showed clinically significant improvement. Although this effect should

still be considered preliminary, it does underscore the potential for clinical VR to present

sophisticated cueing content, in this case a fully naturalistic rendition of self-delivered self-

compassion, that produced significant emotional impact on users in a fashion that would be near

impossible to deliver with previously existing methods.

Behavioral Performance Capture and Retrospective and Intuitive After Action Reviews

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

11 of 47 2/23/2019, 4:23 PM

The review of a client’s behavioral performance in any assessment and training activity typically

involves examination of numeric data and subsequent translation of that information into graphic

representations in the form of tables and graphs. Sometimes videotaping of the actual event is used

for a more naturalistic review and for behavior rating purposes. These methods, though of value, are

typically quite labor intensive to produce and sometimes result in a less than intuitive method for

visualizing and understanding a complex performance record. These challenges are compounded

when the goal of the review is to provide feedback and insight to clients whose impairments may

preclude a useful understanding of traditional forms of data presentation. VR offers the capability to

capture and review a complete digital record of performance in a virtual environment from many

perspectives. For example, performance in a virtual environment can be later observed from the

perspective of the user, from the view of a third party or position within the scenario, and from what is

sometimes called a “God’s eye view” that is observed from above the scene with options to adjust

the position and scale of the view. This can allow a client to observe and repeatedly review their

performance from multiple perspectives. Options for this review also include the modulation of

presentation as in allowing the client to slow down rate of activity and observe each behavioral step

in the sequence in “slow motion” (Rizzo et al., 2004).

Advanced programs that incorporate such methods have been in steady use by the military to

conduct After Action Reviews (AAR; Morrison & Meliza, 1999). In military VR applications, which

often include multiple participants in a shared virtual space, a computerized AAR tool can allow the

behavior of any participant to be reviewed from multiple vantage points at any temporal point in the

digital training exercise. This is now standard procedure for military simulation training but has had

limited application in clinical VR approaches. With the exception of less naturalistic review of paper-

and-pencil results and the occasional review of a client’s videotaped performance from fixed camera

positions, the capacity to provide more intuitive “first-person” perspective views to clients has not

been feasible with existing technology, and thus VR now provides a powerful asset in this area

(Rizzo et al., 2004).

Early efforts to leverage this VR asset appeared as a feature for reviewing navigational performance

in a number of wayfinding and place learning applications (Astur et al., 1998; Jacobs, Laurance, &

Thomas, 1997; Kober et al., 2013; Koenig, Crucian, Dalrymple-Alford, & Dünser, 2010; Skelton et al.,

2000). This has primarily been used in applications where a tracked movement record is vital for

measuring and visualizing the dependent variable of exploratory behavior. A review method was also

developed for replaying a child’s head movements while they are tracking stimuli within a virtual

classroom in a VR assessment of attention (Rizzo et al., 2006). This application took data from a

tracking device positioned on top of the VR HMD and represented the captured movement via a

virtual representation of a person’s head. The virtual head is rendered to face outward from the

screen and a “straightforward” head position represents the attentive gaze at the virtual blackboard

where target hit stimuli are displayed to the child. During video playback after a test session, it is

possible to observe the child’s head movements during discrete periods when distracting stimuli are

presented around the classroom (see https://youtu.be/BQyO3oDMKbI). Head movements away from

the center of the screen represent the child’s actual movements to follow the distracting stimuli on

each side of the classroom instead of the face forward position required to view the target stimuli.

This playback format delivers an extremely intuitive understanding of the distractibility of children

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

12 of 47 2/23/2019, 4:23 PM

diagnosed with attention-deficit/hyperactivity disorder (ADHD) during VR classroom performance

testing that were revealed from the complex statistical analyses of this movement data. The provision

of this type of intuitive behavioral visualization could serve to improve the understanding of the

behavior of an ADHD child by professionals, parents, and perhaps even the tested child in a manner

not possible with graphs and data tables (Rizzo et al., 2004, 2006). Systematic studies of the clinical

use of this form of performance record review have yet to appear in the literature, although this form

of visualization asset illustrates how VR may add value for assessment and intervention that is not

readily available with existing traditional tools.

The “Pause Button” for Midsession Review and Analysis With the Clinician

In any intervention that activates cognitive, behavioral, and emotional processes for a clinical

purpose, clinician review/feedback is an essential component for building a therapeutic alliance and

fostering clients’ self-awareness. Although feedback can be delivered digitally within a simulation for

guiding real-time performance, and retrospectively for past performance review (see previous three

assets), clinical VR interactions can be paused and restarted at precisely the next moment in the

digital sequence or replayed from an earlier juncture for the purposes of face-to-face therapist

engagement/support as needed. It is easy to think of VR as an all-encompassing computerized

environment that delivers all the ingredients for good intervention, but that would be naïve. Rather,

the use of such potent and emotionally evocative simulations should be viewed simply as tools for

extending the skills of a well-trained clinician and as a method that may amplify client engagement

with a therapeutic process that is known to have efficacy in a real-world delivery context. From this

view, the capacity to pause a simulation to engage in clinical dialog at strategic junctures is a

distinction that is often overlooked due to its simplicity. This functionality has relevance across all

areas of clinical intervention and needs to be specifically designed during the VR development

process to augment the therapist-client relationship instead of hindering it.

Specifically, immediate therapist response to client performance is one form of feedback that is

commonly seen in the rehabilitation of clinical populations. This may be of particular value for clinical

populations who have memory difficulties that require more frequent review and feedback during a

training session. Although pausing is of course possible with any assessment or intervention

approach, VRs unique assets offer the opportunity to pause or “freeze time” in the middle of a

functional real-world simulated task. This can result in additive learning benefits, whereby you can

“stop and evaluate” not only individual performance, but also by examining what environmental

elements may be affecting performance. For example, during activities in a VR kitchen for the

completion of a simple task (i.e., making a can of soup), performance may be paused for the

correction of errors (missed procedure steps), evaluation of safety elements of the task (where are

the sharp objects) or discussion of perceptual difficulties (inappropriate visual scanning; Rizzo et al.,

2004). The simulation can then be restarted or backed up to an earlier point to allow for a “redo.”

Similarly, for psychological treatment, when a user is immersed within a provocative simulation where

they are confronting a digital recreation of a traumatic event or an environment designed to deliver

anger or addictive behavior cues, the simulation can be paused for direct coping strategy coaching

with the clinician.

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

13 of 47 2/23/2019, 4:23 PM

Thus, the ability to pause performance “mid-digital stream” allows a clinician to intervene strategically

to enhance client processing and discussion of decision-making, memory strategies, coping

behaviors, assertive language, cognitive restructuring, or any of the myriad clinical tactics that are

commonly applied as the elements of quality evidence-based (and empathy-based) therapeutic care.

Contrary to some of the negative concerns we have heard expressed over the years regarding the

use of technology in clinical practice (“It puts a barrier between the therapist and the client”), the

ability to pause (and later restart) a client’s simulated experience for a direct clinical intervention may

actually serve to remove a key barrier—the lack of an immediate shared experience. Therapy can

involve a lot of discussion of abstract concepts that sometimes do not lend themselves to an easily

shared understanding of the client’s experience of everyday life. A clinician who has the opportunity

for close observation of the client’s behavior within an emotionally or cognitively challenging VR

simulation, and who then can pause it to provide strategic support or reflection, may have an edge

for developing a closer understanding of the client. This edge may reside in the clinician’s newfound

ability to observe the client as they address a challenge that would have previously remained unseen

by the clinician due to its exclusive occurrence outside of the therapy office.

Safe Testing and Training Environments That Minimize the Risks Due to Errors

This is an area where clinical VR provides an obvious asset by creating options for users with

cognitive or sensorimotor impairments to be tested and trained in the safety of a simulated digital

environment. The value of this has already been amply demonstrated in the predecessor field of

aviation simulator research where actual flying accidents dropped precipitously following the early

introduction of even very crude aircraft simulation training (Johnston, 1995). Early on in the clinical

VR domain, this asset served as a driving force for VR system design and research with both clinical

and unimpaired populations. For example, the simple (but potentially dangerous) act of street

crossing has been tested and trained in VR with unimpaired children (McComas, MacKay, & Pivik,

2002; Morrongiello, Corbett, Switzer, & Hall, 2015; Schwebel, McClure, & Severson, 2014),

populations with learning and developmental disabilities (Brown et al., 1998; Josman, Ben-Chaim,

Friedrich, & Weiss, 2008; Strickland, 2001), and adult TBI and stroke groups with neglect (Navarro et

al., 2013; Naveh, Katz, & Weiss, 2000). Other relevant application areas include kitchen safety

(Rose, Brooks, & Attree, 2000), escape from a burning house with children on the autism spectrum

(Strickland, 2001), preventing falls with at risk elderly (Jaffe, Brown, Pierson-Carey, Buckley, & Lew,

2004; Neri et al., 2017), use of public transportation (Mowafty et al., 1995), and driving with a range

of clinical populations (Akinwuntan, Wachtel, & Rosen, 2012; Liu, Miyazaki, & Watson, 1999;

Pietrzak, Pullman, & McGuire, 2014; Rizzo, Reinach, McGehee, & Dawson, 1997; Schultheis &

Mourant, 2001). And, more recently, there has been an increased interest in VR driving applications

to reduce risk in both novice and aged populations (Casutt, Martin, Keller, & Jäncke, 2014; Cox et al.,

2015). In addition to the goal of promoting safe performance in the real world, some researchers

have reported positive results for building a more rational client self-awareness of deficits using a VR

approach. For example, Davis and Wachtel (2000), have reported a number of instances where older

adults, poststroke, had decided not to continue making a return to driving a primary immediate goal

after they had spent time in a challenging VR driving system.

Finally, one concern that may exist with this asset involves the potential that practice of activities that

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

14 of 47 2/23/2019, 4:23 PM

are dangerous in real life, within the safety of a virtual environment, might create a false sense of

security or omnipotence that would put the client at risk upon subsequent action in the real world. In

essence, can safe transfer of training occur in the real world when the consequences of errors are

prevented from occurring in VR? This is a very challenging concern that needs careful consideration.

Perhaps, one option would be to provide a noxious sound cue, contingent on the occurrence of

dangerous errors in VR, as a means to condition a proper attitude of caution in clients. This concern

further underscores the need for a professional to closely monitor client activity in order to recognize

possible patterns of risk taking behavior that could emerge when using VR (Rizzo et al., 2004).

Independent Practice of Therapeutic Activities Outside of the Clinic

Independent home-based physical therapy or cognitive training by clients following a stroke or TBI is

a common and highly recommended component for most approaches to rehabilitation. Similarly, with

standard cognitive-behavioral therapy (CBT) for psychological disorders, it is generally accepted that

by having clients do between-session “homework”, that generalization of skills learned in therapy

session will be promoted in everyday life. Thus, clients are routinely encouraged to engage in

clinician-recommended therapeutic activities independently as part of a general approach to clinical

care. Up until the last few years, access to VR technology for supporting clinical care outside of the

clinic was a hopeful vision, but very limited by the immature state of the technology. Consequently,

there is very little research on the additive value of home-based VR for bolstering clinic-based

interventions on clinical outcomes.

Researchers over the last 20 years have proposed and tested various configurations for pushing VR

game-based physical rehabilitation into home-based systems (Piron et al., 2002; Proffitt & Lange,

2015; Standen et al., 2014). However, as compelling as this idea sounds in concept, limitations due

to the cost of equipment and complexity of set up and use limited the general adoption of this

approach. One challenge for physical rehabilitation early on was seen in the need for specialized

interface devices and body tracking systems required to foster interaction with virtual rehabilitation

task content. This has been somewhat minimized in recent years with various commercially available

camera-based 3D tracking systems like the Microsoft Kinect or the Leap Motion sensor. There are a

number of commercial and noncommercial entities that develop such VR systems based on low-cost

sensors, but primarily they have been focused on clinic-based use (Faria, Andrade, Soares, & Badia,

2016; MindMaze, 2017; SilverFit, 2017). Movement of these systems into the homes of users for

independent practice and online tracking of use/performance by a supervising clinician is only

starting to become technically feasible and future effort in this area is expected to accelerate,

especially in view of the positive findings that have emerged from studies of in-clinic use (Howard,

2017; Klamroth-Marganska et al., 2014).

Efforts to use immersive VR for CBT home-based activities have been similarly hampered by cost

and complexity issues. That is also expected to change in the near future as low-cost VR HMDs that

are easy to operate are now coming into the marketplace. This is in part due to the widespread

access to personal computing, previously limited to standalone computers, but now bolstered by the

ubiquitous presence of mobile phones/devices. Thus, access to computing power is no longer a

significant bottleneck for supporting independent clinical VR practice. Moreover, access to a VR HMD

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

15 of 47 2/23/2019, 4:23 PM

for personal use is no longer a limiting factor as new technology has accelerated the availability and

adoption of low-cost consumer level HMDs. This can be seen in the rapid developments in mobile

phone enabled HMDs. Products such as the Samsung Gear VR or the Google Daydream offer fairly

good fidelity at the price of a mobile phone and a HMD housing, costing less than $100, into which

the phone can be inserted to create a working VR headset. These systems are easy to use and there

is content that is becoming readily available that can be applied for clinical purposes. For example,

low-cost “fear of public speaking” VR software is readily downloadable (Hypergrid Business, 2016)

for these systems. The software allows users to practice their speaking skills in front of a wide range

of virtual audiences along with the presentation of public speaking coaching content. However,

although self-treatment for this form of anxiety when viewed as a skill training intervention appears

on the surface to be relatively benign, it does open the door to other types of self-help VR anxiety

disorder applications. This state of affairs will require a deeper analysis as to the ethical use of such

emotionally evocative software and the issues surrounding VR self-help will be discussed later in this

article.

Adaptable User Interfaces and Sensory Displays to Promote Access

The emerging human computer interaction research area referred to as “3D User Interaction”

(LaViola et al., 2017) recognizes that interaction with VR content requires thoughtful attention to both

design principles and the needs of the targeted user groups. This is especially relevant for clinical

users with sensory or motor impairments where their capacity to receive value from a VR

assessment or rehabilitation approach is always governed by their ability to interact with the VR

content (Rizzo et al., 2004). Although an extensive literature exists in the area of interface design for

persons with disabilities (Barrett, McCrindle, Cook, & Booy, 2002; Darejeh & Singh, 2013; Lanyi et

al., 2012), a full discussion of that area is beyond the scope of this article. However, because VR

content can be interacted with using a wide variety of adaptive interface devices, we will briefly

address how that capability can be leveraged as an asset for clinical VR. This is particularly relevant

as sensory and motor impairments are commonly seen in persons with CNS dysfunction. A question

that often arises in assessment and rehabilitation, concerns the degree to which a client’s

performance reflects CNS-based cognitive dysfunction versus artifacts due to sensorimotor

impairments. VR offers two ways in which this challenge may be addressed in the testing and

training of cognitive and everyday functional abilities in persons with sensorimotor impairments.

One approach places emphasis on the use of adapted human computer interface devices for VR

interaction. Such devices can allow a user with significant motor impairments to interact with VR

assessment and training content, beyond what is possible for similar clinical activities in the physical

world. Interface adaptations can support interaction by leveraging alternative or augmented

movement, speech, expired air, tracked eye movement, and by way of recent advances in brain

computer interfaces (Kaplan et al., 2013; Millan et al., 2010; Remsik et al., 2016). One very basic

example involves the use of a gaming joystick to navigate in a VR scenario modeled after an

amnestic client’s rehabilitation unit that was found to be effective for teaching wayfinding around the

real unit (Brooks et al., 1999). The authors partially attributed the observed positive training effects to

the client’s capability for quicker traversing of the VR training world using a joystick compared with

what her ambulatory impairments (using a walker) would allow in the real environment. This strategy

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

16 of 47 2/23/2019, 4:23 PM

supported efficient use of training time by increasing the number of training trials that were possible

(i.e., 10 trials in VR in the time it would take to complete one with the walker). Quite simply, by

minimizing the impact of the user’s ambulatory impairments, CNS wayfinding functions could be

more efficiently trained.

A second approach can be seen in efforts to tailor the sensory modality of the stimuli presented in

the VR world around the needs of persons with visual impairments. The few efforts in this area have

mainly attempted to build simulated structures around the use of enhanced immersive 3D audio

(Lumbreras & Sanchez, 2000) and tactile stimuli (Connor, 2002). For example, Lumbreras et al.

(2000), aiming to design computer games for blind children, created a 3D audio VR system referred

to as “AudioDOOM.” In this application, blind children used a specialized joystick to navigate the

mazelike game environment exclusively on the basis of 3D audio cues (e.g., footstep sounds, doors

that “creak” open, echoes) while chasing “monsters” around the environment. Following varied

periods of time in the virtual environment, the children are then given Legos to construct their

impression of the structure of the layout. The resulting Lego constructions were noteworthy in their

striking resemblance to the actual structure of the audio-based layout of the maze. Children using

this system (who never actually have “seen” the physical visual world) were able to use the 3D sound

cues to create a spatial-cognitive map of the space and then accurately represent this space with

physical objects (i.e., Legos, clay, sand). Examples of some of these constructions are available on

the Internet (http://www.dcc.uchile.cl/~mlumbrer/audiodoom/audiodoom.html). Such adaptive

interaction approaches in VR offer the potential for factoring out sensorimotor impairments that can

confound clear assessment or rehabilitation of functioning in a way that might not be feasible or valid

within the constraints of the physical world.

Virtual Humans for Addressing Social Interaction and Training

The feasibility for creating clinical VR applications has advanced in part due to substantial progress

in 3D computer graphics rendering that now support the creation of ever more believable context-

relevant “structural” VR environments (e.g., combat scenes, homes, classrooms, offices, markets) for

clinical purposes. However, the next stage in the evolution of clinical VR will involve populating these

environments with virtual human (VH) representations that can engage real human users in credible

and useful interactions. This capability has been around since the 1990s, but the previous limitations

in graphical rendering, natural language processing, speech recognition, and face and gesture

animation made the creation of credible VHs for interaction a costly and labor-intensive process.

Thus, until recently, VHs existed primarily in the domain of high-end special effect studios that

catered to the film or game industry, far from the reach of those who thought to employ them in

clinical health applications.

This is not to say that representations of human forms have not previously appeared in clinical VR

scenarios. In fact, since the mid-1990s, VR applications have routinely employed “primitive” VHs

(e.g., low fidelity graphics, nonlanguage interactive, limited face and gesture expression) to serve as

stimulus elements to enhance the realism of a virtual world simply by their static presence. For

example, VR exposure therapy applications for the treatment of specific phobias (e.g., fear of public

speaking, social phobia) were successfully deployed using immersive simulations that were inhabited

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

17 of 47 2/23/2019, 4:23 PM

by “still-life” rendered characters or 2D photographic sprites (i.e., static full body green screen

captured photo images of a person; Anderson et al., 2005; Klinger, 2005; Pertaub et al., 2002). By

simply adjusting the number and location of such VH representations, the intensity of these anxiety-

provoking VR contexts could be systematically modulated with the aim to gradually habituate phobic

patients to what they feared, leading to improved functioning in the real world with real people.

Despite the primitive nature of these VHs, phobic clients appeared to be especially primed to react to

such representations and thus, they provided the necessary stimulus elements to be effective in

these types of exposure-based cognitive–behavioral treatment scenarios.

Other clinical applications have also used animated graphic VHs as stimulus entities to support and

train social and safety skills in persons with high functioning autism (Padgett, Strickland, & Coles,

2006; Parsons et al., 2012; Rutten et al., 2003) and as distracter stimuli for attention assessments

conducted in a virtual classroom (Rizzo et al., 2006). VHs have also been used effectively for the

conduct of social psychology experiments, essentially replicating and extending findings from studies

conducted with real humans on social influence, conformity, racial bias, and social proxemics

(Bailenson & Beall, 2006; Blascovich et al., 2002; McCall et al., 2009).

As the technology has evolved, VH agents can now be created that control computer generated

bodies and can interact with users through natural language speech and gesture in virtual

environments (Gratch et al., 2002; Rizzo, Kenny, & Parsons, 2011; Rizzo & Talbot, 2016a). Moreover,

with advances in artificial intelligence, VHs can engage in rich conversations (Morbini et al., 2014),

recognize nonverbal cues (Rizzo et al., 2015b, 2016b; Scherer et al., 2014), improve interactional

rapport with users (Park et al., 2013), reason about social and emotional factors (Gratch & Marsella,

2004), and synthesize human communication and nonverbal expressions (Thiebaux et al., 2008).

This has resulted in VH agent systems that serve as: virtual patients for training novice clinicians

(Rizzo et al., 2011, 2016a; Talbot et al., 2012), job interviewers for training young adults on the

autism spectrum to perform better in that context (Bresnahan et al., 2016); clinical interviewers to

reduce stigma (resulting in higher endorsement of clinical symptoms; Rizzo et al., 2015b, 2016b),

and as health care guides and clinical support agents (Rizzo et al., 2015b). For example, results from

of sample of military service members (SMs) who were interviewed by a VH clinical interviewer

before and after a deployment to Afghanistan indicated that SMs revealed more PTSD symptoms to

the VH than they reported on the Post Deployment Health Assessment (Rizzo et al., 2016b). In

another study using the same VH agent system, civilian users reported less concern about being

evaluated, disclosed more personal information, and displayed more sadness in an interview with a

VH agent compared with interacting with a VH avatar that they believed was being operated by a

human-in-the-loop “Wizard of Oz” controller (Lucas et al., 2014).

Thus, VHs now are capable of fostering interactions with real people that can address a wide variety

of clinical concerns. There is a growing literature in this area and it is not hard to see the power of VH

applications to foster roleplay training targeting social interaction, anger management, relapse

prevention for addiction, and in many other areas where clinical populations could benefit from low

social risk interaction with a nonjudgmental VH (Albright, Adam, Serri, Bleeker, & Goldman, 2016;

Bickmore et al., 2016; Rizzo et al., 2016b; Tegos et al., 2016; Zhang et al., 2017). Although some

authors have expressed legitimate concerns about the role of VH “automation” supplanting the role of

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

18 of 47 2/23/2019, 4:23 PM

clinicians (Innes & Morrison, 2017), VHs applications developed thus far, serve more to fill gaps

where a clinical provider is not available, than to aim at replacement of human providers.

Game-Based Interaction to Enhance Motivation and Engagement

Plato was reputed to have said, “You can discover more about a person in an hour of play than in a

year of conversation” (http://www.quotationspage.com/quote/2128.html). This ancient quote may

have particular relevance for future applications of clinical VR. Observing and/or quantifying a

person’s approach or strategy when participating in a gaming activity may provide insight into

cognitive and psychological functioning similar to the types of challenges found in traditional

performance assessments. However, a more compelling clinical direction may involve leveraging

gaming features and incentives for the challenging task of enhancing motivation and engagement

levels in clients participating in rehabilitation or any other clinical activity for that matter. For example,

one possible factor that may contribute to the mixed outcomes reported in cognitive or physical

rehabilitation research may be in part due to the inability to maintain a client’s motivation and

engagement when confronting them with a repetitive series of retraining challenges, whether using

wordlist exercises, range of motion exercises, or real-life functional activities (Rizzo et al., 2004). The

benefits of gamification for enhancing psychological interventions have also been detailed in Granic

et al. (2014) specifically citing research support of its value for improving cognition (e.g., attention),

motivation (e.g., resilience in the face of failure), emotion (e.g., mood management), and social

interaction (e.g., pro-social behavior). In this regard, an understanding of gaming features and their

integration into VR-based rehabilitation systems to enhance client motivation and subsequent clinical

outcomes may be a useful direction to explore.

Rehabilitation, whether cognitive or physical, provides a clear use case for how the integration of

gaming features with VR is well-matched to the various requirements for creating effective

rehabilitation tasks (Lange et al., 2012; Rizzo, 1994, 2004). This can be illustrated by first detailing

the general requirements for good rehabilitation tasks and then examining how they match up with

the features that game-based VR provides. To do this, we conjecture seven core requirements for a

good rehabilitation task.

The rehabilitation task must be as follows:

Grounded in data-based assessment to specify the target activity to be precisely rehabilitated

Adjustable in terms of difficulty level from something that is possible for the user to perform, to a level

representing the desired end-goal performance

Capable of repetitive and hierarchical administration to the user

Quantifiable in order to measure performance and progress

Capable of providing the user with strategic feedback as to the outcome of performance

Relevant to real-world functional activity

Capable of motivating user engagement and interaction with the task

Clinical VR assets are well-matched to meet these requirements, once a rehabilitation objective is

specified by state-of-the-art data-based assessment methodologies. VR’s capacity for stimulus

control (specified earlier in the article) can support the setting of a baseline challenge level that the

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

19 of 47 2/23/2019, 4:23 PM

user is capable of accomplishing. The stimulus control asset can also leverage the tireless capacity

of the computer to generate the repetitive and hierarchical delivery of stimulus challenges across a

range of programmable difficulty levels. In this way, an individual’s rehabilitation activity can be

customized to begin at a stimulus challenge level attainable and comfortable for them, with gradual

titration to higher or lower difficulty levels based on user performance. The interaction between the

user’s behavior and task demands can be automatically scored by the VR software to measure

performance, and provide real-time strategic feedback that can be automatically administered as

needed to shape and modulate performance toward a successful goal. All of this can occur within

simulated VR contexts that embody the complex functional challenges that exist in everyday

ecologically relevant settings. Thus, the experimental control required for rigorous scientific

measurement, analysis, and replication can still be maintained while the user is presented with

challenges that require real-world functional behaviors.

At each step in this process, computer game development principles and evidence-based

rehabilitation task design (Lange et al., 2009, 2010; 2012), can be combined to promote user

motivation and engagement. The VR assets described here follow the same structure for good

computer game design. For example, to maintain motivation, game designers develop content that

provides challenges within what is called the “flow channel.” Schell (2014) detailed the flow channel,

derived from Csikszentmihályi (1990), as, “. . . the narrow margin of challenge that lies between

boredom and frustration, for both of these unpleasant extremes cause our mind to change its focus

to a new activity” (p. 119). By integrating such game development principles with clinical VRs

capacity to deliver systematically controllable simulations, it is now possible to create compelling

rehabilitation tasks to enhance client motivation and engagement beyond what may be possible with

other existing methodologies. The feasibility of translating traditional evidence-based interventions

into computer gaming formats is increasingly being recognized by clinicians and scientists as a

methodology for exploiting the features of games for therapeutic change (Fleming et al., 2016).

Moreover, the growing recognition of the potential value of gamification (and the need for more

research) in health care and the field of “Games for Health” is evidenced by the appearance of

scientific journals and conferences focused on this topic, in addition to an evolving scientific

literature. Because a full review of this area is beyond the scope of this article, the reader is directed

to other detailed reviews (Baranowski et al., 2016; Fleming et al., 2016; Granic et al., 2014; Kato,

2010; Papastergiou, 2009).

Beyond Efficacy: VR as a Tool for Breaking Down Barriers to Care

This final asset is really a more speculative discussion of how VR at the current time may have value

beyond improving the efficacy of a clinical process and rather, is more concerned with how VR could

serve to break down some barriers to care. It is included here because some of these factors may

serve to inform later judgments as to clinical VR’s readiness for improving clinical practice and

research. The main premise here is that the best evidence-based approach for assessing or treating

a clinical health condition serves little value if clients do not seek it out and participate in it. There are

many reasons why these barriers limit client access to care and more detail can be found in

(Andrade et al., 2014; Clement et al., 2015). To more readily consider these barriers, we have

constructed an intuitive model for detailing them, called the 7As. The 7As stand for awareness,

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

20 of 47 2/23/2019, 4:23 PM

anticipated benefit, access, availability of well-trained providers, acceptability for seeking treatment,

adherence, and affordability.

Clinical VR may be strategically well-placed to break down some (but not all) of the barriers that keep

people from receiving the benefits of clinical care. To start, client awareness of the range of available

evidence-based treatment options may be limited. Perhaps some remedy for this exists in the media

exposure that is currently at an all-time high for VR. In addition to the media excitement and interest

in novel efforts to use VR for gaming and entertainment purposes, there has also been significant

coverage of VR health care applications. This may be in part due to a desire in some quarters of

popular culture to promote VR’s image as useful for prosocial purposes, beyond first person shooter

games. Thus, a quick search of the Internet will uncover a large volume of “heartstring tugging��� media reports on VR’s application with clinical conditions, especially those that are at the forefront of

the public consciousness (e.g., PTSD, autism, stroke, Alzheimer’s, depression, opiate addiction). For

better or worse, and despite the occasional scientific and factual errors in the popular press, there is

no doubt that clinical VR applications have received significant media visibility. Whether this builds

public awareness of treatment options that leads to actual help-seeking is still an open question in

need of more research.

As well, the double-edged sword of media claims about anticipated benefit can be problematic. The

balance between overwrought claims of clinical success and actual data points can sometimes err on

the side of higher than warranted expectations. However, when a clinical VR research study does

provide positive evidence, the popular media’s focus on covering that finding is fairly certain, thus

reaching the eyes and ears of people who will hopefully seek help, either for themselves or for a

loved one. For example, our PTSD VR exposure work has garnered significant popular media

reporting that is typically followed by an uptick in client or family member queries as to where

treatment can be accessed. The perception of the “sexiness” of the use of “exotic” VR technology in

the popular culture may also build expectations of success that in the end may drive a stronger

placebo effect in those who undergo VR-based services.

Making treatment more accessible is a factor for people who live in remote locations or who face

transportation challenges, and has served to drive efforts at using teletherapy or online self-help CBT

programming. However, as stated in the “independent practice” section, VR as a tool for pushing

care outside of the clinic is still limited by cost and complexity issues, as well as by ethical concerns.

This may be less limited in the future with the growing availability of low-cost VR technology in the

home, but for now, clinical VR is not seen to reduce the impact of this barrier. Similarly, the

availability of well-trained providers who are properly trained in clinical VR procedures is still limited.

Although many VR approaches follow the procedures and mechanics of traditional forms of therapy

(e.g., VR exposure therapy for anxiety disorders uses the same treatment protocol endorsed for

imaginal exposure approaches), the operation of VR equipment does require some specialized

training. This training is becoming more available either from standalone workshops or CME offerings

at respected conferences, but it is not commonplace at the current time. However, the use of VH

patients for training novice clinicians (Rizzo et al., 2011, 2016a; Talbot et al., 2012) is an emerging

area of focus, and this may have a direct impact on improving clinical use and supporting the greater

availability of well-trained providers.

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

21 of 47 2/23/2019, 4:23 PM

The acceptability of seeking care can be improved by reducing the internal or external perceptions of

stigma that a potential client may feel when admitting that they have a problem. Although this may be

less relevant for those seeking help to address a CNS-related condition, it is often a factor that limits

help-seeking for those with psychological health conditions. This is an area where clinical VR has

some early research support. In a survey study to assess openness to seeking care in 325 active

duty Army SMs (Wilson et al., 2008), results indicated that 83% of the participants reported that they

were neutral-to-very-willing to use some technology as part of a treatment; 71% were equally willing

or more willing to use a treatment based on technology than to talk to a therapist in a traditional

treatment setting. Moreover 20% of SMs, who stated they were not willing to seek traditional

psychotherapy, rated their willingness to use a VR-based treatment as neutral to very willing. One

possible interpretation of this finding is that a subgroup of this sample of SMs with a significant

disinterest in traditional mental health treatment would be willing to pursue treatment with a VR-

based approach. Thus, VR exposure therapy may offer an appealing treatment option for “digital

generation” SMs and Veterans who may be reluctant to seek out what they perceive as traditional

talk therapies. Other research using VR exposure for PTSD and phobias with civilian groups has

shown high levels of treatment satisfaction with VR (Baños et al., 2009; Beck et al., 2007) and in

some reports, participants reported that it was easier to take the first step in confronting fears with

VR compared imaginal exposure. Certainly, more research is needed to determine whether clinical

VR approaches reduce stigma and promote help-seeking. However, one can speculate that younger

groups who have grown up in this “digital age” may actually be more attracted to and comfortable

with participation in a clinical VR approach and this could be a factor for reducing stigma and

increasing the acceptability of VR-based care.

Finally, more research is needed to investigate the impact of clinical VR for promoting adherence to a

full course of treatment. Although a number of small studies have suggested a higher positive

interest in continuing treatment with VR (cf. Bryanton et al., 2006), most research examining

treatment adherence as a specific variable has been underpowered. Although the motivating factors

of clinical VR tools are frequently referred to in the literature, we are not aware of any systematic

evaluations of VR treatment characteristics and their impact on patient attrition for prolonged,

repetitive treatment protocols. We expect factors such as multiplayer and competitive training

content, level of immersion, story-driven/narrated treatment content, or relevance of treatment

content to the patient’s everyday life to be important factors for sustained patient motivation. The

relevance of the aforementioned “flow channel” (Schell, 2014) and its impact on user motivation and

engagement cannot be overstated. Thus, bridging the gap between scientific construction of

evidence-based treatment tasks and artistic design of game-based content seems a worthwhile

target for further investigation.

Affordability has also been an issue that has limited VR treatment access in the past. This is

expected to be less of a limiting factor, now that higher fidelity, yet low-cost systems have come onto

the market. As a point of comparison, it is now possible to purchase a high-fidelity VR HMD (HTC

Corporation, New Taipei City, Taiwan) for $800 that has superior specifications compared with a

system that would have cost $20,000 (NVIS Inc., Reston, VA) to purchase 5 years ago. In addition,

low-cost smartphone-based VR HMDs are likely to achieve parity with computer-tethered systems for

some clinical VR applications and this is predicted to dramatically reduce hardware costs and

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

22 of 47 2/23/2019, 4:23 PM

improve affordability. With large technology companies such as Facebook, Google, Apple, and

Samsung invested in the VR market, we anticipate new and affordable hardware and software to be

released more frequently over the next few years. Moreover, successful companies in the clinical VR

space (e.g., MindMaze, SilverFit, Gesturetek Health) are paving the way for a competitive landscape

of VR tools for clinical assessment and treatment that will inevitably result in more affordable options

for researchers and clinical providers. As these companies continue their R&D work on innovative

VR applications, we hope to see diversity and accessibility in this growing market, not unlike

Google’s Play Store or Apple’s App Store, again with the result of more affordable prices for clinical

end-users and eventually for home-based use by patients.

Discussion—Is Clinical Virtual Reality Ready for Primetime?

The question of clinical VR’s readiness for widespread clinical use can be considered across the

criteria of theory, research, pragmatics, and ethics. On the basis of the clear assets and features that

are available with simulation technology, there is a sound theoretical basis for the development and

implementation of informed clinical VR applications. General simulation technology has a long history

of adding value in aviation simulation, military planning, automotive/aircraft design, and surgical

planning (Virtual Reality Society, 2017). By leveraging these same assets, but in a form factor that

can deliver VR experiences within a clinicians’ office or research laboratory, a new set of virtual tools

become possible for psychology and rehabilitation. Although any given clinical VR application will

likely not leverage all of the VR assets described in this article, a clear specification of what features

can be brought to bear on a clinical target is recommended to guide design, implementation, and

evaluation in a systematic fashion.

A guiding principle in our work is to first look at known processes operating in physical reality that are

believed to contribute to the creation of an evidence-based approach to assessment and

intervention. With that as a starting point, one can specify the VR assets that can underlie and guide

the creation of a VR application to: provide more reliable and valid assessments, amplify treatment

effects, break down barriers to care, or simply reduce costs by automating processes. For example,

we know that the use of imaginal exposure approaches for anxiety disorders are evidence-based in

the physical world. From that, one can see a direct case for using VR to deliver ecologically relevant

simulations, within which we can precisely control and titrate the delivery of progressively more

provocative stimuli to pace exposure for the end goal of promoting extinction learning. Similarly, we

know that the sheer amount of physical rehabilitation activity that a stroke survivor engages in (all

other factors being equal) is related to improved outcomes. From that, it is logical to hypothesize that

if compelling game-based VR rehabilitation tasks are developed, it may be possible to motivate users

to do more repetitions, leading to improved outcomes. These thumbnail examples simply present one

or two of the assets that can inform the rationale for clinical VR use cases, but in reality, there may

be any number of additional features that can be specified and marshalled (e.g., strategic feedback,

cueing stimuli, safety, etc.) for adding value over existing traditional methods. Thus, it is our

perspective that the theoretical basis for using clinical VR is sound and supportive of its “primetime”

application.

The research support for the use of clinical VR is promising, albeit not fully mature. There seems to

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

23 of 47 2/23/2019, 4:23 PM

be a consensus in the literature, that VR can produce equivalent or better outcomes for exposure-

based approaches for anxiety disorder treatment (e.g., Bouchard et al., 2017; Maples-Keller et al.,

2017; Morina et al., 2015; Rizzo et al., 2015a). Consistent findings have also been produced in

support of VR as an effective distraction tool for reducing the perception of pain in patients

undergoing acutely painful medical procedures (e.g., Hoffman et al., 2011; Trost et al., 2015). A

growing body of research is indicating that VR can increase participation in physical rehabilitation,

with patients reporting more motivation to engage in rehab tasks within a game-based VR context

compared with standalone training (e.g., Granic et al., 2014). Cognitive assessment methods using

VR have produced promising results in construct validation studies, and for distinguishing between

clinical groups and healthy controls (e.g., Man et al., 2016; Nir-Hadad et al., 2015; Parsons & Rizzo,

2008b; Rizzo et al., 2006). And finally, the use of Virtual Humans in clinical VR applications has

produced promising results indicating that they can foster credible interactions with real people for

training, as health care guides, and in the role of clinical assessors, but this area is still in a very early

state of maturity (Rizzo et al., 2015b, 2016ab; Scherer et al., 2014; Talbot et al., 2012). By contrast,

whether due to the complexity of the problem space or the lack of standards in VR research

methodology, cognitive rehabilitation studies using VR interventions have provided more mixed

outcomes. Again, there is consensus about the promise of VR cognitive rehabilitation tools (e.g.,

Bogdanova, Yee, Ho, & Cicerone, 2016; Ogourtsova, Silva, Archambault, & Lamontagne, 2015;

Valladares-Rodriguez et al., 2016), but the majority of conducted studies are pilot trials without

sufficient power or the study design needed to draw decisive conclusions about efficacy, transfer of

gained skills to the daily life of clients, long-term outcomes, and cost-effectiveness.

A continued focus on research methodology, selection of outcome measures, quantification of

training transfer to daily life, and the identification of “active ingredients” of clinical VR tools is needed

to advance its thoughtful and scientifically valid use. This includes answering questions about: the

frequency and modality of feedback and cues; treatment doses and frequencies; complexity of VR

tasks and environments; importance of graphical realism and fidelity; selection and usability of

interface devices; relevance of gamification and multiplayer/competitive elements; and many other

factors that inform VR system design. Importantly, these questions need to be posed for each of the

diverse patient populations that stand to benefit from clinical VR tools. In sum, the research is

generally supportive for the “primetime” use of clinical VR in some areas, but there should be no

illusion as to the need for more research investigating the boundary conditions for its safe and

effective application.

The positive outcomes seen in the clinical VR literature thus far are actually quite encouraging when

viewed in the context of the challenges that researchers faced in these areas. First, the general

availability of the technology has only existed for about 25 years and for the first 10 to 15 years of

that, the maturity of the hardware and software was quite variable. During those early years, with the

notable exception of exposure therapy applications, clinical VR research and development (R&D)

was essentially exploratory, primarily characterized by one-off, proof-of-concept, prototype systems.

Although these systems produced interesting results in uncontrolled, small sample size studies, only

a few applications were subjected to rigorous parametric tests by independent researchers. As a

result, most clinical VR review articles include the staple recommendation that, “while current VR

findings are promising, more controlled research with larger sample sizes are needed.” This is not a

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

24 of 47 2/23/2019, 4:23 PM

slight on innovative researchers who had to bear the double burden of acquiring funding for both

system development and clinical tests, with a technology that was sometimes perceived by grant

reviewers as being too “science-fiction-y” to support good science! Rather, it is just an observation on

the challenges that have slowed the progression of tightly controlled research in some clinical VR

areas. Thus, when one considers that psychology as a science has been around for about 125 years

with a focus on studying human behavior and interaction in the physical world, it makes sense that

we may need a few more years to evolve the science for how humans behave and interact in the

virtual world.

By contrast, the pragmatics for developing and using clinical VR systems are quite favorable. Over

the last 10 years, the technology has gradually advanced enough to support widespread VR system

development beyond what was only possible within very specialized research institutes. This has

now been recognized by the Gartner Inc. (2016) with their elevation of VR from the “trough of

disillusionment” to the “slope of enlightenment” in the hype cycle for emerging technologies. A key

factor for VR’s recent expansion is a growing VR development community that thrives on access to

affordable design tools and VR hardware. Development software (e.g., game engines Unity3D,

Unreal Engine, Amazon Lumberyard) has seen a large boost in popularity over the last 5 years and

has even found its way into high school and college computer science curriculums. Any interested

student, educator, hobbyist, or entrepreneur can pick up these tools for free and begin developing VR

applications without much upfront investment or any of the barriers that VR R&D teams faced in the

past. We expect this momentum and growth of the VR developer community to translate to a surge

of new VR applications, including clinical VR tools. The online PC distribution platform and

community Steam (Valve Corporation, 2017) is currently listing more than 1900 VR-enabled PC

games. We anticipate similar distribution platforms to emerge for clinical VR content that will provide

greater access to affordable libraries of archetypic treatment and assessment scenarios for health

care providers and researchers.

As we look to the future, we see clinical VR as one of the larger domains of general VR usage. In the

recent Goldman Sachs (2016) market analysis looking at the future of VR in 2025, we of course see

that Gaming and Entertainment garners the largest market share. Although this is to be expected

with the public’s chronic demand for new and better ways to consume media, the little noticed item in

that market analysis is that “healthcare” comes in second for the VR market share. This is not a

surprise to researchers and clinicians who have worked in this area over the years, especially as we

see health care costs becoming one of the largest line items in the U.S. Government budget, after

Defense. Interest in clinical VR by actual therapists also seems to be substantial. Norcross et al.

(2013) surveyed 70 psychotherapy experts regarding interventions they predicted to increase in the

next decade and VR was ranked 4th out of 45 options with other computer-supported methods

occupying 4 out of the top 5 positions.

The ethical use of VR needs to be considered thoughtfully in any assessment of its future primetime

impact on psychological practice or science. Current VR technology now allows for the creation of

emotionally evocative virtual experiences. With clinical VR, we often aim to leverage that capability

for a positive impact in client care. But if we accept that it is possible to create experiences that can

evoke strong emotions for a positive clinical purpose, we must also accept the probability of some

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

25 of 47 2/23/2019, 4:23 PM

risks for the occurrence of unforeseen negative emotional reactions. Thus, the question of safe and

ethical use of VR has been addressed in detail at various junctures (Madary & Metzinger, 2016;

Rizzo, Schultheis, & Rothbaum, 2003; Yellowlees, Holloway, & Parish, 2012; Tart, 1993). Although

there are a variety of ethical issues for the general application of VR beyond its clinical use (e.g.,

motion sickness side effects, overuse, violent content, etc.), our focus here is limited to the use of VR

as a tool for clinical diagnosis and treatment.

Thus far, a significant literature has emerged in support of the positive impact of well-designed,

theory-informed VR applications on mental health and physical functioning. These applications are

typically administered within the controlled and safe context of the therapy setting, supervised by a

well-trained clinician. However, what happens if these types of VR experiences become commodity

products that are readily accessible to anyone who self-diagnoses their clinical condition and then

uses VR treatment content as a “self-help” therapy? Although some might say this is not much

different than purchasing a self-help book and following the instructions and recommendations

therein, VR experiences may have more impact on a user than what may occur from reading a book.

Similar to most areas of mental health care, there is also a risk that this form of self-diagnosis and

treatment is based on inaccurate or counterproductive information. Another kind of ethical challenge

can also emerge if a clinician decides that VR would be great for generating a buzz for their practice

and result in more business, but the clinician hasn’t had sufficient training in its use and safe

application. Thus, there are issues of concern here from the perspective of patients and providers.

Consequently, there is a need for ethical guidelines regarding the safe and informed use of clinical

VR applications, much like the way that pharmaceutical treatments are managed by a well-trained

and qualified physician.

In the area of clinical practice, the American Psychological Association’s ethical code provides a

clear and well-endorsed set of guidelines that can serve as a good starting point for understanding

and proactively addressing some of the basic issues for the creation and use of VR applications in

clinical practice (APA, 2003). Three core areas of clinical practice concerns and recommendations

can be derived from these guidelines (two of which come directly from the APA code):

2.01 Boundaries of Competence: (a) Psychologists provide services, teach and conduct research

with populations and in areas only within the boundaries of their competence, based on their

education, training, supervised experience, consultation, study or professional

experience.Recommendation: VR-delivered mental health assessment/treatment may require

fundamentally different skill sets than what is needed for traditional “talk therapy” approaches.

Clinicians need to have specialized training, and possibly in the future, some level of certification in

the safe and ethical use of VR for therapy.

2.04 Bases for Scientific and Professional Judgments: Psychologists’ work is based on established

scientific and professional knowledge of the discipline.Recommendation: VR applications that are

developed for clinical assessment and treatment must be based on a theoretical framework and

documented with some level of research before they can be endorsed as evidence-based and

marketed as such. In an emerging area like VR where unique and specific guidelines have yet to be

established, the practitioner must be fully transparent about the evidence base for the approach and

take precautions to preserve the safety and integrity of the patient.

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

26 of 47 2/23/2019, 4:23 PM

Self-Diagnosis/Self-Treatment.

Although not cited as an APA standard, the issues regarding patient self-diagnosis and self-treatment

deserve further mention. Mental health conditions can be extremely complex and in some instances

the self-awareness of the patient may be compromised. This can oftentimes lead to a faulty self-

diagnosis as well as the problems that arise when the patient searches for symptom information on

the Internet where reliable and valid content can be questionable. The same issues come into play

with self-treatment. The problems that can ensue are twofold:

The patient makes errors in either or both areas and achieves no clinical benefit, or worse,

aggravates the existing condition with an ineffective or inappropriate VR approach that actually does

more harm.

By pursuing a “seductive” VR self-help approach that is misaligned with their actual needs or has no

evidence for its efficacy, the patient could miss the opportunity to receive quality evidence-based

care that is designed and delivered based on the informed judgment of a trained expert diagnostician

or clinical care provider.

These two negative impacts could occur if a company produces a VR approach without sufficient

validation and markets it to the public as a valid test or cure. This has been seen over the years with

many forms of quack medicine, and there needs to be some principle about the promotion of a VR

application that has the consumer’s protection in mind. This issue is particularly important at the

current time in view of all the public exposure, hype, and genuine excitement surrounding VR. There

are many new companies emerging in the health care space, essentially being driven by venture

capitalists and game developers, without any credible expert clinical and/or research guidance. Such

companies could not only do harm to users, but the uninformed development and overhype of the

benefits to be derived from a VR clinical application leading to negative effects could serve to create

the general impression that VR is a “snake oil” approach and lead to people not seeking (or

benefiting from) an otherwise well-validated VR approach.

An example of a gray area in this domain concerns one of the most common fears that people report

- public speaking. Technically, in an extreme form where it significantly impairs social and

occupational functioning, public speaking anxiety would qualify as a phobia and be diagnosed as an

anxiety disorder. However, because most people do have some level of subclinical fear of public

speaking (that they eventually get over with practice), this has been one of the first areas where

widespread consumer access to public speaking VR exposure therapy software has occurred

(Hypergrid Business, 2016). Users can practice their presentation “skills” on a low-cost smartphone-

based VR HMD (e.g., Google Cardboard/Daydream, Samsung Gear VR) in front of various types of

audiences and settings. In this case, most clinicians would not show much concern for this type of

self-help skills training approach and the potential for damaging effects to a user appears to be fairly

minimal. But, from this example, can we now expect that applications will be made readily available

for other and perhaps more complex anxiety disorder-based phobias (fear of flying, social phobia,

fear of driving, arachnophobia, fear of intimacy, etc.) or even for PTSD treatment? Consequently, it

appears that ethical guidelines may be needed to support the safe use of clinical VR.

In conclusion, interest in the clinical uses of VR technology has accelerated and will likely continue to

be fueled by a societal zeitgeist in which this form of immersive and interactive technology inspires

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

27 of 47 2/23/2019, 4:23 PM

the public’s attention and imagination. Although previously hamstrung by costs, complexity, and

clinician unfamiliarity with VR equipment, the technology has evolved dramatically in the consumer

marketplace with new low-cost, hi-fidelity, product offerings that are poised to drive wider scale

adoption. This will result in a probable future scenario in which VR devices will become like toasters

—although you may not use it every day, every household will have one. When such market

penetration occurs, the general public will have more access to a range of VR experiences. This may

serve to accelerate the uptake of clinical VR as users, who would be more familiar with the

technology, begin to imagine its value beyond the world of digital games.

The momentum generated by the growing public awareness of VR coupled with advances in the

technology has created a unique opportunity for psychology and rehabilitation. Our analysis of the

theoretical underpinnings and research findings to date leads us to predict that the application of

clinical VR will have a significant impact on future research and practice. The pragmatic issues that

may influence its adoption as a tool across many areas of psychology also appear favorable, but

professional guidelines will be needed to promote its safe and ethical use. Such guidelines should

inform the development of principles for clinical VR application design, distribution, practice, and

training. Although there is still much work to be done to advance the science in this area, we strongly

believe that clinical VR applications will become indispensable tools in the toolbox of psychological

researchers and practitioners and will only grow in relevance and popularity in the future. Thus, it is

our assessment that clinical VR is indeed ready for primetime!

References

Adamovich, S. V., Fluet, G. G., Tunik, E., & Merians, A. S. (2009). Sensorimotor training in virtual

reality: A review. NeuroRehabilitation, 25, 29–44.

Akinwuntan, A. E., Wachtel, J., & Rosen, P. N. (2012). Driving simulation for evaluation and

rehabilitation of driving after stroke. Journal of Stroke and Cerebrovascular Diseases, 21, 478–486.

10.1016/j.jstrokecerebrovasdis.2010.12.001

Albright, G., Adam, C., Serri, D., Bleeker, S., & Goldman, R. (2016). Harnessing the power of

conversations with virtual humans to change health behaviors. mHealth, 2, 44.

10.21037/mhealth.2016.11.02

American Psychological Association. (2003). Ethical principles of psychologists and code of conduct.

Retrieved from http://www.apa.org.library.capella.edu/ethics/code/

Anderson, P. L., Zimand, E., Hodges, L. F., & Rothbaum, B. O. (2005). Cognitive behavioral therapy

for public-speaking anxiety using virtual reality for exposure. Depression and Anxiety, 22, 156–158.

10.1002/da.20090

Andrade, L. H., Alonso, J., Mneimneh, Z., Wells, J. E., Al-Hamzawi, A., Borges, G., . . .Kessler, R. C.

(2014). Barriers to mental health treatment: Results from the WHO World Mental Health surveys.

Psychological Medicine, 44, 1303–1317. 10.1017/S0033291713001943

Astur, R. S., Ortiz, M. L., & Sutherland, R. J. (1998). A characterization of performance by men and

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

28 of 47 2/23/2019, 4:23 PM

women in a virtual Morris water task: A large and reliable sex difference. Behavioural Brain

Research, 93, 185–190. 10.1016/S0166-4328(98)00019-9

Astur, R. S., St Germain, S. A., Tolin, D., Ford, J., Russell, D., & Stevens, M. (2006). Hippocampus

function predicts severity of post-traumatic stress disorder. Cyberpsychology & Behavior, 9, 234–240.

10.1089/cpb.2006.9.234

Astur, R. S., Taylor, L. B., Mamelak, A. N., Philpott, L., & Sutherland, R. J. (2002). Humans with

hippocampus damage display severe spatial memory impairments in a virtual Morris water task.

Behavioural Brain Research, 132, 77–84. 10.1016/S0166-4328(01)00399-0

Astur, R. S., Tropp, J., Sava, S., Constable, R. T., & Markus, E. T. (2004). Sex differences in a virtual

Morris water task and a virtual eight-arm maze. Behavioural Brain Research, 151, 103–115.

10.1016/j.bbr.2003.08.024

Aukstakalnis, & Blatner, D. (1992). Silicon mirage: The art and science of virtual reality. Berkeley, CA:

Peachpit Press.

Badia, S. B., Fluet, G. G., Llorens, R., & Deutsch, J. E. (2016). Virtual reality for sensorimotor

rehabilitation post stroke: Design principles and evidence. In Neurorehabilitation technology (pp.

573–603). Cham, Switzerland: Springer International Publishing.

Bailenson, J. N., & Beall, A. C. (2006). Transformed social interaction: Exploring the digital plasticity

of avatars. In Schroeder, R. & Axelsson, A. ’s (Eds.), Avatars at work and play: Collaboration and

interaction in shared virtual environments (pp. 1–16). Berlin, Germany: Springer-Verlag.

Baños, R. M., Botella, C., Guillen, V., García-Palacios, A., Quero, S., Bretón-López, J., & Alcañiz, M.

(2009). An adaptive display to treat stress-related disorders: EMMA’s world. British Journal of

Guidance and Counseling, 37, 347–356.

Baños, R. M., Guillen, V., Quero, S., García-Palacios, A., Alcaniz, M., & Botella, C. (2011). A virtual

reality system for the treatment of stress-related disorders: A preliminary analysis of efficacy

compared to a standard cognitive behavioral program. International Journal of Human-Computer

Studies, 69, 602–613. 10.1016/j.ijhcs.2011.06.002

Baranowski, T., Blumberg, F., Buday, R., DeSmet, A., Fiellin, L. E., Green, C. S., . . .Young, K.

(2016). Games for health for children—Current status and needed research. Games for Health

Journal, 5, 1–12. 10.1089/g4h.2015.0026

Barrett, J., McCrindle, R. J., Cook, G. K., & Booy, D. A. (2002). Accessible interface design: A review

of access to information and communication technology for older and disabled people. Reading, UK:

Department of Computer Science, The University of Reading.

Beck, A. T., Epstein, N., Brown, G., & Steer, R. A. (1988). An inventory for measuring clinical anxiety:

Psychometric properties. Journal of Consulting and Clinical Psychology, 56, 893–897.

10.1037/0022-006X.56.6.893

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

29 of 47 2/23/2019, 4:23 PM

Beck, J. G., Palyo, S. A., Winer, E. H., Schwagler, B. E., & Ang, E. J. (2007). Virtual reality exposure

therapy for PTSD symptoms after a road accident: an uncontrolled case series. Behavior Therapy,

38, 39–48.

Beidel, D. C., Frueh, B. C., Neer, S. M., & Lejuez, C. W. (2017). The efficacy of Trauma Management

Therapy: A controlled pilot investigation of a three-week intensive outpatient program for combat-

related PTSD. Journal of Anxiety Disorders, 50, 23–32. 10.1016/j.janxdis.2017.05.001

Bickmore, T. W., Utami, D., Matsuyama, R., & Paasche-Orlow, M. K. (2016). Improving access to

online health information with conversational agents: A randomized controlled experiment. Journal of

Medical Internet Research, 18(1), e1. 10.2196/jmir.5239

Blascovich, J., Loomis, J., Beall, A., Swinth, K., Hoyt, C., & Bailenson, J. (2002). Immersive virtual

environment technology: Not just another research tool for social psychology. Psychological Inquiry,

13, 103–124. 10.1207/S15327965PLI1302_01

Blume, F., Hudak, J., Dresler, T., Ehlis, A. C., Kühnhausen, J., Renner, T. J., & Gawrilow, C. (2017).

NIRS-based neurofeedback training in a virtual reality classroom for children with attention-

deficit/hyperactivity disorder: Study protocol for a randomized controlled trial. Trials, 18, 41.

10.1186/s13063-016-1769-3

Bogdanova, Y., Yee, M. K., Ho, V. T., & Cicerone, K. D. (2016). Computerized cognitive rehabilitation

of attention and executive function in acquired brain injury: A systematic review. The Journal of Head

Trauma Rehabilitation, 31, 419–433. 10.1097/HTR.0000000000000203

Bohil, C. J., Alicea, B., & Biocca, F. A. (2011). Virtual reality in neuroscience research and therapy.

Nature Reviews Neuroscience, 12, 752–762.

Bordnick, P. S., Yoon, J. H., Kaganoff, E., & Carter, B. (2013). Virtual reality cue reactivity

assessment: A comparison of treatment- vs. nontreatment-seeking smokers. Research on Social

Work Practice, 23, 419–425. 10.1177/1049731513482377

Botella, C., Serrano, B., Baños, R. M., & Garcia-Palacios, A. (2015). Virtual reality exposure-based

therapy for the treatment of post-traumatic stress disorder: A review of its efficacy, the adequacy of

the treatment protocol, and its acceptability. Neuropsychiatric Disease and Treatment, 11,

2533–2545. 10.2147/NDT.S89542

Bouchard, S., Dumoulin, S., Robillard, G., Guitard, T., Klinger, É., Forget, H., . . .Roucaut, F. X.

(2017). Virtual reality compared with in vivo exposure in the treatment of social anxiety disorder: A

three-arm randomised controlled trial. The British Journal of Psychiatry, 210, 276–283.

10.1192/bjp.bp.116.184234

Bresnahan, T., Rizzo, A. A., Burke, S. L., Partin, M., Ahlness, R. M., & Trimmer, M. (2016). Using

virtual interactive training agents (VITA) with adults with autism and other developmental disabilities.

Paper presented at the 2016 Proceedings of the International Conference on Disability, Virtual

Reality and Associated Technologies. Retrieved from http://www.icdvrat.org/2016/papers

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

30 of 47 2/23/2019, 4:23 PM

/ICDVRAT2016_S02N2_Bresnahan_etal.pdf

Brooks, B. M., McNeil, J. E., Rose, F. D., Greenwood, R. J., Attree, E. A., & Leadbetter, A. G. (1999).

Route learning in a case of amnesia: A preliminary investigation into the efficacy of training in a

virtual environment. Neuropsychological Rehabilitation, 9, 63–76.

Brown, D. J., Kerr, S. J., & Bayon, V. (1998). The development of the virtual city: A user-centered

approach. In P.Sharkey, D.Rose, and J.Lindstrom (Eds.), In Proceedings of the 2nd European

conference on disability, virtual reality and associated technologies (pp. 11–16). Reading UK:

University of Reading.

Bryanton, C., Bossé, J., Brien, M., McLean, J., McCormick, A., & Sveistrup, H. (2006). Feasibility,

motivation, and selective motor control: Virtual reality compared to conventional home exercise in

children with cerebral palsy. Cyberpsychology & Behavior, 9, 123–128. 10.1089/cpb.2006.9.123

Burgers, C., Eden, A., van Engelenburg, M., & Buningh, S. (2015). How feedback boosts motivation

and play in a brain-training game. Computers in Human Behavior, 48, 94–103.

10.1016/j.chb.2015.01.038

Carlbring, P., Westling, B. E., Ljungstrand, P., Ekselius, L., & Andersson, G. (2001). Treatment of

panic disorder via the Internet: A randomized trial of a self-help program. Behavior Therapy, 32,

751–764. 10.1016/S0005-7894(01)80019-8

Casutt, G., Martin, M., Keller, M., & Jäncke, L. (2014). The relation between performance in on-road

driving, cognitive screening and driving simulator in older healthy drivers. Transportation Research

Part F: Traffic Psychology and Behaviour, 22, 232–244. 10.1016/j.trf.2013.12.007

Chirico, A., Lucidi, F., De Laurentiis, M., Milanese, C., Napoli, A., & Giordano, A. (2016). Virtual

reality in health system: Beyond entertainment. A mini-review on the efficacy of VR during cancer

treatment. Journal of Cellular Physiology, 231, 275–287. 10.1002/jcp.25117

Chou, Y. H., Weingarten, C. P., Madden, D. J., Song, A. W., & Chen, N. K. (2012). Applications of

virtual reality technology in brain imaging studies. In C.Eichenberg (Ed.), Virtual reality in

psychological, medical and pedagogical applications. Retreived from10.5772/48445

Christiansen, C., Abreu, B., Ottenbacher, K., Huffman, K., Masel, B., & Culpepper, R. (1998). Task

performance in virtual environments used for cognitive rehabilitation after traumatic brain injury.

Archives of Physical Medicine and Rehabilitation, 79, 888–892. 10.1016/S0003-9993(98)90083-1

Clement, S., Schauman, O., Graham, T., Maggioni, F., Evans-Lacko, S., Bezborodovs, N., . .

.Thornicroft, G. (2015). What is the impact of mental health-related stigma on help-seeking? A

systematic review of quantitative and qualitative studies. Psychological Medicine, 45, 11–27.

10.1017/S0033291714000129

Connor, B., Wing, A. M., Humphreys, G. W., Bracewell, R. M., & Harvey, D. A. (2002). Errorless

learning using haptic guidance: Research in cognitive rehabilitation following stroke. In P.Sharkey, C.

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

31 of 47 2/23/2019, 4:23 PM

S.Lanyi, & P.Standen (Eds.), In Proceedings of the 4th international conference on disability, virtual

reality, and associated technology (pp. 77–86). Reading, UK: University of Reading.

Costanzo, M. E., Leaman, S., Jovanovic, T., Norrholm, S. D., Rizzo, A. A., Taylor, P., & Roy, M. J.

(2014). Psychophysiological response to virtual reality and sub-threshold PTSD symptoms in recently

deployed military. Psychosomatic Medicine, 76, 670–677. 10.1097/PSY.0000000000000109

Costas, R., Carvalho, L., & de Aragon, D. (2000). Virtual city for cognitive rehabilitation. In P.Sharkey,

A.Cesarani, L.Pugnetti, & A.Rizzo (Eds.), In Proceedings of the 3rd international conference on

disability, virtual reality, and associated technology (pp. 305–313). Reading, UK: University of

Reading.

Cox, S. M., Cox, D. J., Kofler, M. J., Moncrief, M. A., Johnson, R. J., et al. (2015). Driving simulator

performance in novice drivers with autism spectrum disorder: The role of executive functions and

basic motor skills. Journal of Autism and Developmental Disorders, 46, 1379–1391.

10.1007/s10803-015-2677-1

Cromby, J., Standen, P., Newman, J., & Tasker, H. (1996). Successful transfer to the real world of

skills practiced in a virtual environment by student with severe learning disabilities. In P. M.Sharkey

(Ed.), In Proceedings of the 1st European conference on disability, virtual reality and associated

technologies (pp. 305–313). Reading, UK: University of Reading.

Cruz-Neira, C., Sandin, D. J., & DeFanti, T. A. (1993, September). Surround-screen projection-based

virtual reality: The design and implementation of the CAVE. In M. C.Whitton (Ed.), In Proceedings of

the 20th annual conference on Computer graphics and interactive techniques (pp. 135–142). New

York, NY: ACM.

Csikszentmihályi, M. (1990). Flow: The psychology of optimal experience. New York, NY: Harper &

Row.

Cuijpers, P., van Straten, A., & Andersson, G. (2008). Internet-administered cognitive behavior

therapy for health problems: A systematic review. Journal of Behavioral Medicine, 31, 169–177.

10.1007/s10865-007-9144-1

Darejeh, A., & Singh, D. (2013). A review on user interface design principles to increase software

usability for users with less computer literacy. Journal of Computational Science, 9, 1443–1450.

10.3844/jcssp.2013.1443.1450

Dascal, J., Reid, M., IsHak, W. W., Spiegel, B., Recacho, J., Rosen, B., & Danovitch, I. (2017). Virtual

reality and medical inpatients: A systematic review of randomized, controlled trials. Innovations in

Clinical Neuroscience, 14(1–2), 14–21.

Davies, R. C., Johansson, G., Boschian, K., Lindén, A., Minör, U., & Sonesson, B. (1998). A practical

example using virtual reality in the assessment of brain injury. In P.Sharkey, D.Rose, & J.Lindstrom

(Eds.), In Proceedings of the 2nd European conference on disability, virtual reality and associated

techniques (pp. 61–68). Reading, UK: University of Reading.

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

32 of 47 2/23/2019, 4:23 PM

Davis, E. S., & Wachtel, J. (2000, January22). Interactive driving simulation as a tool for insight

development and motivation in a rehabilitation setting. Paper presented at the 8th Annual Medicine

Meets Virtual Reality Conference, Newport Beach, CA.

DeFanti, T. A., Acevedo, D., Ainsworth, R. A., Brown, M. D., Cutchin, S., et al. (2011). The Future of

the CAVE. Central European Journal of Engineering, 1, 16–37.

Deutsch, J. E., Latonio, J., Burdea, G., & Boian, R. (2001). Post-stroke rehabilitation with the Rutgers

Ankle System: A case study. Presence: Teleoperators and Virtual Environments, 10, 416–430.

10.1162/1054746011470262

Deutsch, J. E., & Westcott McCoy, S. (2017). Virtual reality and serious games in neurorehabilitation

of children and adults: Prevention, plasticity, and participation. Pediatric Physical Therapy, 29, S23–

S36. 10.1097/PEP.0000000000000387

DeVault, D., Mell, J., & Gratch, J. (2015, March). Toward natural turn-taking in a virtual human

negotiation agent. In Andrist, D.Bohus, E.Horvitz, B.Mutlu, & D.Schlangen (Eds.), AAAI Spring

Symposium on Turn-taking and Coordination in Human-Machine Interaction (pp. 2–9). Stanford, CA:

AAAI Press.

Difede, J., Cukor, J., Jayasinghe, N., Patt, I., Jedel, S., Spielman, L., . . .Hoffman, H. G. (2007).

Virtual reality exposure therapy for the treatment of posttraumatic stress disorder following

September 11, 2001. The Journal of Clinical Psychiatry, 68, 1639–1647. 10.4088/JCP.v68n1102

Difede, J., Cukor, J., Wyka, K., Olden, M., Hoffman, H., Lee, F. S., & Altemus, M. (2013, November).

D-cycloserine augmentation of exposure therapy for posttraumatic stress disorder: A pilot

randomized clinical trial. Neuropsychopharmacology, 39, 1052–1058.

Elkind, J. S., Rubin, E., Rosenthal, S., Skoff, B., & Prather, P. (2001). A simulated reality scenario

compared with the computerized Wisconsin card sorting test: An analysis of preliminary results.

Cyberpsychology & Behavior, 4, 489–496. 10.1089/109493101750527042

Falconer, C. J., Rovira, A., King, J. A., Gilbert, P., Antley, A., Fearon, P., . . .Brewin, C. R. (2016).

Embodying self-compassion within virtual reality and its effects on patients with depression. British

Journal of Psychiatry Open, 2, 74–80. 10.1192/bjpo.bp.115.002147

Faria, A. L., Andrade, A., Soares, L. I., & Badia, S. B. (2016). Benefits of virtual reality-based

cognitive rehabilitation through simulated activities of daily living: A randomized controlled trial with

stroke patients. Journal of Neuroengineering and Rehabilitation, 13, 96. 10.1186/s12984-016-0204-z

Fish, J. E., Manly, T., Kopelman, M. D., & Morris, R. G. (2015). Errorless learning of prospective

memory tasks: An experimental investigation in people with memory disorders. Neuropsychological

Rehabilitation, 25, 159–188. 10.1080/09602011.2014.921204

Fleming, T. M., Bavin, L., Stasiak, K., Hermansson-Webb, E., Merry, S. N., Cheek, C., . . .Hetrick, S.

(2016). Computer-games for mental health: Current status and promising directions. Frontiers in

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

33 of 47 2/23/2019, 4:23 PM

Psychiatry, 7, 215.

Folley, B. S., Astur, R., Jagannathan, K., Calhoun, V. D., & Pearlson, G. D. (2010). Anomalous neural

circuit function in schizophrenia during a virtual Morris water task. NeuroImage, 49, 3373–3384.

10.1016/j.neuroimage.2009.11.034

Foloppe, D. A., Richard, P., Yamaguchi, T., Etcharry-Bouyx, F., & Allain, P. (2015). The potential of

virtual reality-based training to enhance the functional autonomy of Alzheimer’s disease patients in

cooking activities: A single case study. Neuropsychological Rehabilitation, 25, 1–25.

10.1080/09602011.2015.1094394

Freeman, D., Bradley, J., Antley, A., Bourke, E., DeWeever, N., Evans, N., . . .Clark, D. M. (2016).

Virtual reality in the treatment of persecutory delusions: Randomised controlled experimental study

testing how to reduce delusional conviction. The British Journal of Psychiatry, 209, 62–67.

10.1192/bjp.bp.115.176438

Freeman, D., Reeve, S., Robinson, A., Ehlers, A., Clark, D., Spanlang, B., & Slater, M. (2017). Virtual

reality in the assessment, understanding, and treatment of mental health disorders. Psychological

Medicine, 47, 1–8. 10.1017/S003329171700040X

Gamito, P., Oliveira, J., Brito, R., Lopes, P., Rodelo, L., Pinto, L., & Morais, D. (2016). Evaluation of

cognitive functions through the Systemic Lisbon Battery: Normative data. Methods of Information in

Medicine, 55, 93–97. 10.3414/ME14-02-0021

Gamito, P., Oliveira, J., Coelho, C., Morais, D., Lopes, P., Pacheco, J., . . .Barata, A. F. (2017).

Cognitive training on stroke patients via virtual reality-based serious games. Disability and

Rehabilitation, 39, 385–388. 10.3109/09638288.2014.934925

Gartner, Inc. (2016, July19). Hype cycle for emerging technologies, 2016. Retrieved from

http://www.gartner.com/newsroom/id/3412017

Goldman-Sachs. (January 13, 2006). Virtual and augmented reality understanding the race for the

next computing platform. Retrieved from http://www.goldmansachs.com/our-thinking/pages

/technology-driving-innovation-folder/virtual-and-augmented-reality/report.pdf

Gold, J. I., Kim, S. H., Kant, A. J., Joseph, M. H., & Rizzo, A. S. (2006). Effectiveness of virtual reality

for pediatric pain distraction during i.v. placement. Cyberpsychology & Behavior, 9, 207–212.

10.1089/cpb.2006.9.207

Goldman Sachs. (2016). Virtual & augmented reality: The next big computing platform?Retrieved

from http://www.goldmansachs.com/our-thinking/pages/virtual-and-augmented-reality-report.html

Granic, I., Lobel, A., & Engels, R. C. (2014). The benefits of playing video games. American

Psychologist, 69, 66–78. 10.1037/a0034857

Gratch, J., & Marsella, S. (2004). A domain independent framework for modeling emotion. Journal of

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

34 of 47 2/23/2019, 4:23 PM

Cognitive Systems Research, 5, 269–306. 10.1016/j.cogsys.2004.02.002

Gratch, J., Rickel, J., Andre, E., Cassell, J., Petajan, E., & Badler, N. (2002). Creating interactive

virtual humans: Some assembly required. IEEE Intelligent Systems, 17, 54–61.

10.1109/MIS.2002.1024753

Harrison, A., Derwent, G., Enticknap, A., Rose, F. D., & Attree, E. A. (2002). The role of virtual reality

technology in the assessment and training of inexperienced powered wheelchair users. Disability and

Rehabilitation, 24, 599–606. 10.1080/09638280110111360

Hoffman, H. G., Chambers, G. T., Meyer, W. J., III, Arceneaux, L. L., Russell, W. J., Seibel, E. J., . .

.Patterson, D. R. (2011). Virtual reality as an adjunctive non-pharmacologic analgesic for acute burn

pain during medical procedures. Annals of Behavioral Medicine, 41, 183–191.

10.1007/s12160-010-9248-7

Hoffman, H. G., Richards, T. L., Bills, A. R., Van Oostrom, T., Magula, J., Seibel, E. J., & Sharar, S. R.

(2006). Using FMRI to study the neural correlates of virtual reality analgesia. CNS Spectrums, 11,

45–51. 10.1017/S1092852900024202

Holden, M. K. (2005). Virtual environments for motor rehabilitation: Review. CyberPsychology and

Behavior, 8, 187–211.

Howard, M. C. (2017). A meta-analysis and systematic literature review of virtual reality rehabilitation

programs. Computers in Human Behavior, 70, 317–327. 10.1016/j.chb.2017.01.013

Hypergrid Business. (2016, August13). 5 VR apps that help you be a better public speaker. Retrieved

from http://www.hypergridbusiness.com/2016/08/5-vr-apps-that-help-you-be-a-better-public-speaker/

Innes, M., & Morrison, B. (2017). Projecting the future impact of advanced technologies: Will a robot

take my job?InPsych, 39. Retrieved from https://www.psychology.org.au/inpsych/2017/april/innes/

Jack, D., Boian, R., Merians, A. S., Tremaine, M., Burdea, G. C., Adamovich, S. V., . . .Poizner, H.

(2001). Virtual reality-enhanced stroke rehabilitation. IEEE Transactions on Neural Systems and

Rehabilitation Engineering, 9, 308–318. 10.1109/7333.948460

Jacobs, W. J., Laurance, H. E., & Thomas, K. (1997). Place learning in virtual space I: Acquisition,

overshadowing and transfer. Learning and Motivation, 28, 521–541. 10.1006/lmot.1997.0977

Jaffe, D. L., Brown, D. A., Pierson-Carey, C. D., Buckley, E. L., & Lew, H. L. (2004). Stepping over

obstacles to improve walking in individuals with poststroke hemiplegia. Journal of Rehabilitation

Research and Development, 41, 283–292. 10.1682/JRRD.2004.03.0283

Jentsch, F., & Curtis, M. (2017). Simulation in aviation training. New York, NY: Routledge.

Jin, W., Choo, A., Gromala, D., Shaw, C., & Squire, P. (2016). A virtual Reality game for chronic pain

management: A randomized, controlled clinical study. Studies in Health Technology and Informatics,

220, 154–160.

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

35 of 47 2/23/2019, 4:23 PM

John, N. W., Pop, S. R., Day, T. W., Ritsos, P. D., & Headleand, C. J. (2017). The implementation and

validation of a virtual environment for training powered wheelchair manoeuvres. IEEE Transactions

on Visualization and Computer Graphics. Advance online publication. 10.1109/TVCG.2017.2700273

Johnston, R. (1995). Is it live or is it memorized?Virtual Reality Special Report, 2, 53–56.

Josman, N., Ben-Chaim, H. M., Friedrich, S., & Weiss, P. L. (2008). Effectiveness of virtual reality for

teaching street-crossing skills to children and adolescents with autism. International Journal on

Disability and Human Development, 7, 49–56. 10.1515/IJDHD.2008.7.1.49

Josman, N., Kizony, R., Hof, E., Goldenberg, K., Weiss, P. L., & Klinger, E. (2014). Using the virtual

action planning-supermarket for evaluating executive functions in people with stroke. Journal of

Stroke and Cerebrovascular Diseases, 23, 879–887. 10.1016/j.jstrokecerebrovasdis.2013.07.013

Kaplan, A. Y., Shishkin, S. L., Ganin, I. P., Basyul, I. A., & Zhigalov, A. Y. (2013). Adapting the P300-

based brain-computer interface for gaming: A review. IEEE Transactions on Computational

Intelligence and AI in Games, 5, 141–149. 10.1109/TCIAIG.2012.2237517

Kato, P. M. (2010). Video games in health care: Closing the gap. Review of General Psychology, 14,

113–121. 10.1037/a0019441

Keefe, R. S. E., Davis, V. G., Atkins, A. S., Vaughan, A., Patterson, T., Narasimhan, M., & Harvey, P.

D. (2016). Validation of a Computerized test of Functional Capacity. Schizophrenia Research,

175(1–3), 90–96. 10.1016/j.schres.2016.03.038

Klamroth-Marganska, V., Blanco, J., Campen, K., Curt, A., Dietz, V., Ettlin, T., . . .Riener, R. (2014).

Three-dimensional, task-specific robot therapy of the arm after stroke: A multicentre, parallel-group

randomised trial. Lancet Neurology, 13, 159–166. 10.1016/S1474-4422(13)70305-3

Klinger, E. (2005, July 7–10). Virtual reality therapy for social phobia: Its efficacy through a control

study. Paper presented at Cybertherapy 2005, Basal, Switzerland.

Kober, S. E., Wood, G., Hofer, D., Kreuzig, W., Kiefer, M., & Neuper, C. (2013). Virtual reality in

neurologic rehabilitation of spatial disorientation. Journal of Neuroengineering and Rehabilitation.

Advance online publication. 10.1186/1743-0003-10-17

Koenig, S. T. (2012). Individualized virtual reality rehabilitation after brain injuries(Unpublished

doctoral dissertation). Human Interface Technology Lab New Zealand, College of Engineering,

University of Canterbury, Christchurch, New Zealand.

Koenig, S. T., Crucian, G. P., Dalrymple-Alford, J. C., & Dünser, A. (2010, August). Assessing

navigation in real and virtual environments: A validation study. Paper presented at the P. Sharkey, &

J. Sanchez (Eds.), Proceedings of the International Conference Series on Disability, Virtual Reality

and Associated Technologies, Vina del Mar, Chile.

Koenig, S. T., Krch, D., Chiaravalloti, N., Lengenfelder, J., Nikelshpur, O., Lange, B. S., . . .Rizzo, A.

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

36 of 47 2/23/2019, 4:23 PM

A. (2012). User-centered development of a virtual reality cognitive assessment. In P.Sharkey, &

E.Klinger (Eds.), Proceedings of the 9th International Conference on Disability, Virtual Reality and

Associated Technologies. Laval, France.

Krch, D., Koenig, S., Lavrador, S., Rizzo, A., & Chiaravalloti, N. D. (2013, August 26–29). Pilot results

from a virtual reality executive function task. Paper presented at the proceedings of the International

Conference on Virtual Rehabilitation, Philadelphia, PA.

Krueger, M. W. (1993). The experience society. Presence, 2, 162–168. 10.1162/pres.1993.2.2.162

Lamson, R. J. (1994). Virtual therapy of anxiety disorders. Cyberedge Journal, 4, 6–8.

Lange, B., Flynn, S., Proffitt, R., Chang, C. Y., & Rizzo, A. S. (2010). Development of an interactive

game-based rehabilitation tool for dynamic balance training. Topics in Stroke Rehabilitation, 17,

345–352. 10.1310/tsr1705-345

Lange, B. S., Flynn, S. M., & Rizzo, A. A. (2009). Initial usability assessment of off-the-shelf video

game consoles for clinical game-based motor rehabilitation. The Physical Therapy Review, 14,

355–363. 10.1179/108331909X12488667117258

Lange, B., Koenig, S., Chang, C.-Y., McConnell, E., Suma, E., Bolas, M., & Rizzo, A. (2012).

Designing informed game-based rehabilitation tasks leveraging advances in virtual reality. Disability

and Rehabilitation, 34, 1863–1870. 10.3109/09638288.2012.670029

Lanyi, C. S., Brown, D. J., Standen, P., Lewis, J., & Butkute, V. (2012). Results of user interface

evaluation of serious games for students with intellectual disability. Acta Polytechnica Hungarica, 9,

225–245.

Larson, E. B., Feigon, M., Gagliardo, P., & Dvorkin, A. Y. (2014). Virtual reality and cognitive

rehabilitation: A review of current outcome research. NeuroRehabilitation, 34, 759–772.

LaViola, J. J., Jr., Kruijff, E., McMahan, R. P., Bowman, D., & Poupyrev, I. P. (2017). 3D user

interfaces: Theory and practice (2nd ed.). Boston, MA: Addison Wesley.

Levin, M. F., Weiss, P. L., & Keshner, E. A. (2015). Emergence of virtual reality as a tool for upper

limb rehabilitation: Incorporation of motor control and motor learning principles. Physical Therapy, 95,

415–425. 10.2522/ptj.20130579

Levy, C. E., Halan, S., Silverman, E. P., Marsiske, M., Lehman, L., Omura, D., & Lok, B. C. (2015).

Virtual environments and virtual humans for military mild traumatic brain injury and posttraumatic

stress disorder: An emerging concept. American Journal of Physical Medicine & Rehabilitation, 94(4),

e31–e32. 10.1097/PHM.0000000000000248

Liu, L., Miyazaki, M., & Watson, B. (1999). Norms and validity of the DriVR: A virtual reality driving

assessment for persons with head injuries. Cyberpsychology & Behavior, 2, 53–67.

10.1089/cpb.1999.2.53

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

37 of 47 2/23/2019, 4:23 PM

Lucas, G. M., Gratch, J., King, A., & Morency, L.-P. (2014). It’s only a computer: Virtual humans

increase willingness to disclose. Computers in Human Behavior, 37, 94–100.

10.1016/j.chb.2014.04.043

Lumbreras, M., & Sanchez, J. (2000). Usability and cognitive impact of the interaction with 3D virtual

interactive acoustic environments by blind children. In P.Sharkey, A.Cesarani, L.Pugnetti, & A.Rizzo

(Eds.), In Proceedings of the 3rd international conference on disability, virtual reality, and associated

technology (pp. 129–136). Reading, UK: University of Reading.

Madary, M., & Metzinger, T. K. (2016). Real virtuality: A code of ethical conduct. recommendations for

good scientific practice and the consumers of VR-technology. Frontiers in Robotics and AI. Advance

online publication. 10.3389/frobt.2016.00003

Man, D. W. K., Ganesan, B., Yip, C. C. K., Lee, C. O. P., Tsang, S. Y. L., Yu, P. W. P., . . .Shum, D. H.

K. (2016). Validation of the virtual-reality prospective memory test (Hong Kong Chinese version) for

individuals with first-episode schizophrenia. Neuropsychological Rehabilitation. Advance online

publication. 10.1080/09602011.2016.1251949

Maples-Keller, J. L., Bunnell, B. E., Kim, S. J., & Rothbaum, B. O. (2017). The use of reality

technology in the treatment of anxiety and other psychiatric disorders. Harvard Review of Psychiatry,

25, 103–113. 10.1097/HRP.0000000000000138

Matheis, R. J., Schultheis, M. T., Tiersky, L. A., DeLuca, J., Millis, S. R., & Rizzo, A. (2007). Is

learning and memory different in a virtual environment?The Clinical Neuropsychologist, 21, 146–161.

10.1080/13854040601100668

McCall, C., Blascovich, J., Young, A., & Persky, S. (2009). Proxemic behaviors as predictors of

aggression towards Black (but not White) males in an immersive virtual environment. Social

Influence, 4, 138–154.

McComas, J., MacKay, M., & Pivik, J. (2002). Effectiveness of virtual reality for teaching pedestrian

safety. Cyberpsychology & Behavior, 5, 185–190. 10.1089/109493102760147150

McGeorge, P., Phillips, L. H., Crawford, J. R., Garden, S. E., Della Sala, S., Milne, A. B., . .

.Callander, J. (2001). Using virtual environments in the assessment of executive dysfunction.

Presence, 10, 375–383. 10.1162/1054746011470235

Merians, A. S., Fluet, G. G., Qiu, Q., Saleh, S., Lafond, I., & Adamovich, S. V. (2010). Integrated arm

and hand training using adaptive robotics and virtual reality simulations. In P.Sharkey & J.Sánchez

(Eds.), In Proceedings of the 2010 international conference on disability, virtual reality and associated

technology (pp. 213–222). Reading, UK: University of Reading.

Merians, A. S., Jack, D., Boian, R., Tremaine, M., Burdea, G. C., Adamovich, S. V., . . .Poizner, H.

(2002). Virtual reality-augmented rehabilitation for patients following stroke. Physical Therapy, 82,

898–915.

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

38 of 47 2/23/2019, 4:23 PM

Millan, J. R., Rupp, R., Müller-Putz, G. R., Murray-Smith, R., Giugliemma, C., et al. (2010).

Combining brain-computer interfaces and assistive technologies: State-of-the-art and challenges.

Frontiers in Neuroscience. Advance online publication. 10.3389/fnins.2010.00161

MindMaze. (2017). MindMaze MindMotion. Retrieved from https://www.mindmaze.com/mindmotion/

Morbini, F., DeVault, D., Sagae, K., Gerten, J., Nazarian, A., & Traum, D. (2014). FLoReS: A forward

looking, reward seeking, dialogue manager. In J.Mariani, S.Rosset, M.Garnier-Rizet, & L.Devillers

(Eds.), Natural interaction with robots, knowbots and smartphones (pp. 313–325). New York, NY:

Springer. 10.1007/978-1-4614-8280-2_28

Morina, N., Ijntema, H., Meyerbr€oker, K., & Emmelkamp, P. M. G. (2015). Can virtual reality

exposure therapy gains be generalized to real-life? A meta-analysis of studies applying behavioral

assessments. Behaviour Research and Therapy, 74, 18e24.

Morrison, J. E., & Meliza, L. L. (1999). Foundations of the after action review process. [Special Rep.

42]. Ft. Belvoir, VA: U. S. Army Research Institute for the Behavioral and Social Sciences.

Morrongiello, B. A., Corbett, M., Switzer, J., & Hall, T. (2015). Using a virtual environment to study

pedestrian behaviors: How does time pressure affect children’s and adults’ street crossing

behaviors?Journal of Pediatric Psychology, 40, 697–703. 10.1093/jpepsy/jsv019

Mosadeghi, S., Reid, M. W., Martinez, B., Rosen, B. T., & Spiegel, B. M. R. (2016). Feasibility of an

immersive reality intervention for hospitalized patients: An observational cohort study. JMIR Mental

Health, 3(2), e28. 10.2196/mental.5801

Mowafty, L., & Pollack, J. (1995). Train to travel. Ability, 15, 18–20.

Mühlberger, A., Jekel, K., Probst, T., Schecklmann, M., Conzelmann, A., Andreatta, M., . . .Romanos,

M. (2016). The influence of methylphenidate on hyperactivity and attention deficits in children with

ADHD: A virtual classroom test. Journal of Attention Disorders. Advance online publication.

10.1177/1087054716647480

Navarro, M. D., Lloréns, R., Noé, E., Ferri, J., & Alcañiz, M. (2013). Validation of a low-cost virtual

reality system for training street-crossing. A comparative study in healthy, neglected and non-

neglected stroke individuals. Neuropsychological Rehabilitation, 23, 597–618.

10.1080/09602011.2013.806269

Naveh, Y., Katz, N., & Weiss, P. (2000). The effect of interactive virtual environment training on

independent safe street crossing of right CVA patients with unilateral spatial neglect. In P.Sharkey,

A.Cesarani, L.Pugnetti, & A.Rizzo (Eds.), In Proceedings of the 3rd international conference on

disability, virtual reality, and associated technology (pp. 243–248). Reading, UK: University of

Reading.

Neisser, U. (1978). Memory: What are the important questions? In M. M.Gruneberg, P. E.Morris, & R.

N.Sykes (Eds.), Practical aspects of memory (pp. 3–24). London, England: Academic Press.

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

39 of 47 2/23/2019, 4:23 PM

Neri, S. G. R., Cardoso, J. R., Cruz, L., Lima, R. M., de Oliveira, R. J., Iversen, M. D., & Carregaro,

R. L. (2017). Do virtual reality games improve mobility skills and balance measurements in

community-dwelling older adults? Systematic review and meta-analysis. Clinical Rehabilitation.

Advance online publication. 10.1177/0269215517694677

Nir-Hadad, S. Y., Weiss, P. L., Waizman, A., Schwartz, N., & Kizony, R. (2015). A virtual shopping

task for the assessment of executive functions: Validity for people with stroke. Neuropsychological

Rehabilitation, 27, 808–833.

Norcross, J. C., Pfund, R. A., & Prochaska, J. O. (2013). Psychotherapy in 2022. A Delphi poll on its

future. Professional Psychology, Research and Practice, 44, 363–370. 10.1037/a0034633

Norrholm, S. D., Jovanovic, T., Gerardi, M., Breazeale, K. G., Price, M., Davis, M., . . .Rothbaum, B.

O. (2016). Baseline psychophysiological and cortisol reactivity as a predictor of PTSD treatment

outcome in virtual reality exposure therapy. Behaviour Research and Therapy, 82, 28–37.

10.1016/j.brat.2016.05.002

Ogourtsova, T., Silva, W. S., Archambault, P. S., & Lamontagne, A. (2015). Virtual reality treatment

and assessment for post-stroke unilateral spatial neglect: A systematic review. Neuropsychological

Rehabilitation, 27, 409–454.

Opriş, D., Pintea, S., García-Palacios, A., Botella, C., Szamosközi, Ş., & David, D. (2012). Virtual

reality exposure therapy in anxiety disorders: A quantitative meta-analysis. Depression and Anxiety,

29, 85–93. 10.1002/da.20910

Padgett, L. S., Strickland, D., & Coles, C. D. (2006). Case study: Using a virtual reality computer

game to teach fire safety skills to children diagnosed with fetal alcohol syndrome. Journal of Pediatric

Psychology, 31, 65–70. 10.1093/jpepsy/jsj030

Papastergiou, M. (2009). Exploring the potential of computer and video games for health and

physical education: A literature review. Computers & Education, 53, 603–622.

10.1016/j.compedu.2009.04.001

Park, S., Scherer, S., Gratch, J., Carnevale, P., & Morency, L. P. (2013, September). Mutual

behaviors during dyadic negotiation: Automatic prediction of respondent reactions. In T.Pun,

C.Pelachaud, & N.Sebe (Eds.), 2013 Humaine Association Conference on Affective Computing and

Intelligent Interaction (ACII) (pp. 423–428). Los Alomitos, CA: IEEE.

Parsons, S., Bauminger, N., Cobb, S., Gal, E., Glover, T., Weiss, P. L., . . .Rietdijk, W. (2012,

July6–8). Collaborative technologies for supporting social conversation and collaboration in the

classroom: Overview and findings of the COSPATIAL project. Paper presented at the First

International Conference on Innovative Technologies for Autism Spectrum Disorders: Tools, Trends

and Testimonials, Valencia, Spain.

Parsons, T. D. (2015). Virtual reality for enhanced ecological validity and experimental control in the

clinical, affective, and social neurosciences. Frontiers in Human Neuroscience, 9, 660.

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

40 of 47 2/23/2019, 4:23 PM

10.3389/fnhum.2015.00660

Parsons, T. D., Carlew, A. R., Magtoto, J., & Stonecipher, K. (2015). The potential of function-led

virtual environments for ecologically valid measures of executive function in experimental and clinical

neuropsychology. Neuropsychological Rehabilitation, 27, 777–807. 10.1080/09602011.2015.1109524

Parsons, T. D., & Rizzo, A. A. (2008a). Affective outcomes of virtual reality exposure therapy for

anxiety and specific phobias: A meta-analysis. Journal of Behavior Therapy and Experimental

Psychiatry, 39, 250–261. 10.1016/j.jbtep.2007.07.007

Parsons, T. D., & Rizzo, A. A. (2008b). Initial validation of a virtual environment for assessment of

memory functioning: Virtual reality cognitive performance assessment test. Cyberpsychology &

Behavior, 11, 17–25. 10.1089/cpb.2007.9934

Parsons, T. D., Rizzo, A. A., Rogers, S., & York, P. (2009). Virtual reality in paediatric rehabilitation: A

review. Developmental Neurorehabilitation, 12, 224–238. 10.1080/17518420902991719

Passig, D., Tzuriel, D., & Eshel-Kedmi, G. (2016). Improving children’s cognitive modifiability by

dynamic assessment in 3D Immersive Virtual Reality environments. Computers & Education, 95,

296–308. 10.1016/j.compedu.2016.01.009

Pertaub, D-P., Slater, M., & Barker, C. (2002). An Experiment on Public Speaking Anxiety in

Response to Three Different Types of Virtual Audience. Presence, 11, 68–78.

10.1162/105474602317343668

Pietrzak, E., Pullman, S., & McGuire, A. (2014). Using Virtual Reality and Videogames for Traumatic

Brain Injury Rehabilitation: A Structured Literature Review. Games For Health Journal: Research,

Development, and Clinical Applications, 3, 202–214. 10.1089/g4h.2014.0013

Piron, L., Tonin, P., Atzori, A. M., Zanotti, E., Massaro, C., Trivello, E., & Dam, M. (2002). Virtual

environment system for motor tele-rehabilitation. Studies in Health Technology and Informatics, 85,

355–361.

Powers, M. B., & Emmelkamp, P. M. G. (2008). Virtual reality exposure therapy for anxiety disorders:

A meta-analysis. Journal of Anxiety Disorders, 22, 561–569. 10.1016/j.janxdis.2007.04.006

Proffitt, R., & Lange, B. (2015). Feasibility of a customized, in-home, game-based stroke exercise

program using the Microsoft Kinect® Sensor. International Journal of Telerehabilitation, 7, 23–34.

10.5195/IJT.2015.6177

Pugnetti, L., Mendozzi, L., Motta, A., Cattaneo, A., Barbieri, E., & Brancotti, A. (1995). Evaluation and

retraining of adults’ cognitive impairment: Which role for virtual reality technology?Computers in

Biology and Medicine, 25, 213–227. 10.1016/0010-4825(94)00040-W

Putrino, D. (2014). Telerehabilitation and emerging virtual reality approaches to stroke rehabilitation.

Current Opinion in Neurology, 27, 631–636. 10.1097/WCO.0000000000000152

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

41 of 47 2/23/2019, 4:23 PM

Regenbrecht, H., Hoermann, S., Ott, C., Müller, L., & Franz, E. (2014). Manipulating the experience

of reality for rehabilitation applications. Proceedings of the IEEE, 102, 170–184.

10.1109/JPROC.2013.2294178

Remsik, A., Young, B., Vermilyea, R., Kiekhoefer, L., Abrams, J., Evander Elmore, S., . .

.Prabhakaran, V. (2016). A review of the progression and future implications of brain-computer

interface therapies for restoration of distal upper extremity motor function after stroke. Expert Review

of Medical Devices, 13, 445–454. 10.1080/17434440.2016.1174572

Riva, G. (2011). The key to unlocking the virtual body: Virtual reality in the treatment of obesity and

eating disorders. Journal of Diabetes Science and Technology, 5, 283–292.

10.1177/193229681100500213

Rizzo, A. A. (1994). Virtual reality applications for the cognitive rehabilitation of persons with

traumatic head injuries. In H. J.Murphy (Ed.), In Proceedings of the 2nd international conference on

virtual reality and persons with disabilities. Retrieved from http://www.csun.edu/cod/conf

/1994/proceedings/Table94.htm

Rizzo, A. A., Bowerly, T., Buckwalter, J. G., Klimchuk, D., Mitura, R., & Parsons, T. D. (2006). A

virtual reality scenario for all seasons: The virtual classroom. CNS Spectrums, 11, 35–44.

10.1017/S1092852900024196

Rizzo, A. A., Buckwalter, J. G., Forbell, E., Reist, C., Difede, J., Rothbaum, B. O., . . .Talbot, B.

(2013). Virtual reality applications to address the wounds of war. Psychiatric Annals, 43, 123–138.

10.3928/00485713-20130306-08

Rizzo, A. A., Buckwalter, J. G., Humphrey, L., van der Zaag, C., Bowerly, T., Chua, C., . . .Sisemore,

D. (2000). The virtual classroom: A virtual environment for the assessment and rehabilitation of

attention deficits. Cyberpsychology & Behavior, 3, 483–499. 10.1089/10949310050078940

Rizzo, A. A., Buckwalter, J. G., & Neumann, U. (1997). Virtual reality and cognitive rehabilitation: A

brief review of the future. The Journal of Head Trauma Rehabilitation, 12, 1–15.

10.1097/00001199-199712000-00002

Rizzo, A. A., Cukor, J., Gerardi, M., Alley, S., Reist, C., Roy, M., . . .Difede, J. (2015a). Virtual reality

exposure therapy for PTSD due to military combat and terrorist attacks. Journal of Contemporary

Psychotherapy, 45, 255–264. 10.1007/s10879-015-9306-3

Rizzo, A., Difede, J., Rothbaum, B. O., & Reger, G. (2010). Virtual Iraq/Afghanistan: Development

and early evaluation of a virtual reality exposure therapy system for combat-related PTSD. Annals of

the New York Academy of Sciences, 1208, 114–125. 10.1111/j.1749-6632.2010.05755.x

Rizzo, A. A., Kenny, P., & Parsons, T. (2011). Intelligent virtual humans for clinical training.

International Journal of Virtual Reality and Broadcasting, 8(3). Retrieved from http://www.jvrb.org

/8.2011/

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

42 of 47 2/23/2019, 4:23 PM

Rizzo, A. A., Lucas, G., Gratch, J., Stratou, G., Morency, L.-P., Shilling, R., & Scherer, S. (2016b).

Clinical interviewing by a virtual human agent with automatic behavior analysis. In P.Sharkey and A.

A.Rizzo (Eds.), 2016 Proceedings of the international conference on disability, virtual reality and

associated technologies (pp. 57–64). Reading, UK: University of Reading. Retrieved from

http://www.icdvrat.org/2014/papers/ICDVRAT2014_S03N3_Rizzo_etal.pdf

Rizzo, A. A., Schultheis, M. T., Kerns, K., & Mateer, C. (2004). Analysis of Assets for Virtual Reality

Applications in Neuropsychology. Neuropsychological Rehabilitation, 14(1/2), 207–239.

10.1080/09602010343000183

Rizzo, A., Schultheis, M. T., & Rothbaum, B. O. (2003). Ethical issues for the use of virtual reality in

the psychological sciences. In S. S.Bush, & M. L.Drexler (Eds.), Ethical issues in clinical

neuropsychology (pp. 243–280). Amsterdam, the Netherlands: Swets & Zeitlinger.

Rizzo, A. A., Shilling, R., & Forbell, E. D., Scherer, S., Gratch, J., & Morency, L.-P. (2015b).

Autonomous virtual human agents for healthcare information support and clinical interviewing. In D.

D.Luxton (Ed.), Artificial intelligence in mental healthcare practice (pp. 53–80). New York, NY:

Academic Press.

Rizzo, A. A., Strickland, D., & Bouchard, S. (2004). Issues and challenges for virtual environments in

telerehabilitation. Telemedicine Journal and e-Health, 10, 184–195. 10.1089/tmj.2004.10.184

Rizzo, A. A., & Talbot, T. (2016a). Virtual reality standardized patients for clinical training. In C.

D.Combs, J. A.Sokolowski, & C. M.Banks (Eds.), The digital patient: Advancing medical research,

education, and practice (pp. 257–272). New York, NY: Wiley. 10.1002/9781118952788.ch18

Rizzo, M., Reinach, S., McGehee, D., & Dawson, J. (1997). Simulated car crashes and crash

predictors in drivers with Alzheimer disease. Archives of Neurology, 54, 545–551.

10.1001/archneur.1997.00550170027011

Robertson, I. (1990). Does computerized cognitive rehabilitation work? A review. Aphasiology, 4,

381–405. 10.1080/02687039008249090

Rose, D., Brooks, B. M., & Attree, E. A. (2000). Virtual reality in vocational training of people with

learning disabilities. In P.Sharkey, A.Cesarani, L.Pugnetti, & A.Rizzo (Eds.), In Proceedings of the 3rd

international conference on disability, virtual reality, and associated technology (pp. 129–136).

Reading, UK: University of Reading.

Rose, F. D., Attree, E. A., Brooks, B. M., & Andrews, T. K. (2001). Learning and memory in virtual

environments—A role in neurorehabilitation? Questions (and occasional answers) from UEL.

Presence, 10, 345–358. 10.1162/1054746011470208

Rose, F. D., Brooks, B. M., & Rizzo, A. A. (2005). Virtual reality in brain damage rehabilitation:

Review. Cyberpsychology & Behavior, 8, 241–262. 10.1089/cpb.2005.8.241

Rothbaum, B. O., & Hodges, L. F. (1999). The use of virtual reality exposure in the treatment of

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

43 of 47 2/23/2019, 4:23 PM

anxiety disorders. Behavior Modification, 23, 507–525. 10.1177/0145445599234001

Rothbaum, B. O., Hodges, L. F., Kooper, R., Opdyke, D., Williford, J. S., & North, M. (1995). Virtual

reality graded exposure in the treatment of acrophobia: A case report. Behavior Therapy, 26,

547–554. 10.1016/S0005-7894(05)80100-5

Rothbaum, B. O., Hodges, L. F., Ready, D., Graap, K., & Alarcon, R. D. (2001). Virtual reality

exposure therapy for Vietnam veterans with posttraumatic stress disorder. The Journal of Clinical

Psychiatry, 62, 617–622. 10.4088/JCP.v62n0808

Rothbaum, B. O., Price, M., Jovanovic, T., Norrholm, S. D., Gerardi, M., Dunlop, B., . . .Ressler, K. J.

(2014). A randomized, double-blind evaluation of D-cycloserine or alprazolam combined with virtual

reality exposure therapy for posttraumatic stress disorder in Iraq and Afghanistan War veterans. The

American Journal of Psychiatry, 171, 640–648. 10.1176/appi.ajp.2014.13121625

Roy, M. J., Costanzo, M. E., Blair, J. R., & Rizzo, A. A. (2014). Compelling evidence that exposure

therapy for PTSD normalizes brain function. Studies in Health Technology and Informatics, 199,

61–65.

Roy, M. J., Francis, J., Friedlander, J., Banks-Williams, L., Lande, R. G., Taylor, P., . . .Rothbaum, B.

(2010). Improvement in cerebral function with treatment of posttraumatic stress disorder. Annals of

the New York Academy of Sciences, 1208, 142–149. 10.1111/j.1749-6632.2010.05689.x

Rutten, A., Cobb, S., Neale, H., Kerr, S., Leonard, A., Parsons, S., & Mitchell, P. (2003). The AS

interactive project: Single-user and collaborative virtual environments for people with high-functioning

autistic spectrum disorders. Journal of Visualization and Computer Animation, 14, 233–241.

10.1002/vis.320

Sbordone, R. J., & Long, C. (1996). Ecological validity of neuropsychological testing. Boca Raton,

FL: CRC Press.

Schell, J. (2014). The art of game design: A book of lenses. Boca Raton, FL: CRC Press.

10.1201/b17723

Scherer, S., Stratou, G., Lucas, G., Mahmoud, M., Boberg, J., Gratch, J., . . .Morency, L. P. (2014).

Automatic audiovisual behavior descriptors for psychological disorder analysis. Image and Vision

Computing, 32, 648–658. 10.1016/j.imavis.2014.06.001

Schneider, S. M., Kisby, C. K., & Flint, E. P. (2011, April). Effect of virtual reality on time perception in

patients receiving chemotherapy. Supportive Care in Cancer, 19, 555–564.

10.1007/s00520-010-0852-7

Schultheis, M. T., & Mourant, R. R. (2001). Virtual reality and driving: The road to better assessment

of cognitively impaired populations. Presence, 10, 436–444. 10.1162/1054746011470271

Schwebel, D. C., McClure, L. A., & Severson, J. (2014). Teaching children to cross streets safely: A

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

44 of 47 2/23/2019, 4:23 PM

randomized, controlled trial. Health Psychology, 33, 628–638. 10.1037/hea0000032

Shipman, S. L., & Astur, R. S. (2008). Factors affecting the hippocampal BOLD response during

spatial memory. Behavioural Brain Research, 187, 433–441. 10.1016/j.bbr.2007.10.014

SilverFit. (2017). SilverFit. [Application]. Retrieved from http://silverfit.com/en/applications

Skelton, R. W., Bukach, C. M., Laurance, H. E., Thomas, K. G., & Jacobs, J. W. (2000). Humans with

traumatic brain injuries show place-learning deficits in computer-generated virtual space. Journal of

Clinical and Experimental Neuropsychology, 22, 157–175.

10.1076/1380-3395(200004)22:2;1-1;FT157

Slater, M., & Sanchez-Vives, M. V. (2016, December). Enhancing our lives with immersive virtual

reality. Frontiers in Robotics and Artificial Intelligence. Advance online publication.

10.3389/frobt.2016.00074

Sohlberg, M. K. M., & Mateer, C. A. (1989). Introduction to cognitive rehabilitation: Theory and

practice. New York, NY: Guilford Press.

Sohlberg, M. K. M., & Mateer, C. A. (2001). Cognitive rehabilitation: An integrative

neuropsychological approach. New York, NY: Guilford Press.

Spek, V., Cuijpers, P., Nyklícek, I., Riper, H., Keyzer, J., & Pop, V. (2007). Internet-based cognitive

behaviour therapy for symptoms of depression and anxiety: A meta-analysis. Psychological

Medicine, 37, 319–328. 10.1017/S0033291706008944

Stamm, B. H. (1998). Clinical applications of telehealth in mental health care. Professional

Psychology, Research and Practice, 29, 536–542. 10.1037/0735-7028.29.6.536

Standen, P. J., Threapleton, K., Connell, L., Richardson, A., Brown, D. J., Battersby, S., . . .Platts, F.

(2014). Patients’ use of home-based virtual reality system to provide rehabilitation of the upper limb

following stroke. Physical Therapy, 95, 350–359. Retrieved from http://irep.ntu.ac.uk/id/eprint/25917

/1/221608_PubSub3351_Brown.pdf

Stanton, D., Foreman, N., & Wilson, P. (1998). Uses of virtual reality in clinical training: Developing

the spatial skills of children with mobility impairments. In G.Riva, B.Wiederhold, & E.Molinari (Eds.),

Virtual reality in clinical psychology and neuroscience (pp. 219–232). Amsterdam, the Netherlands:

IOS Press.

Strickland, D. (2001, August16). Virtual reality and multimedia applications for mental health

assessment and treatment. Paper presented at the ACM SIGGRAPH 2001 Conference, Los

Angeles, CA.

Talbot, T. B., Sagae, K., John, B., & Rizzo, A. A. (2012). Sorting out the Virtual Patient: How to exploit

artificial intelligence, game technology and sound educational practices to create engaging role-

playing simulations. International Journal of Gaming and Computer-Mediated Simulations, 4, 1–19.

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

45 of 47 2/23/2019, 4:23 PM

10.4018/jgcms.2012070101

Tarr, M. J., & Warren, W. H. (2002). Virtual reality in behavioral neuroscience and beyond. Nature

Neuroscience, 5(Suppl.), 1089–1092. 10.1038/nn948

Tart, C. T. (1993). Mind embodied: Computer generated virtual reality as a new, interactive dualism.

In K.Ramakrishna Rao (Ed.), Cultivating consciousness (pp. 123–137). Santa Barbara, CA: Praeger.

Tashjian, V. C., Mosadeghi, S., Howard, A. R., Lopez, M., Dupuy, T., Reid, M., . . .Spiegel, B. (2017).

Virtual reality for management of pain in hospitalized patients: Results of a controlled trial. JMIR Ment

Health, 4(1), e9. 10.2196/mental.7387

Tegos, S., Demetriadis, S., Papadopoulos, P. M., & Weinberger, A. (2016). Conversational agents for

academically productive talk: A comparison of directed and undirected agent interventions.

International Journal of Computer-Supported Collaborative Learning, 11, 417–440.

10.1007/s11412-016-9246-2

Thiebaux, M., Marsella, S., Marshall, A. N., & Kallmann, M. (2008, May). Smartbody: Behavior

realization for embodied conversational agents. In L.Padgham, & D.Parkes (Eds.), In Proceedings of

the 7th international joint conference on Autonomous agents and multiagent systems-Volume 1 (pp.

151–158). New York, NY: ACM.

Trost, Z., Zielke, M., Guck, A., Nowlin, L., Zakhidov, D., France, C. R., & Keefe, F. (2015). The

promise and challenge of virtual gaming technologies for chronic pain: The case of graded exposure

for low back pain. Pain Management, 5, 197–206. 10.2217/pmt.15.6

Valladares-Rodríguez, S., Pérez-Rodríguez, R., Anido-Rifón, L., & Fernández-Iglesias, M. (2016).

Trends on the application of serious games to neuropsychological evaluation: A scoping review.

Journal of Biomedical Informatics, 64, 296–319. 10.1016/j.jbi.2016.10.019

Valve Corporation. (2017). Steam. Retrieved from http://store.steampowered.com/

Virtual Reality Society. (2017). Applications of virtual reality. Retrieved from https://www.vrs.org.uk

/virtual-reality-applications/

Wall, K. J., Cumming, T. B., Koenig, S. T., Pelecanos, A. M., & Copland, D. A. (2017). Using

technology to overcome the language barrier: The Cognitive Assessment for Aphasia App. Disability

and Rehabilitation. Advance online publication. 10.1080/09638288.2017.1294210

Wiederhold, B. K., & Wiederhold, M. D. (1998). A review of virtual reality as a psychotherapeutic tool.

Cyberpsychology & Behavior, 1, 45–52. 10.1089/cpb.1998.1.45

Wilson, B. A. (1997). Cognitive rehabilitation: How it is and how it might be. Journal of the

International Neuropsychological Society, 3, 487–496.

Wilson, B. A., Baddeley, A., Evans, J., & Shiel, A. (1994). Errorless learning in the rehabilitation of

memory impaired people. Neuropsychological Rehabilitation, 4, 307–326.

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

46 of 47 2/23/2019, 4:23 PM

10.1080/09602019408401463

Wilson, B. A., & Evans, J. J. (1996). Error-free learning in the rehabilitation of people with memory

impairments. The Journal of Head Trauma Rehabilitation, 11, 54–64.

10.1097/00001199-199604000-00006

Wilson, J. A., Onorati, K., Mishkind, M., Reger, M. A., & Gahm, G. A. (2008). Soldier attitudes about

technology-based approaches to mental health care. Cyberpsychology & Behavior, 11, 767–769.

10.1089/cpb.2008.0071

Yang, Y. J. D., Allen, T., Abdullahi, S. M., Pelphrey, K. A., Volkmar, F. R., & Chapman, S. B. (2017).

Brain responses to biological motion predict treatment outcome in young adults with autism receiving

Virtual Reality Social Cognition Training: Preliminary findings. Behaviour Research and Therapy, 93,

55–66. 10.1016/j.brat.2017.03.014

Yellowlees, P. M., Holloway, K. M., & Parish, M. B. (2012). Therapy in virtual environments—Clinical

and ethical issues. Telemedicine Journal and e-Health, 18, 558–564. 10.1089/tmj.2011.0195

You, S. H., Jang, S. H., Kim, Y. H., Hallett, M., Ahn, S. H., Kwon, Y. H., . . .Lee, M. Y. (2005). Virtual

reality-induced cortical reorganization and associated locomotor recovery in chronic stroke: An

experimenter-blind randomized study. Stroke, 36, 1166–1171.

10.1161/01.STR.0000162715.43417.91

Zhang, Z., Bickmore, T. W., & Paasche-Orlow, M. K. (2017). Perceived organizational affiliation and

its effects on patient trust: Role modeling with embodied conversational agents. Patient Education

and Counseling, 100, 1730–1737. 10.1016/j.pec.2017.03.017

Submitted: June 22, 2017 Revised: July 13, 2017 Accepted: July 14, 2017

This publication is protected by US and international copyright laws and its content may not be

copied without the copyright holders express written permission except for the print or download

capabilities of the retrieval software used for access. This content is intended solely for the use of the

individual user.

Source: Neuropsychology. Vol. 31. (8), Nov, 2017 pp. 877-899)

Accession Number: 2018-03502-004

Digital Object Identifier: 10.1037/neu0000405

EBSCOhost http://web.a.ebscohost.com.library.capella.edu/ehost/delivery?sid=e7be7...

47 of 47 2/23/2019, 4:23 PM