Human Computer Interaction

Stress training and simulator complexity: why sometimes more is less

Jennifer G. Tichon* and Guy M. Wallis

The School of Human Movement Studies, The University of Queensland, St Lucia, 4072, Australia

(Received 20 January 2009; final version received 15 October 2009)

Through repeated practice under conditions similar to those in real-world settings, simulator training prepares an individual to maintain e!ective performance under stressful work conditions. Interfaces o!ering high fidelity and immersion can more closely reproduce real-world experiences and are generally believed to result in better learning outcomes. However, absolute fidelity in stress training is not critical for skills to be transferable. The present study compared the performance outcomes achieved by trainees using two di!erent simulator types to complete a training program aimed at improving decision-making skills. The purpose of this research was to assess both the overall level of training e!ectiveness and to determine whether performance levels were influenced when high (160 degree curved wide screen) versus low fidelity (small cab-based flat screen) simulator types were in use. Sixty-three train drivers drove for 40 min on a simulated track on which they encountered four major high stress driving events. One year later, 42 of the original drivers returned and repeated the training scenario a second and third time. Results revealed trainees using the lower fidelity flat screen simulator made fewer errors in both years than trainees using the high fidelity curved screen simulator. The implications of these results are discussed.

Keywords: task analysis; simulator; stress; cognitive training; presence

1. Introduction

In work environments where heavy, human operated machinery is at work, accidents are an unfortunate but often inevitable part of everyday life. An essential element, in training to respond to workplace accidents e!ectively, is how to train sta! to cope with ill-defined problems in the face of high stress environments constrained by factors such as insu"cient or unreliable information, time constraints and or conditions of threat of physical injury (Salas et al. 1999). Emergency response skills, particularly those that degrade under stress such as critical thinking and decision-making, have been highlighted across a number of industries as in need of attention.

The question of how to most e!ectively train higher-order mental abilities in these situations is now emerging as a focus of research interest. Suitable training in critical thinking under stress can hugely reduce the impact of workplace accidents, not only in terms of the immediate physical well-being of sta! and clients, but also the long-term mental health of those involved. Simulators through recreating real-world situations in virtual environments provide a compel- ling opportunity for meeting the need for high a!ect training. Accordingly simulator training has been established as a core component of safety training programs. High fidelity, interactive simulation is

typically achieved through complete immersion in brief, stressful and complex scenarios.

2. Stress exposure training via simulation

When training goes beyond the acquisition of stan- dard, required knowledge and abilities and instead aims to prepare trainees to perform e!ectively in a stressful environment, it is commonly referred to as stress exposure training (Driskell and Johnston 1998). Stress exposure training focuses on developing those cognitive skills required to maintain e!ective perfor- mance under stress. The overall goal of training via simulation is to build confidence in sta! in their own ability to perform under adverse conditions (Stetz et al. 2006). Sub-goals of stress exposure training include gaining specific knowledge of and familiarity with the operational environment to assist trainees to form accurate expectations of the environment, increasing their ability to predict outcomes, avoid errors and decrease their propensity to be distracted by novel sensations (Driskell and Johnston 1998, Hulse and Memon 2006). Simulator training has been shown to be e!ective when the trainee experiences success or a sense of mastery during the training (Maschuw et al. 2008). This confidence translates into being less aroused physiologically, less distracted and more likely to focus on the task.

*Corresponding author. Email: [email protected]

Behaviour & Information Technology Vol. 29, No. 5, September–October 2010, 459–466

ISSN 0144-929X print/ISSN 1362-3001 online ! 2010 Taylor & Francis DOI: 10.1080/01449290903420184 http://www.informaworld.com

Simulators are used for instruction and training in areas such as commercial and military aviation, battle field management, building construction, and first responder agencies because they provide a safe alternative to replicating high-risk real-world training scenarios. Real-world renditions of these situations are too costly and dangerous to provide opportunities for sta! to repeatedly practise responding to high stress workplace situations (Tichon et al. 2003). It is the ability of immersive simulation to engender the same psychological and psychological responses experienced in the real world that has been seen as valuable (Romano and Brna 2001).

3. High fidelity or low fidelity?

A crucial element of the approach is to provide the opportunity for repeated practice under operational conditions similar to those likely to be encountered in real-world settings. The aim is to incite the same emotional (stress) reaction and stems from research demonstrating that for some tasks normal training procedures did not improve task performance when the task was later performed under stress conditions (Zakay and Wooler 1984).

The use of simulator technology alone, however, cannot ensure a successful training outcome without prior consideration of the best use of its features (Cannon-Bowers and Salas 1998, Wallis et al. 2007). Prior research has described a number of variables of interest to simulation-based critical decision-making training but the best use of these is not always agreed on (Thomas 2003). There is a debate over the level of fidelity required for successful training outcomes. Some suggest that when using simulation for cognitive skills training, the fidelity should only match the require- ments of the training such as facilitating reflection and learning in the training experience (Ho!man et al. 2001). Yet, it is also recognised that the similarity of the training environment to the actual conditions under which the trainee will perform in the real world is an important factor in simulation training design. While realistic simulations do not always translate directly to training e!ectiveness, good outcomes have been re- ported when simulations have reproduced realistic tasks and a!orded trainees an engrossing experience they can relate to their real world (Baker et al. 2005). A study of simulator training in underground cave structures reported task performance was two to three times faster in the high-immersion condition, and question responses were 3 to 10 times more accurate (Bowman and McMahan 2007).

Despite increasing a!ordability high-end immersive technologies such as head mounted displays and multi- screen stereoscopic projection displays remain too

expensive for widespread use. In training applications significant improvements in performance using simu- lators in comparison to alternative training support tools would have to be proven to justify the expense. Consequently, questions such as ‘How much immer- sion is enough?’ and ‘When are driving environments real enough?’ are being more and more frequently asked (Walshe et al. 2005, Bowman and McMahon 2007). Research has suggested that in driving applica- tions specifically di"culties can arise if users are not immersed su"ciently in simulated driving scenes (Nanyue et al. 2006). Is there a compromise between the traditional alternatives of textbooks and classroom teaching and the high-end, high fidelity that can still produce a level of realism su"cient to gain the required performance outcomes of stress exposure training?

The overall purpose of this research was to assess the level of training e!ectiveness in terms of enhanced decision-making under stress. To achieve this, compar- isons among performance outcomes over three di!erent training exposures was made. Additionally, the study design allowed for comparison of the performance results of trainees using a high fidelity curved wide- screen simulator with the performance of trainees using a smaller cab-based flat screen simulator. Reactions to four separate stressful events were examined for e!ect of simulator type on learning outcomes.

As it is the illusion of being in a di!erent place that is essential for the e!ectiveness of these training applications (Romano and Brna 2001, Baker et al. 2005), the two simulator types were also compared in terms of their e!ectiveness in generating immersion and thereby engaging the driver’s attention. Presence is a variable widely accepted as being positively related to enhanced learning and performance in immersive training environments and therefore highly desirable (Witmer and Singer 1998). It facilitates the environ- ment replicated in the simulator being interpreted as real by the trainee thereby making it more likely decision-making skills developed will be transferred to the real world. Presence has been described ‘as the subjective experience of being in one place or environ- ment, even when one is physically situated in another’ (Witmer and Singer 1998). Research investigating the role of presence in facilitating the e!ectiveness of simulator-based educational training indicates the two seem to be inextricably tied.

4. Method

4.1. Participants

4.1.1. Year one

In 2006, the first year of the project, 63 participants volunteered to take part in the study. The 61 males and

460 J.G. Tichon and G.M. Wallis

2 females possessed an average of 14.8 (SD ! 9.68) years’ experience driving trains, with a range between 1 and 30 years of experience. The drivers were aged from 29 to 58 years of age with a mean age of 43.3 (SD ! 8.14) years.

Twelve participants were trained in the wide-screen reality centre while 51 were trained in the cab-based simulator. The unequal sample size for the simulators was due to drivers being given the opportunity to self- select the simulator they would use for training. This training was part of their annual training and drivers were allowed to choose their simulator as per normal operating procedures. As drivers were being assessed via their simulation performance they were allowed to select the simulator they felt the most comfortable and confident using. Self-selection was also allowed for the study as it did not result in a driver being randomly assigned a simulator with which they were unfamiliar. A no-control group design was used at the request of the rail company which required all drivers to undergo the annual safety training.

4.1.2. Year two

Forty-two participants who participated in the 2006 study above returned in 2007. However, due to a malfunction with the wide-screen simulator eight of these drivers were reassigned the cab-based simulator. As this was not the simulator they had used in 2006 these eight sets of results could not be used in this study. Because of the expense of bringing active drivers o!-line for the research it could not be rescheduled for another time. Of the remaining 34 participants, 33 were males and 1 was a female. They possessed an average of 12.9 (SD ! 9.94) years’ experience driving trains, with a range between 1 and 32 years of experience. The drivers were aged from 30 to 59 years of age with a mean age of 43.5 (SD ! 8.66) years. Again partici- pants volunteered to take part in the study.

4.2. Equipment

Results were obtained from two simulator types:

(1) Cab-based. A reconfigurable, full-size, in-cab simulator which can reproduce the arrange- ment of standard in-cab train controls. Visual display input was provided via a flat projection screen (3 m wide 6 2 m high) and driven by a single-channel, SGI Onyx 2 (Four 400 MHz CPUs).

(2) Wide-screen. A curved screen reality centre 7.3 m in diameter o!ering a 160 degree field-of- view from a centrally located train control desk. The desk has force-feedback controls

but no enclosure. The system is driven by a three-channel SGI Onyx 3000 (Twenty-four 500 MHz CPUs).

4.3. Training scenario

The training scenario involved entering a worksite operating under Track Work Authority (TWA), encountering detonators and an outer hand signaller at the bottom of the flyover. Drivers then had to stop at a station and remember prior warnings when departing the station before encountering further detonators, an inner hand signaller and finally the worksite. A second incident involved a Condition A!ecting the Network (CAN). Specifically, the CAN was a failed level crossing attended by hand signallers. After receiving advice of the failed crossing the driver next encounters school children trespassing on the track, followed by a station and finally a tunnel. These interleaved events were designed to interfere with the driver’s concentration potentially resulting in the driver forgetting to prepare for the failed crossing. Finally, drivers were required to pass a failed absolute signal at stop immediately before a station located in a tunnel. Each of these events were further complicated through increasing the driver’s workload pressures by rapidly changing operational conditions such as variations to track infrastructure including curvature, grade, speed and signal siting and sequence. The rate at which these conditions were changed often resulted in a driver feeling under intense time pressure. Incidents also included the receipt of information into the driving cab via multiple sources, both visual and auditory.

4.4. Procedures

To determine whether stress training via simulation was improving performance the decision making skills of 63 drivers undertaking their annual simulator training were recorded in 2006. Rail incidents encoun- tered during the simulator training exposure were analysed to compile a checklist of correct actions the driver must perform simultaneously for each key event in addition to a checklist of possible errors. From the 4 major incidents encountered in the training scenario outlined above, 36 correct actions were required to maintain safe and e!ective train travel on the network and 45 possible errors were identified. All drivers attended a classroom refresher course on rules and regulations and undertook a 5 min practice run on the simulator before their simulation training. Trainers completed the checklist of correct actions and possible errors while drivers undertook the 40 min simulator training run.

Behaviour & Information Technology 461

In 2007, 34 of the drivers who had participated in 2006 returned for training to undertake the same simulated driving scene using the same simulator type they had been tested on the previous year. This is referred to as Trial 1. To test the premise that repeated exposure will inoculate drivers and result in enhanced decision-making skills, the drivers were also required to undertake the driving scene after a short half hour time delay. This is referred to as Trial 2. Comparing across exposures, Trial 1 against Trial 2, decision making performance could then be assessed after a short time delay (on the same day) and after a long time delay (12 month interval). All the same perfor- mance data were gathered during the first and second sessions as was collected in 2006 and drivers used the same simulator type which they had used in the first instance.

In 2007 in between the two simulator exposures the drivers completed two surveys the Presence Question- naire (Witmer et al. 2005) and the IGroup Presence Questionnaire (Schubert et al. 2001). The surveys were selected to measure the concept of presence in two ways, through introspective self-report and causal factor self-report measures to gain data on the e!ectiveness of both the wide-screen and the cab-based simulators in generating immersion. The IPQ measures spatial presence by asking questions that require participants report on their own individual experience. The PQ explores four factors thought to underlie presence: involvement, sensory fidelity, adaptation and interface quality.

5. Results

Results were converted to an absolute percentage error score allowing scores across simulator types to be compared as seen in Table 1.

5.1. Repeated practice

The comparison between the 2006 and 2007 Trial 1 results indicated that total driver error rates were higher in 2007 (11.7%) than 2006 (5.2%). However, in 2007, the opportunity for repeated practice resulted in

improved error rates. A 2 (simulator) 6 4 (decision point) 6 2 (trial) mixed ANOVA was performed on the 2007 survey data and revealed two significant main e!ects, one for trial (F ! 13.84; df ! 1, 31, p ! 0.001) and the other for decision points (F ! 17.29; df ! 1.31, p ! 0.000). The main e!ect of trial revealed that the error rate was significantly greater during the driver’s first exposure (Trial 1) than during their second exposure (Trial 2) (11.7% vs. 2.6%, respec- tively). Therefore, when given the opportunity for a second exposure in 2007 driver’s error rates decreased significantly.

5.2. Error rates by simulator type in 2006

The total error rates by simulator were then compared to determine whether the choice of simulator altered the e!ectiveness of the training.

In 2006, 12 participants were trained on the wide- screen while 51 were trained on the smaller cab-based simulator. A 2 (simulator) 6 4 (decision points) mixed ANOVA was performed and revealed to be significant (F ! 55.39, df ! 1. 61, p ! 0.018) for the main e!ect of simulator. This result indicates that participants had greater error rates when using the 160 degree curved wide-screen (9%) than the flat cab-based screen (4%) over the four major decision points investigated. This result was interesting in that the e!ect of simulator held over all sections of the virtual track. At each point in the scene when decisions had to be made to avert accidents or other incidences drivers using the wide- screen consistently made more errors than their colleagues using the cab-based screen as seen in Table 2. The e!ect of simulator was not influenced by the type of problem being negotiated on the virtual track. No significant e!ect was found for the main e!ect of decision points.

5.3. Individual error types

In terms of specific errors, in 2006, the highest error rates were all recorded in the wide-screen simulator. The highest number of errors was recorded (58%) when drivers were required to report the Children on

Table 1. Error rates for curved and flat screen simulators.

2006 2007 (Trial 1) 2007 (Trial 2)

CS (n ! 12) FS (n ! 51) CS (n ! 4) FS (n ! 30) CS (n ! 4) FS (n ! 30)

Average error rate 3.4 1.5 4.75 4.2 2.25 1.4 Error rate as % of total number of actions 9.5 4.2 13.2 11.6 6.25 2.0

Total % (simulator totals combined) 5.2 11.7 2.6

CS, curved screen; FS, flat screen.

462 J.G. Tichon and G.M. Wallis

Track incident to the guard after the signaller had been advised via radio. The second highest error rate (50%) was recorded again in the wide-screen simulator during the absolute signal at stop incident, during which the driver is required to either advise the signaller to standby or wait to answer the radio after the train is stopped. The third highest error rate, again recorded using the wide-screen, occurred in the Failed Level Crossing incident (42%).

5.4. Error rates by simulator type in 2007

Because of a simulator malfunction a!ecting some drivers, in 2007 it was not possible to test the significance of di!erences between the two simulators in that year. Nonetheless, the general trend was consistent with the 2006 results, i.e. that the cab-based screen was producing better training outcomes than the wide-screen. In addition to making fewer errors on both Trial 1 and Trial 2, the drivers using the cab- based screen also made larger improvements between the two trials as seen in Table 1.

However, the 2 (simulator) 6 4 (decision point) 6 2 (trial) mixed ANOVA performed did reveal a significant main e!ect for decision points (F ! 17.29, df ! 1.31, p ! 0.000). The main e!ect of decision points indicates that almost all of the error rates were all significantly di!erent from each other which means that the error rate for the Track Authority Work condition (24%) was significantly higher than the error

rate for the Children on Track decision point (21%) which was significantly higher from the Failed Level Crossing decision point (10%) but Failed Level Crossing decision point error rate was not significantly di!erent from the error rate in the Absolute Signal at Stop decision point (8%).

5.5. Presence

Despite the di!erence in skill levels achieved using the cab-based screen versus the wide-screen; the drivers rated the two simulators as equal on the presence/ immersion measures. Both simulators scored 69% on the PQ and 61% on the IPQ as seen in Table 3. Additionally, both simulators were rated equally by drivers across all of the four factors underlying the PQ. There was a greater range between the highest and lowest scores for both questionnaires received for the cab-based simulator. However, the greatest deviance of scores for individual factors occurred with involvement and adaptation for the cab-based which was also the case for the wide-screen.

6. Discussion

In terms of the overall training program, the findings revealed performance did improve significantly when drivers were given the opportunity to repeat their simulator experience on the same day. This suggests that providing a repeat training session in the simulator may have had a cumulative e!ect on training e!ectiveness. Practice should result in greater con- fidence and improved performance. However, the results also indicate that, although performance improved across time within a day, retention over longer periods was not so good. Despite a year’s additional driving experience in the interim, drivers returning in 2007 performed significantly worse than during their training in 2006. Clearly some of the skills tested in the simulator involve circumstances which the drivers would rarely if ever experience in the real world, and this lack of repeated exposure may explain

Table 2. 2006 decision point error rates by simulator.

Decision points Curved screen (n ! 12) (%)

Flat screen (n ! 51) (%)

Track work authority 9 7 Failed level crossing 8 4 Children on track 11 5 Absolute signal at stop 10 3

Total *9 *4

*significant di!erence at p 5 0.05.

Table 3. Presence scores for curved and flat screen simulators.

Simulator Flat screen

Mean SD Curved screen

Mean SD Highest possible

score Flat screen

(%) Curved screen

(%)

PQ (Total) 121 18.5 121 8.5 175 69.1 69.1 PQ factors: Involve 51.7 9.28 51.7 4.04 77 67.1 67.1 Sensory 18.3 4.69 18.3 1.53 28 65.35 65.35 Adaptation 35.7 5.77 36.3 5.03 49 72.8 74 Interface 15 2.21 14.3 1.53 21 71.4 68 IPQ 51.4 9.02 52 4.58 84 61.2 61.9

Behaviour & Information Technology 463

why some of the skills degraded over time but does not explain why they degraded to a point below the 2006 level.

Drivers were given a standard 5 min warm-up before each simulator exposure so it is unlikely that the di!erence in skill levels attained were a result of di!ering opportunities to access shorter or longer warm-up times. However, there may be a number of explanations for this apparently counter-intuitive result. It may be that classroom revision time under- taken prior to the simulation test di!ered dramatically from one year to the next. This time was not examined as part of this study. If revision was more targeted one year to the exact problems the drivers would encounter during their simulation session this would have provided much better preparation time. Alternatively, perhaps the fact that drivers were able to self-select the simulator they felt most comfortable with and the driving scene they undertook was unchanged from the previous year, this resulted in over-confidence which translated into risk-taking in the simulator and consequently more errors and mistakes. In their second attempt in 2007, the drivers demonstrated they did have high skill levels. This begs the question as to whether the improvement was a product of the practice in the first trial or whether in fact the drivers were more focused as a result of having made far too many errors on their first attempt.

The role of spaced repetitions in learning theory may also provide a future research direction to explore in an attempt to explain why skills decayed. There is very little research on how procedural learning takes place in simulated environments. However, there has been a great deal of research on how di!erent spacing of repetitions in time a!ects the strength of memory and how the resulting findings could be applied in the practice of e!ective learning (Wozniak 1995). Memory can be strengthened through appropriate spacing time intervals; however, programs must be careful the spacing of training repetitions does not leave so much time between sessions that the learned memory trace becomes completely inaccessible. This theory would suggest it is highly probable that altering the training program in a simulator may assist to address the skill decay problem. In future research, it would be interesting to investigate how repetition after varying time intervals either improve or degrade retention specifically when simulators are in use. In large companies when relocating sta! to central simulators is expensive it is particularly important to determine the longest inter-repetition interval that avoids retrieval failures.

When comparing learning e!ects across simulator type, the cab-based simulator produced higher decision making performance scores than the larger wide-screen

simulator. This was surprising as it is generally accepted that higher-end immersive interfaces are more successful in rendering scenes similar to real- world settings. It has been high-end immersive facilities that have demonstrated that simulation works. How- ever, researchers who have been working to increase understanding of immersion and identify the compo- nents that produce them acknowledge there is still far to go (Bowman and McMahan 2007). So what might the reasons be for this apparently counterintuitive discrepancy? Is it simply the case that occasionally less is more? Possibly, a smaller screen and smaller simulator might help aid concentration and reduce distraction. On the other hand, the cab-based simula- tors are in one sense closer to a real train. Their narrow field of view provides a visible frame of reference, something lacking in the large curved screen simulator. It may be that this frame of reference contributes to the realism and fidelity of the simulation, albeit covertly (as these di!erences did not emerge in the presence questionnaires).

Equally so it could also be that the perceptual field scanning for the wide-screen simulator is larger than is ever done in a real train, and drivers have neither the time, under the pressures of training for degraded conditions, nor the experience to scan e!ectively a ‘larger than life’ presentation. The rail training provider had installed wide-screen simulators to enable teams of people to be trained in team communication scenarios. However, the aim of multi-use for the simulators may have resulted in a detrimental impact for the solo driver training. A high perceptual immersion achieved via the full visual field may have been achieved at the cost of validity where the actual visual experience does not represent the real-world setting.

Both types of simulators used in the study achieved relatively low presence scores indicating drivers did not experience the virtual world as highly engaging. One of the main issues that drivers had was that they ‘. . .had no sense of train momentum and no feel of going up or down grades’. Drivers reported being able to adapt quickly to the controls and interface but the discre- pancy between how natural the simulator felt in comparison to a real train may have been significant enough to reduce the immersive impact of the experience for some drivers.

Beyond specific issues of simulator design, the results of this study have highlighted a potential problem with the use of presence questionnaires in predicting training outcomes. Despite the significant di!erence in number of errors made when training was undertaken in the cab-based simulator versus the wide-screen simulators, the drivers rated the two simulators as equal on the presence/immersion mea- sures. The presence analysis indicates that in terms of

464 J.G. Tichon and G.M. Wallis

subjective feedback neither the cab-based nor the wide- screen simulators were experienced as significantly more or less immersive or engaging. The main di!erence between them was that the scores for the cab-based simulator were more variable and this system was rated slightly more highly in terms of the interface which it o!ers.

In terms of having a stress inoculation e!ect, in the absence of high levels of presence, it is unclear if skill improvements between training sessions on the same day were a product of increased confidence in coping with high stress events or whether it was merely the result of an increasing level of comfort in driving through that particular scenario. Retesting drivers using a new track route along which the same problematic events (decision points) are embedded but in a di!erent sequence would reveal whether drivers do maintain skill levels when under stress but without the benefit of familiarity.

7. Future directions

In the future it would be interesting to see to what extent the visual reference frame is responsible for the discrepancy in performance between the two simula- tors. It would also be helpful to test alternative measures of immersion and presence to see if they correlate more accurately with the training outcomes described here. The low presence levels did not predict the good training outcomes achieved on the simulators. In addition to enhancing performance, presence is also purported to be a strong indicator of whether training will ultimately transfer to the real world. If the higher performance scores are being influenced by levels of familiarity with the scene it could be that the lower presence scores should be more closely studied as an indicator of training transference. To investigate the e!ectiveness of the simulator training in the real world one approach would be to compare the annual incident rates for the year before training and the year after training for those drivers who have undergone simulation train- ing. Ideally, a control group of drivers who did not receive any simulator training would also be com- pared against those who have. Similarly, incident rates of drivers who trained on the flat cab-based versus the curved wide-screen simulators could be investigated to determine if simulator type has an impact in the real world.

There is an important role for interactive simula- tors in replicating degraded events and establishing them as a core component of rail training programs. However, any simulator of any level of complexity can only be as good as the educational program in which it is embedded. Many simulators are underutilised due

to failure to integrate them into a curriculum that enhances their potential. Future research investigating the optimal repetition spacing for simulator training sessions would contribute greatly to knowledge on how long memory traces built from virtual experiences last before requiring refresher training to ensure they remain accessible to personnel relying on the training to maintain safe operations.

Acknowledgements The financial support of the Australian Research Council is gratefully acknowledged.

References

Baker, D.P., et al., 2005. Medical teamwork and patient safety: the evidence-based relation. AHRQ Publication No. 05-0053. Rockville, MD: Agency for Healthcare Research and Quality.

Bowman, D.A. and McMahan, R.P., 2007. Virtual reality: how much immersion is enough? Computer, 40 (7), 36–43.

Cannon-Bowers, J.A. and Salas, E., 1998. Individual and team decision making under stress: theoretical under- pinnings. In: J.A. Cannon-Bowers and E. Salas, eds. Making decisions under stress: implications for individual and team training. Washington, DC: American Psycho- logical Association, 17–38.

Driskell, J.E. and Johnston, J.H., 1998. Stress exposure training. In: J.A. Cannon-Bowers and E. Salas, eds. Making decisions under stress: implications for individual and team training. Washington, DC: American Psycho- logical Association, 191–217.

Ho!man, H., et al., 2001. Virtual reality monitoring: phenomenal characteristics of real, virtual and false memories. Cyberpsychology and Behavior, 4, 565–572.

Hulse, L.M. and Memon, A., 2006. Fatal impact? The e!ects of emotional arousal and weapon presence on police o"cer’s memories for a simulated crime. Legal and Criminological Psychology, 11, 313–325.

Maschuw, K., et al., 2008. The impact of self-belief on laparoscopic performance of novices and experienced surgeons. World Journal of Surgery, 32, 1911–1916.

Nanyue, A.Y., et al., 2006. Modeling of virtual tra"c environment for driving simulator. In: Proceedings of 7th international conference of computer-aided industrial design and conceptual design, 7–9 November 2006. Hangzhou, China: IEEE Press, 1–5.

Romano, D.M. and Brna, P., 2001. Presence and reflection in training: support for learning to improve quality decision-making skills under time limitations. Cyberpsy- chology and Behaviour, 4, 265–277.

Salas, E., et al., 1999. Team training in virtual environ- ments. In: K.M. Stanney, ed. Handbook of virtual environment. Mahwah, NJ: Lawrence Erlbaum Associ- ates, 873–892.

Schubert, T., Friedmann, F., and Regenbrecht, H., 2001. The experience of presence: factor analytic insights. Presence: Teleoperators and Virtual Environments, 10, 266–282.

Stetz, M., Wiederhold, B.K., and Wildzunas, R., 2006. Usefulness of virtual reality stress inoculation training for military medical females. Cyberpsychology and Behavior, 9, 719–720.

Behaviour & Information Technology 465

Thomas, M.J.W., 2003. Operational fidelity in simulation- based training: the use of data from threat and error management analysis in instructional systems design. In: Proceedings of SimTecT2003, 26–29 May 2003. Adelaide, Australia: SIAA Press, 91–95.

Tichon, J., et al., 2003. Education and training in virtual environments for disaster management. In: Proceedings of the World Conference on Educational Multimedia, Hypermedia, Honolulu, Hawaii, 1191– 1194.

Wallis, G., Tichon, J., and Mildred, T., 2007. Speed perception as an objective measure of presence in virtual environments. In: Proceedings of SimTect2007, Brisbane, Australia, 527–531.

Walshe, D., et al., 2005. Virtually driving: are the driving environments ‘real enough’ for exposure therapy with accident victims? An exploratory study. Cyberspychology and Behavior, 8 (6), 532–537.

Witmer, B.G., Jerome, C.J., and Singer, J., 2005. The factor structure of the presence questionnaire. Presence: Tele- operators and Virtual Environments, 14, 298–312.

Witmer, B.G. and Singer, M.J., 1998. Measuring presence in virtual environments: a presence questionnaire. Presence: Teleoperators and Virtual Environments, 7, 225–240.

Wozniak, P.A., 1995. Economics of learning. Doctoral Dissertation. University of Economics, Wroclaw.

Zakay, D. and Wooler, S., 1984. Time pressure, training and decision e!ectiveness. Ergonomics, 27, 273–284.

466 J.G. Tichon and G.M. Wallis

Copyright of Behaviour & Information Technology is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.