Driver Attention in Automatic Transmission Cars

Contents lists available at ScienceDirect

Applied Ergonomics

journal homepage: www.elsevier.com/locate/apergo

The effect of navigation display clutter on performance and attention allocation in presentation- and simulator-based driving experiments

Carl Pankok Jr.a,∗, David Kaberb

a Department of Information Science, College of Computing and Informatics, Drexel University, Philadelphia, PA 19104, USA b Edward P. Fitts Department of Industrial & Systems Engineering, North Carolina State University, Raleigh, NC 27695, USA

A R T I C L E I N F O

Keywords: Display clutter Driving simulator Driver performance Attention allocation

A B S T R A C T

Display clutter can have differential effects based on environmental factors, such as workload, stress, and ex- periment paradigm. The objectives of the current study were to assess the effects of display clutter on driver performance and attention allocation and compare results across two experimental paradigms. Forty-two par- ticipants searched high- and low-clutter in-car navigation displays for routine information either during a static, presentation-based experiment or in a dynamic, driving simulator experiment. Results revealed display clutter to significantly alter attention allocation and degrade performance in the presentation experiment, but had little to no effect on driver performance or attention allocation in the driving simulator experiment. Results suggest that display clutter may have its greatest effect on performance and attention allocation in domains requiring ex- tended attention to the cluttered display compared to tasks in which the cluttered display acts as a support tool for secondary tasks.

1. Introduction

Advances in technology have led to the widespread use of in- formation displays in automobiles. Driver information overload can have disastrous effects in safety-critical situations and, therefore, it is imperative that information displays be designed to ensure key in- formation is presented to drivers without exceeding available atten- tional resources. Related to this, displays should be designed to reduce clutter, or display imagery that unintendedly obscures or confuses other information or that may not be relevant to the task at hand (Kaber et al., 2008). There is myriad research on the effects of display clutter on aviation task performance and workload (e.g., Kim et al., 2011), map search tasks (e.g., Wickens et al., 2000), and supervisory tasks (e.g., St. John et al., 2005); however, there has been little research investigating the effects of display clutter on driver performance and attention allo- cation.

1.1. Effects of display clutter

Trends identified in the existing literature suggest that response time (RT) to task demands increases as a function of increasing in- formation display clutter (see review by Moacdieh and Sarter, 2015a). Boston and Braun (1996) reported RT to an unexpected event in a ship navigation task was significantly longer when participants used high

clutter displays compared to low clutter displays. Similarly, pilots have been shown to respond 0.5 s slower (a statistically significant differ- ence) to changes in display symbology and to the presence of targets in the environment when using higher-clutter displays, particularly when there was high contrast between display elements and the background (i.e., making any clutter more salient; Ververs and Wickens, 1996). Over a series of five experiments involving map search as part of military and aviation tasks, a statistically significant 4 s RT benefit was reported with low clutter maps compared to high clutter maps (Wickens et al., 2000). Further, in their military task, the authors found a highly significant linear relationship between the level of clutter and RT.

Several studies have also been conducted on visual target detection times. In a naval ship monitoring task, St. John et al. (2005) reported a statistically significant 25% increase in the time to identify high-risk intruders in high clutter displays compared to low clutter displays. (Beck et al., 2010) also reported a statistically significant increase in time to detect a target elevation marker in aeronautical charts with higher clutter. (Moacdieh et al., 2013) reported that increasing the level of clutter in a primary flight display significantly degraded detection time of visual alerts and messages that appeared on the display during high-workload phases of flight, but there was no statistically significant effect of clutter during low-workload periods. Findings suggested that the influence of clutter on performance may be sensitive to external or environmental effects, such as workload. Other research in the aviation

https://doi.org/10.1016/j.apergo.2018.01.008 Received 3 October 2017; Received in revised form 15 January 2018; Accepted 21 January 2018

∗ Corresponding author. E-mail address: [email protected] (C. Pankok).

Applied Ergonomics 69 (2018) 136–145

Available online 03 February 2018 0003-6870/ © 2018 Elsevier Ltd. All rights reserved.

T

domain indicates that using cluttered displays in an aircraft cockpit can lead to compromises in flight path control (e.g., Wickens and Long, 1994; Kim et al., 2011). These findings suggest that high levels of dis- play clutter can be detrimental to task performance beyond simple RT effects and that the manner of information presentation is influential.

Recent research has focused on the effects of cluttered displays on attention allocation in various contexts. Moacdieh and Sarter (2012) presented a comprehensive list of eye tracking metrics that were sig- nificantly altered by varying levels of clutter in a Where's Waldo? search task. Interestingly, Moacdieh and Sarter (2012) reported a statistically insignificant effect of clutter level in a display on glance duration to target areas of interest (AOIs), which is counter to Lim et al. (2010), who found a significant effect of the level of clutter on the mean glance time in their driving experiment. In another study, Moacdieh et al. (2013) reported statistically significant differences in primary flight display clutter to affect various eye tracking metrics, including an in- crease in the number of fixations, fixation frequency, and fixation time on a target AOI. Similarly, in a mock warfare task, van Orden et al. (2001) found fixation frequency to be among the most predictive variables relating eye tracking to clutter. Finally, Moacdieh and Sarter (2015b) found significant effects of clutter on number of fixations, but reported no significant effect of clutter on mean fixation duration in an electronic medical record search task. On this basis, additional research is needed to clarify the relationship of display clutter to gaze behavior in target detection tasks.

1.2. Environmental factors and display clutter

Previous research suggests that the task domain or other environ- mental factors can influence the effects of display clutter and bias op- erator perceptions of clutter, particularly for simplistic measures of clutter (i.e., unidimensional overall clutter ratings). For example, in their experiment on the effects of clutter on physician use of an elec- tronic medical records system, Moacdieh and Sarter (2015b) reported significant interactions between display clutter level and stress level (manipulated by imposing differing time limits on task completion) on search performance and attention allocation, suggesting that the effects of clutter were dependent on the environmental factor of stress.

In an earlier experiment in the aviation domain, Moacdieh et al. (2013) reported no significant difference in unidimensional clutter ratings between “medium” and “high” clutter displays, suggesting that a simple clutter rating (CR) response was not sensitive enough to dif- ferentiate between the two levels of clutter. The authors also com- mented that participants preferred the “medium” clutter displays to the “low” and “high” clutter displays due to the presentation of the ideal amount of information. Similar findings were reported for pilot per- formance in a runway approach by Kim et al. (2011) and Kaber et al. (2013a). These results suggest a lack of operator understanding of what information constitutes clutter compared to task-relevant information, leading to flawed perceptions of display clutter.

Finally, Kaber et al. (2013b) analyzed data over three experiments to assess how the environmental factors of display dynamics (static presentation vs. dynamic simulator) and flight domain (fixed-wing vs. vertical takeoff and landing) affected pilot perceptions of display clutter. The results revealed that CRs were smaller when the displays were used in conjunction with a flight simulation compared to when pilots were given a verbal description of a flight situation and then shown a static image of the same display configurations. Simple CRs were also reported to be higher in the more-complicated vertical takeoff and landing simulation than in the simpler fixed-wing simulation. The experiment also assessed differences in a structured Clutter Score (CS) developed by Kaber et al. (2008), which measured perceptions of dis- play clutter as a rank-weighted sum of “clutter subdimensions,” or di- mensions that contribute to perceptions of clutter. Kaber et al. (2013b) also reported significant CS differences between the two simulation types, with higher CSs being reported for the more-difficult vertical

takeoff and landing simulation than for the simpler fixed-wing simu- lation. Overall, these results suggest that performance and perceptions of clutter can be biased by various task and environmental factors.

1.3. Motivation

Most of the existing research on the effects of display clutter has been performed in the aviation and military domains, with relatively little research performed on the effects of display clutter on driver performance and attention allocation. There also appears to be some conflicting evidence on attentional effects of display clutter. Furthermore, the existing literature suggests that the effects of clutter are dependent on various factors, such as workload, stress, and task setup. Given these shortcomings in the literature, a dual-experiment paradigm was designed to assess the effects of display clutter on driver attention allocation and performance, including: (1) a static display image presentation experiment and (2) a dynamic driving simulator experiment. The task in both experiments involved asking drivers routine navigation questions, requiring them to search high- and low- clutter combinations of in-car navigation displays.

1.4. Hypotheses

Based on the literature, it was expected that high clutter displays would degrade task performance (Hypothesis (H)1), high clutter dis- plays would require increased driver attention (H2), and high clutter displays would yield greater perceptions of clutter (H3) than low clutter displays. It was also expected that there would be differential effects of clutter based on the nature of presentation of the experimental stimuli (i.e., static presentation vs. dynamic simulation; H4).

2. Methods

2.1. Participants

Twenty-two participants were recruited for the presentation-based experiment, with balanced representation of male and female drivers. In order to recruit a uniform sample of participants in terms of driving skills, all participants were required to be younger than 60 years of age, as Chen et al. (2007) demonstrated that accident rates are uniform for drivers between the ages of 25 and 60. All drivers had 20/20 vision or corrective lenses. The sample as a whole had 9.7 ± 6.8 (mean ± sd) years of driving experience.

An additional 20 participants, also under 60 years of age, were re- cruited for the driving simulator experiment, with balanced re- presentation of male and female drivers. All participants had 20/20 vision or corrective lenses, and had 10.05 ± 8.09 years of driving experience. There were no drivers that participated in both experi- ments. A t-test assuming unequal variances among the experiment samples revealed no evidence of a difference in experience between the presentation-based participants and the simulation participants (t (37.3) = 0.139, p = 0.890). Furthermore, analyses of data from both the presentation- and simulator-based experiments revealed no sig- nificant effect of expertise on any of the responses that were measured in the two experiments; consequently, the samples were considered uniform in terms of driver capabilities.

2.2. Apparatus

Both experiments used the driving simulator setup shown in Fig. 1(a), which included three 38-inch high definition television monitors providing a 135-degree field of view of the driving environ- ment. To the right of the forward-view was a tablet computer (iPad 2, Apple, Inc.), which presented the navigation display corresponding to the driving scenario. In the presentation experiment, static images of various driving scenarios were presented to participants sitting in the

C. Pankok, D. Kaber Applied Ergonomics 69 (2018) 136–145

137

driver seat of the simulator via the center television only. Queries in the presentation experiment were overlaid on the virtual vehicle window view. The driving simulation experiment presented graphical imagery of a driving environment on all three televisions. The simulation was developed with the STISIM Drive software package (Systems Tech- nology, Inc.). Similar systems have been used in prior studies with performance results validated against real-life driving behavior (e.g., Wang et al., 2010).

Driver visual attention allocation was captured using a FaceLab 5 eye tracking system (Seeing Machines, Inc.). The system hardware, shown in Fig. 1(b), consists of two cameras and an infrared light- emitting pod. The equipment is non-intrusive in driver visual attention as compared to a head-mounted eye-tracking system. The cameras capture the reflection of the infrared light on the eyes along with out- lines of the pupils. These two points of reflection on the surface of the eye (the glint and edge of the pupil) allow for determination of gaze direction and projection onto a model of the driving simulator setup within the FaceLab software environment. The system records eye movements at 60 Hz with an accuracy of 0.5°–1° of rotational error.

2.3. Independent variables

Two independent variables were manipulated in the presentation experiment, including: (1) the level of display clutter; and (2) the na- vigation query category. Clutter level was represented by low- and high- clutter display configurations, as presented in Fig. 2. Query category was represented by subsets of queries developed to target different areas of the navigation display. Eight queries were formulated and assigned a “navigation category”. Some of the queries were formulated based on navigation queries previously used by Dingus et al. (1989) and

Lansdown (2000). Others were developed with the intent of requiring drivers to search different areas of the navigation display in order to find task-relevant information. The queries were developed so that the features included in the high clutter displays but not in the low clutter displays (e.g., alternate route traffic level) were irrelevant to the query, and thus considered clutter according to the criteria identified by Kaber et al. (2008). The same eight queries, presented in Table 1, were used in both experiments (presentation and driving simulation).

Clutter level was the only independent variable manipulated in the driving simulator experiment. The displays presented in Fig. 2 were used as the low- and high-clutter configurations. Edge Density, a display feature-based measure of clutter, which involves calculating the pro- portion of “edge points” in an image (Rotman et al., 1994), was larger for the high clutter displays (mean 10.6% edge pixels) than for the low clutter displays (mean 3.4% edge pixels), suggesting that the display configurations represented differing levels of display clutter.

2.4. Tasks

In the presentation experiment, participants were first shown a static image of a driving situation (e.g., approaching an intersection). Participants were permitted 5 s to examine the driving situation image, after which they were presented with the navigation query, which was overlaid on the virtual vehicle window view. When the participant in- dicated that (s)he was finished reading the query and ready to respond, a navigation display screenshot appeared on the tablet computer to the driver's right. Participants were told to respond verbally to the navi- gation query as quickly and as accurately as possible. After the re- sponse, the participant was asked to subjectively rate the level of per- ceived clutter on the navigation display.

Fig. 1. (a) Driving simulator setup and (b) eye tracking system used in both experiments.

Fig. 2. (a) Low clutter and (b) high clutter display configurations used in both experiments.

C. Pankok, D. Kaber Applied Ergonomics 69 (2018) 136–145

138

In the driving simulator experiment, participants drove one of two routes in a simulated environment on a clear day with no adverse weather conditions. All roadways presented six lanes in total with three lanes in each direction of travel and stoplights at regular intervals (of 2000 ft; see Fig. 3). The speed limit on all roadways was 35 mph and traffic density on the road was comparable to Level of Service C (stable flow, ability to maneuver through lanes is restricted, lane changes re- quire driver awareness, and minimum vehicle spacing is about 220 ft; Transportation Research Board, 2010). At four pre-determined points along the drive (corresponding to one specific time for each query in a scenario), the driver was posed with a navigation query (via an auto- mated voice through the simulator sound system) and asked to verbally provide navigation information that could only be obtained from the navigation display. Immediately following the response to a query, the simulation was paused and the participant subjectively rated the level of perceived display clutter. Subsequently, the participant continued driving until either the next query was posed or until (s)he arrived at the route's destination. Participants were not instructed to prioritize one task over the other; they were told to treat the scenario as if they were driving a real-world route to the destination programmed into the na- vigation system. The instruction was intended to facilitate a realistic assessment of navigation display clutter.

2.5. Experiment design

In the presentation experiment, a full crossing of the levels of dis- play clutter (low and high) and eight navigation queries was presented to participants and replicated once, resulting in 32 total experiment

trials per participant. The order of presentation of both sets of trials (2 sets of 16 trials) was randomized in order to account for any potential order effects. (Trial order was included as a predictor in all initial sta- tistical models to assess the effectiveness of the randomization proce- dure. See the Data Analyses section for additional information.)

The driving simulator experiment utilized a combined Latin square design (Giesbrecht and Gumpertz, 2004; pg. 126) with the level of display clutter assigned to each combination of participant and driving scenario. Testing of each level of clutter was replicated once for each participant, resulting in four total driving scenarios (2 levels of clutter * 2 replications). Each driving scenario was defined by: (1) one of two routes; and (2) a set of four queries. (During and experiment session, all participants were exposed to each of the eight total queries on two occasions.) Each scenario contained different street names in order to prevent participant learning of routes and driving to a destination without using the navigation display. The order of presentation of clutter conditions across scenarios was randomized for each participant in order to minimize any learning or fatigue effects.

2.6. Dependent variables

Three categories of dependent variables were collected for both experiments, including performance, attention allocation, and sub- jective perceptions of clutter. In the presentation experiment, perfor- mance measures included query RT and accuracy, based on post-ex- periment video analysis. For the simulation experiment, performance measures included query response accuracy, lane deviations, and speed deviations. Lane deviation was determined as the absolute deviation of the vehicle center from the center of the lane, and speed deviation was determined as the average absolute deviation from the posted speed limit. We were unable to calculate RT in the driving conditions because the navigation queries were posed aurally to the drivers while they were engaged in the driving task. The durations of the queries differed, preventing us from establishing a starting point in order to calculate RT. For this reason, we compared only query response accuracy, lane de- viation, and speed deviation in the driving simulator experiment to the performance metrics from the presentation experiment.

Regarding attention allocation, the number of fixations and longest glance to the navigation display were measured in the presentation experiment. Similarly, fixation frequency and longest glance to the navigation display were calculated for the driving simulator experi- ment. A fixation was any focus of visual attention with a gaze speed < 100 deg/s for a minimum duration of 100 ms (Holmqvist et al., 2011) while a glance was determined as the total time the focus of attention remained on the navigation display, including fixations and saccades. A count of the number of fixations to the navigation display was calculated for the presentation experiment. Fixation frequency in the simulator experiment was determined as the number of fixations on the navigation display divided by the total number of fixations to all areas of interest (AOIs; i.e., the screens in the simulator) in the driving scene. Since presentation experiment participants were not required to simultaneously scan the out-window-view while responding to queries, drivers were not expected to allocate attentional resources to the roadway; therefore, fixation count was a more accurate measure of attention allocation than fixation frequency. In the driving simulator experiment, however, participants were required to balance the de- mands of driving and navigation, making fixation frequency a more accurate indicator of attention allocation.

Finally, the subjective reduced clutter score (RCS) developed by Kaber et al. (2013a) and an overall CR were collected after each navi- gation query to measure driver perceptions of clutter in-the-moment. Kaber et al. (2013a) performed a factor analysis revealing three con- tributors to display clutter, including Consistency/Similarity, Dy- namics/Variability, and Colorfulness. The RCS is a rank-weighted sum of these three terms. In both experiments, attention allocation metrics and performance measures were measured and summarized for the

Table 1 Navigation queries used in both experiments.

Category Query

Next Link How far, in miles, are you from your next turn? What is the name of the road onto which you will be turning next?

Lane Positioning Which lane should you be in at the upcoming intersection in order to reach your destination? After you turn onto the next street, which lane should you be in to be ready for the following turn?

Congestion on Current Route

How heavy is the congestion on the road on which you're currently traveling? How heavy is the congestion on the road onto which you're turning?

Destination Information How far, in miles, are you from your destination? What is the expected time of arrival at your destination?

Fig. 3. Screenshot of the driving simulator environment.

C. Pankok, D. Kaber Applied Ergonomics 69 (2018) 136–145

139

same time window, which started when the navigation query was posed to the driver and ended when the driver answered the query.

2.7. Procedure

Both experiments followed a similar procedure. Participants were initially asked to complete an informed consent form and demographic questionnaire, after which their useful field of view (UFOV) was as- sessed using the Visual Awareness software package (Visual Awareness, Inc.). The software assesses overall visual target processing speed, speed of selective attention in switching focus from one target to an- other, and speed of processing when attention is divided among mul- tiple targets. If participants failed to meet minimum UFOV criteria for the experiment (processing speed < 30 msec, divided attention < 350 msec, and selective attention < 500 msec), they were disqualified from participation.

Participants were subsequently introduced to the driving simulator, including the vehicle cab, television screens displaying the forward field of view, and the eye tracking system. They were given an in- troduction to the RCS subdimensions and were required to complete a training trial without queries followed by another presenting five na- vigation queries. If a participant responded incorrectly to more than one assessment trial, (s)he was disqualified from participation. The eye tracking system was configured for each participant after the training trials.

Before experiment trials, participants were asked to complete pair- wise rankings of the importance of RCS subdimensions to perceived clutter. Participants had access to the subdimension definitions throughout the entire experiment. In the presentation experiment, participants were exposed to two blocks of 16 queries. In the driving simulator experiment, participants drove four trials, each containing four queries. Between trials, participants were asked to exit the vehicle cab for 5 min. In the driving simulator experiment, participants were administered a simulator sickness questionnaire (Kennedy et al., 1993) after each trial to monitor for motion sickness symptoms. After com- pletion of all trials, participants were debriefed on the experiment and escorted out of the lab. The presentation experiment lasted approxi- mately 1.25 h, and the driving simulator experiment lasted approxi- mately 2 h for each participant. Participants in both experiments were compensated 20 USD/hour for their time.

2.8. Data Analyses

Analysis of Variance (ANOVA) was used to assess the effects of the independent variables on response measures. Query response accuracy across all trials was 98.6% (low clutter trials: 97.6%, high clutter trials: 99.3%) in the presentation experiment and 99.1% (low clutter trials: 100.0%, high clutter trials: 98.1%) in the driving simulator experiment, so analyses were conducted only on trials for which participants re- sponded correctly (i.e., sufficient attention allocation leading to accu- rate navigation awareness). For the presentation experiment, statistical models included terms for Participant, Trial Number (if significant), Clutter Level, Query Category, and the interaction of Clutter Level and Query Category. No transformations produced adherence to the homo- scedasticity and residual normality assumptions for any of the perfor- mance or attention allocation responses; however, use of a response ranking procedure yielded results similar to results of ANOVAs on raw responses. Therefore, analyses performed on the untransformed re- sponses were considered valid, according to (Montgomery, 1991, pg. 128), and are presented below. A Z-Score transformation was used for the CRs and RCSs to account for between-subject variability (i.e., dif- ference in internal scaling) in ratings.

A similar analysis approach was taken for the driving simulator experiment. ANOVA models included terms for Participant, Trial Number (if significant), Scenario, Query, and Clutter Level. Z-Score transforma- tions were applied to the CR and RCS responses, and to meet the

parametric test assumptions of homoscedasticity and residual nor- mality, a rank transform was applied to speed deviation, and a square- root transform was applied to lane deviation and the attention alloca- tion measures. Unless otherwise noted, a significance level of α = 0.05 was used to identify significance. Partial η2 statistics were reported to convey effect sizes.

3. Results

All plots in this section include error bars representing one standard deviation from the mean. Numbers overlaid at the bottom of bars are untransformed response means, regardless of data transformation used in the ANOVA.

3.1. Task performance

An ANOVA on RT in the presentation experiment revealed a sig- nificant main effect of clutter (F(1,584) = 10.746, p = 0.001*, partial η2 = 0.005). Fig. 4 reveals participants responded 0.16 s faster to queries when using low-clutter displays as compared to high clutter displays.

An ANOVA on the square root transform of lane deviation in the simulator experiment revealed no significant effect of display clutter level (F(1,248) = 0.135, p = 0.714, partial η2 = 0.001). Similarly, a nonparametric ANOVA on the ranked speed deviation revealed no significant effect of clutter (F(1,290) = 2.384, p = 0.124, partial η2 = 0.008). Fig. 5(a) shows the mean lane deviation of 0.86 ft for both the high and low clutter conditions. Only response variability differed slightly between the two conditions. Fig. 5(b) shows that the mean speed deviation from the limit was 1.51 mph for high clutter displays and 1.27 mph for low clutter displays with substantial variability under both conditions.

3.2. Attention allocation

In the presentation experiment, an ANOVA on the number of fixa- tions revealed a significant effect of display clutter (F(1,427) = 8.681, p = 0.003*, partial η2 = 0.008). Fig. 6(a) shows that the high clutter display required, on average, 1.61 fixations, while low clutter display required only 1.39 fixations to find the information required to respond

Fig. 4. The effect of clutter on response time in the presentation experiment.

C. Pankok, D. Kaber Applied Ergonomics 69 (2018) 136–145

140

to the navigation query. Opposite to this, an ANOVA on the square root transform of fixation frequency in the simulator experiment revealed no significant effect of display clutter (F(1,207) = 0.728, p = 0.394, par- tial η2 = 0.004). Fig. 6(b) shows that there was very little change in mean fixation frequency, with low clutter displays requiring 27% of fixations and high clutter requiring 28% of fixations.

An ANOVA on the longest glance duration in the presentation ex- periment revealed no main effect of clutter (F(1,428) = 1.049, p = 0.306, partial η2 = 0.001). Results in the simulation experiment were similar, as an ANOVA on the square root transform of the longest driver glance to the navigation display revealed no effect of clutter (F (1,208) = 0.001, p = 0.978, partial η2 < 0.001). Fig. 7 shows only a very small difference in the mean longest glance to high and low clutter displays in each experiment. However, in absolute terms, the mean

longest glance (and associated variability) was substantially longer in the presentation experiment than in the simulator experiment.

3.3. Perceptions of display clutter

An ANOVA on the presentation experiment CR Z-score revealed a significant main effect of level of display clutter (F(1,604) = 1714.406, p < 0.001*, partial η2 = 0.369), and an ANOVA on the simulator experiment CR Z-score also revealed an effect of clutter (F (1,267) = 5.491, p = 0.020*, partial η2 = 0.020). Fig. 8 shows the mean CR was higher for the high clutter displays vs. low clutter in both experiments. The magnitudes of the CRs, however, were different in the experiments, with the presentation study generally yielding larger CRs than the simulation experiment.

Fig. 5. The effect of clutter on (a) lane deviation and (b) speed deviation in the simulator experiment.

Fig. 6. The effect of clutter on (a) number of fixations in the presentation experiment and (b) fixation frequency in the simulator experiment.

C. Pankok, D. Kaber Applied Ergonomics 69 (2018) 136–145

141

ANOVAs on the RCS Z-scores revealed a significant effect of clutter in both the presentation experiment (F(1,605) = 876.637, p < 0.001*, partial η2 = 0.189) and the simulation experiment (F (1,267) = 185.542, p < 0.001*, partial η2 = 0.410). Different from the CR results, however, the magnitudes of the RCSs between experi- ments were similar, as shown in Fig. 9.

4. Discussion

Hypothesis (H)1 posited that high clutter displays would degrade driver task performance compared to low clutter displays. Results supported H1 for RT in the presentation experiment, but not for lane or speed deviations in the simulator experiment. Concerning the pre- sentation experiment, many studies have also reported longer RT with

higher clutter (see review by Moacdieh and Sarter, 2015a). Since par- ticipants were required to complete a series of training sessions and performance assessments to ensure they could correctly identify navi- gation information on displays, they were likely of a similar skill level in responding to queries, leading to relatively small RT differences between high and low displays. In the simulator experiment, the lack of driver vehicle control differences among clutter conditions was likely due to similar longest glance durations to the two display types. Prior research has associated 1800–2000 ms of continuous off-road gaze with significant degradations in vehicle control (e.g., Wierwille, 1993; Kun et al., 2009). In the present simulator experiment, the mean longest glance to the navigation display was 714 ms and 719 ms for low- and high-clutter displays, respectively. Consequently, with mean longest glance durations being less than half of the established vehicle control

Fig. 7. The effect of clutter on mean longest glance duration to the navigation in (a) the presentation experiment and (b) the simulator experiment.

Fig. 8. The effect of clutter on CRs in (a) the presentation experiment and (b) the simulator experiment.

C. Pankok, D. Kaber Applied Ergonomics 69 (2018) 136–145

142

uncertainty threshold, participants were able to produce a high degree of lane maintenance and speed control across both display configura- tions. In fact, only 3.4% of glances in the simulator experiment ex- ceeded 1800 ms; that is, there were very few glances to the navigation display that could be expected to lead to vehicle control uncertainty. Comparing these results with previous research, Wickens and Long (1994) reported that high clutter displays degraded lateral and vertical control for pilots on a simulated runway approach, but had no effect on airspeed error. Opposite to the results in our simulator experiment, Kim et al. (2011) and Kaber et al. (2013a) reported that commercial pilot flying performance generally increased with increasing “clutter”. However, it is important to note that, in these studies, the authors commented that additional imagery appearing in “moderate” vs. “low” clutter displays was likely information that pilots used to perform tasks; therefore, referring to the additional data as “clutter” was possibly a mislabeling (based on the definition of clutter).

H2 posited that high clutter displays were expected to require greater driver attention than low clutter displays. This hypothesis was partially supported by the clutter effect on fixation count in the pre- sentation experiment, but was generally not supported by the lack of effect on fixation frequency in the simulator experiment and longest glance duration in both experiments. The increased number of fixations required by high clutter displays in the presentation experiment cor- roborates findings of Beck et al. (2010) and Moacdieh and Sarter (2012), who also reported increased fixations with increased display clutter in map search tasks. Beck and colleagues suggested that there was a direct cause-effect relationship between the increased number of fixations required to perceive visual information in high clutter displays and increased RT associated with such displays. The higher number of fixations in the presentation experiment suggests participants had greater difficulty in searching high clutter displays due to more visual imagery than in low clutter displays, likely leading to the increased RT associated with high clutter displays. In the simulator experiment, the small differences in fixation frequency and longest glance duration magnitudes for low- and high-clutter displays suggests that drivers at- tempted to avoid clutter within displays, leading to a lack of effect. This finding is not unprecedented, as Lohrenz and Beck (2010) found par- ticipants tended to avoid areas of high clutter in a map search task. Further contributing to the lack of a significant difference was the driving scenario, which was purposely chosen to be a difficult scenario.

Participants drove in fairly dense traffic in an urban setting with a large amount of visual information to be processed. The demanding scenario did not afford drivers the luxury of allocating a larger proportion of attention to the navigation display. This explanation is in-line with the findings of Metz et al. (2011), who observed that drivers are very adept at adapting attention patterns to efficiently handle situational demands of driving. Comparing these results with existing findings in the clutter literature, Moacdieh and Sarter (2012) reported mean glance duration to target AOIs were not significantly affected by clutter in a Where's Waldo search task, but mean fixation duration and the number of fixations increased with increasing clutter. Similarly, van Orden et al. (2001) reported that fixation frequency was highly predictive of clutter in a mock warfare task in which participants were asked to classify displayed aircraft as friend or foe. In physician search of an electronic medical records display, Moacdieh and Sarter (2015b) reported fixation count increased with higher clutter, but clutter had no significant effect on mean fixation duration. In general, more research is needed to completely characterize the relationship of display clutter and attention allocation, and any dependencies on task demands and domain.

High clutter displays were expected to yield higher subjective per- ceptions of clutter, according to H3. Significant effects of clutter on CR and RCS in both experiments supported the hypothesis. Results across both experiments provide evidence that the RCS was able to differ- entiate between display conditions in a consistent manner (unlike CR values) and can be used effectively in the driving domain (in addition to the aviation domain, for which it was initially developed). In the flight simulation experiment conducted by Moacdieh et al. (2013), pilots stated that they preferred “moderate” clutter conditions to “low” and “high” conditions because moderate clutter provided enough informa- tion to perform flight tasks, but not too much information that it made the display distracting or difficult to search. This finding suggests that pilots and human factors analysts may differ in their identification of clutter vs. task-relevant information and that some of the imagery provided in Moacdieh et al. “moderate” condition may not have been clutter (similar to the issue observed with the Kim et al. (2011) study).

H4 stated that clutter would have differential effects on the response measures based on the nature of the experiment task design. Results supported the hypothesis, as the effects of clutter on performance, at- tention allocation, and perceived clutter substantially differed between the two experiments. Regarding performance, it should be noted that

Fig. 9. The effect of clutter on RCS in (a) the presentation experiment and (b) the simulator experiment.

C. Pankok, D. Kaber Applied Ergonomics 69 (2018) 136–145

143

differences in the tasks posed by each experiment prevented us from measuring the same performance metrics across experiments. Therefore, it is possible that the observed performance differences could be attributed to the metrics themselves (RT vs. vehicle control performance) vs. the clutter manipulation. Having said this, clutter was found to significantly alter attention allocation and performance in the presentation experiment but not in the driving simulation (despite all other experimental variables being fixed) indicating a clear sensitivity of clutter effects to static vs. dynamic imagery as well as part-task vs. multi-tasking performance. Participants in the presentation experiment did not have to balance the competing demands of a difficult driving task, like the simulator participants, leading to longer glances at the navigation display and a significant effect of clutter on fixations. These results confirm a trend in the literature, which has demonstrated that tasks requiring more attention to the display are more susceptible to the effects of display clutter. For example, many map search tasks (e.g., Lohrenz and Beck, 2010) and supervisory tasks (e.g., van Orden et al., 2001) that require extended attention to a display have revealed sig- nificant performance and attention allocation effects of clutter, but the results are less pronounced in aviation displays where “clutter” may support performance in another task (e.g., Kaber et al., 2013a). Related to this, Moacdieh and Sarter (2015b) reported that task difficulty and stress significantly interacted with clutter in the electronic medical records display search, which required extended gazes to a cluttered screen. The authors reported that differences in attention allocation were more pronounced between high- and low-clutter displays when task difficulty and stress were increased.

Differences in the nature of our experimental tasks also had differ- ential effects on CRs. In both experiments, there were much larger ranges of CR values than RCSs, a finding that was also reported by Kaber et al. (2013a) for commercial pilot perceptions of display clutter. Kaber and colleagues suggested that simple, unidimensional CRs could be highly biased, unlike the structured RCS method, which attempts to account for both operator goals in display use and display feature in- fluences on clutter. This bias was evident across the present experi- ments. Figs. 8 and 9 revealed the means and standard deviations of RCSs to be fairly consistent across experiments, which was not the case for the overall CRs. Since both experiment samples were asked to provide subjective assessments of clutter for the same displays when posed with the same navigation queries, the unidimensional CR was heavily influenced by the experiment context. That is, participants who were not exposed to competing driving demands in the presentation experiment generally rated clutter as much higher than participants who were required to use the navigation displays while controlling the driving simulator. Such a difference between the two studies was not observed for the highly-structured RCS. It is possible that the decreased attention allocated to the display by drivers in the simulator experiment led participants to underestimate the amount of clutter in the display, evidenced by the CRs across both experiments. This supports the results reported by Kaber et al. (2013b), who found lower CRs for commercial pilots in a simulator experiment compared to commercial pilots in a static, presentation-based experiment similar to the presentation ex- periment as part of this study. This comparison provides evidence that the RCS method may be superior for subjective measurement of per- ceptions of clutter when considering the potential for bias in ratings by external and environmental factors, such as the domain or context within which ratings are made.

Taken as a whole, the results suggest that drivers adopted a clutter avoidance strategy when there was high demand for attentional and cognitive resources in the driving simulation experiment. It is possible that participants leveraged their expectancy of information content and position on displays, rather than on salience of the information, in order to streamline search performance, thus avoiding cluttered areas (Horrey et al., 2006; Pankok and Kaber, 2017; Wickens et al., 2003). This strategy also explains the subjective CR results, as participants who avoided the cluttered displays would be expected to provide lower

overall unidimensional CRs.

5. Conclusions

The objectives of the current research were to assess the effect of display clutter on driver performance, attention allocation, and sub- jective perceptions of clutter as well as assess the interaction between clutter and the nature of the task (i.e., presentation-based vs. simulator) on those same responses. Clutter had a significant effect on perfor- mance and attention allocation in the presentation experiment, but no effect in the simulator. Furthermore, there were differential effects of clutter on the unidimensional CR between experiments, but the highly- structured RCS yielded similar results across experiments. Results sug- gest that clutter has a much greater effect on tasks in which a display requires extended glances and a high proportion of attention (e.g., map search tasks, supervisory control tasks, etc.) vs. tasks for which the display is merely used as a support tool for performance (e.g., using a navigation display while driving). Finally, the results suggest that the RCS is a more reliable measure of subjective perceptions of clutter than overall CRs.

5.1. Limitations and future work

Some aspects of the experiments as part of this study may limit generalizability of results to various task domains and operator goal states. First, the driving simulation was paused during each test trial in order to collect driver subjective perceptions of display clutter. It is possible that such pauses were intrusive to performance and that other real-time rating collection approaches could contribute to simulation realism, potentially altering results. A different scenario containing less challenging driving scenarios, such as freeway driving under low traffic density, might also have increased driver attention and performance sensitivity to display clutter. Finally, only two levels of display clutter were investigated here since previous research suggested that there was a linear relationship between clutter and performance degradations (Wickens et al., 2000). Considering more display configurations (si- milar to the approach taken by Kim et al., 2011; Kaber et al., 2013a) may have revealed a higher-order relationship and a differential effect on outcomes.

Acknowledgment

This research was supported by a grant from the U.S. National Institute for Occupational Safety and Health (NIOSH: No. 2 T42 OH008673-08). The opinions expressed in this report are those of the authors and do not necessarily reflect the views of NIOSH.

References

Beck, M.R., Lohrenz, M.C., Trafton, J.G., 2010. Measuring search efficiency in complex visual search tasks: global and local clutter. J. Exp. Psychol. Appl. 16 (3), 238–250. https://doi.org/10.1037/a0019633.

Boston, B.N., Braun, C.C., 1996. Clutter and display conformality: changes in cognitive capture. In: Proceedings of the Human Factors and Ergonomics Society 40th Annual Meeting, pp. 57–61.

Chen, X., Gao, L., Wei, Z., Li, Q., 2007. Correlation between driver factors and traffic accident rate. J. Beijing Inst. Technol. (Soc. Sci. Ed.) 33, 697–701.

Dingus, T.A., Antin, J.F., Hulse, M.C., Wierwille, W.W., 1989. Attentional demand re- quirements of an automobile moving-map navigation system. Transport. Res. Pol. Pract. 23 (4), 301–315. https://doi.org/10.1016/0191-2607(89)90013-7.

Giesbrecht, F.G., Gumpertz, M.L., 2004. Planning, Construction, and Statistical Analysis of Comparative Experiments. John Wiley and Sons, Inc, Hoboken, NJ.

Holmqvist, K., Nystrom, M., Andersson, R., Dewhurst, R., Jarodzka, H., van de Weijer, J., 2011. Eye Tracking: a Comprehensive Guide to Methods and Measures. Oxford University Press, New York.

Horrey, W.J., Wickens, C.D., Consalus, K.P., 2006. Modeling drivers' visual attention al- location while interacting with in-vehicle technologies. J. Exp. Psychol. Appl. 12 (2), 67. https://doi.org/10.1037/1076-898X.12.2.67.

Kaber, D.B., Alexander, A.L., Stelzer, E.M., Kim, S., Kaufmann, K., Hsiang, S., 2008. Perceived clutter in advanced cockpit displays: measurement and modeling with experienced pilots. Aviat Space Environ. Med. 79 (11), 1007–1018. https://doi.org/

C. Pankok, D. Kaber Applied Ergonomics 69 (2018) 136–145

144

10.3357/ASEM.2319.2008. Kaber, D., Kaufmann, K., Alexander, A.L., Kim, S., Naylor, J.T., Prinzel III, L.J., ... Gil, G.,

2013a. Testing and validation of a psychophysically defined metric of display clutter. J. Aero. Inf. Syst. 10 (8), 359–368.

Kaber, D.B., Naylor, J.T., Gil, G., Pankok, C., Kim, S., 2013b. Influence of flight domain and cockpit display dynamics on clutter perceptions. J. Aero. Inf. Syst. 10 (12), 550–559. https://doi.org/10.2514/1.I010036.

Kennedy, R.S., Lane, N.E., Berbaum, K.S., Lilienthal, M.G., 1993. Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. Int. J. Aviat. Psychol. 3 (3), 203–220. https://doi.org/10.1207/s15327108ijap0303_3.

Kim, S., Prinzel, L.J., Kaber, D.B., Alexander, A.L., Stelzer, E.M., Kaufmann, K., Veil, T., 2011. Multidimensional measure of display clutter and pilot performance for ad- vanced head-up display. Aviat Space Environ. Med. 82 (11), 1013–1022. https://doi. org/10.3357/ASEM.3017.2011.

Kun, A.L., Paek, T., Medenica, Z., Memarovic, N., Palinko, O., 2009. Glancing at personal navigation devices can affect driving: experimental results and design implications. In: Proceedings of the 1st International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pp. 129–136.

Lansdown, T.C., 2000. Driver visual allocation and the introduction of intelligent trans- port systems. In: Proceedings of the Institution of Mechanical Engineers Part D: Journal of Automobile Engineering, vol. 214. pp. 645–652. https://doi.org/10.1243/ 0954407001527510.

Lim, J.H., Tsimhoni, O., Liu, Y., 2010. Investigation of driver performance with night vision and pedestrian detection systems- Part 1: empirical study on visual clutter and glance behavior. IEEE Trans. Intell. Transport. Syst. 11 (3), 670–677. https://doi.org/ 10.1109/TITS.2010.2049843.

Lohrenz, M.C., Beck, M.R., 2010. Evidence of clutter avoidance in complex scenes. In: Proceedings of the Human Factors and Ergonomics Society 54th Annual Meeting, pp. 1355–1359.

Metz, B., Schomig, N., Kruger, H., 2011. Attention during visual secondary tasks in driving: adaptation to the demands of the driving task. Transport. Res. Part F: Psychology and Behaviour 14 (5), 369–380. https://doi.org/10.1016/j.trf.2011.04. 004.

Moacdieh, N.M., Prinet, J.C., Sarter, N.B., 2013. Effects of modern primary flight display clutter: evidence from performance and eye tracking data. In: Proceedings of the Human Factors and Ergonomics Society 57th Annual Meeting, pp. 11–15.

Moacdieh, N.M., Sarter, N.B., 2012. Eye tracking metrics: a toolbox for assessing the effects of clutter on attention allocation. In: Proceedings of the Human Factors and Ergonomics Society 56th Annual Meeting, pp. 1366–1370. https://doi.org/10.1177/ 1071181312561391.

Moacdieh, N., Sarter, N., 2015a. Display clutter: a review of definitions and measurement

techniques. Hum. Factors 57 (1), 61–100. https://doi.org/10.1177/ 0018720814541145.

Moacdieh, N., Sarter, N., 2015b. Clutter in electronic medical records: examining its performance and attentional costs using eye tracking. Hum. Factors 57 (4), 591–606. https://doi.org/10.1177/0018720814564594.

Montgomery, D.C., 1991. Design and Analysis of Experiments. John Wiley & Sons, New York, NY.

Pankok, C., Kaber, D., 2017. Influence of task knowledge and display features on driver attention to cluttered navigation displays. In: Proceedings of the Human Factors and Ergonomics Society 61st Annual Meeting, pp. 1768–1772. http://doi.org/10.1177/ 1541931213601924.

Rotman, S.R., Tidhar, G., Kowalczyk, M.L., 1994. Clutter metrics for target detection systems. IEEE Trans. Aero. Electron. Syst. 30 (1), 81–91. https://doi.org/10.1109/7. 250408.

St John, M., Smallman, H.S., Manes, D.I., Feher, B.A., Morrison, J.G., 2005. Heuristic automation for decluttering tactical displays. Hum. Factors 47 (3), 509–525. https:// doi.org/10.1177/154193120404800330.

Transportation Research Board, 2010. Highway Capacity Manual 5th Edition. Transportation Research Board, Washington, D. C.

van Orden, K.F., Limbert, W., Makeig, S., Jung, T., 2001. Eye activity correlates of workload during a visuospatial memory task. Hum. Factors 43 (1), 111–121. https:// doi.org/10.1518/001872001775992570.

Ververs, P.M., Wickens, C.D., 1996. The effect of clutter and lowlighting symbology on pilot performance with head-up displays. In: Proceedings from the Human Factors and Ergonomics Society 40th Annual Meeting, pp. 62–66.

Wang, Y., Mehler, B., Reimer, B., Lammers, V., D'Ambrosio, L.A., Coughlin, J.K., 2010. The validity of driving simulation for assessing differences between in-vehicle in- formational interfaces: a comparison with field testing. Ergonomics 53 (3), 404–420. https://doi.org/10.1080/00140130903464358.

Wickens, C.D., Goh, J., Helleberg, J., Horrey, W.J., Talleur, D.A., 2003. Attentional models of multitask pilot performance using advanced display technology. Hum. Factors 45 (3), 360–380. http://doi.org/10.1518/hfes.45.3.360.27250.

Wickens, C.D., Kroft, P., Yeh, M., 2000. Data base overlay in electronic map design: testing a computational model. In: Proceedings of the IEA 2000/HFES 2000 Congress, pp. 451–454.

Wickens, C.D., Long, J., 1994. Conformal symbology, attention shifts, and the head-up display. In: Proceedings of the Human Factors and Ergonomics Society 38th Annual Meeting, pp. 6–10.

Wierwille, W.W., 1993. Visual and manual demands of in-car controls and displays. In: Peacock, B., Karwowski, W. (Eds.), Automotive Ergonomics. Taylor & Francis, London, pp. 299–320.

C. Pankok, D. Kaber Applied Ergonomics 69 (2018) 136–145

145