Topic: Downfall of Brick and Mortar stores

Webster & Ahuja/Enhancing the Design of Web Navigation Systems

Qarterly RESEARCH ARTICLE ENHANCING THE DESIGN OF W E B NAVIGATION SYSTEMS: THE INFLUENCE OF USER DISORIENTATION ON ENGAGEMENT AND PERFORMANCE^

By; Jane Webster Queen's School of Business 143 Union Street Kingston, ON K7L 3N6 CANADA [email protected]

Jaspreet S. Ahuja 132 Alfred Patterson Drive Markham, ON L6E1K9 CANADA [email protected]

Abstract

This paper draws on research from a wide literature base to develop a model relating Web navigation systems, disorienta- tion. engagement, user performance, and intentions. The mode! is tested in an experimental study examining the effects of one simple and two global navigation systems. Although well-accepted design guidelines were followed for the first global navigation system, it was not superior to the simple system. However, the second global navigation system resulted in lower disorientation than the simple system. Based on the study's results, two design guidelines to govern the development of future Web-based systems are suggested.

Ron Wcbcr was the accepting senior editor for Ihis paper. John Gallaugher iind Anthony R. Hendrickson served as reviewers. The third reviewer and the associate editor chose to remain anonymoiis.

Keywords: Disorientation. losu engagement, flow. World Wide Web, Internet, design. Web-based learning, electronic commerce, electronic shopping. Web navigation, commercial Web site, performanee, eftlciency, intentions

Readers need a .sense of context, of their place with- in an organization of information. In paper docu- ments this sense of "where you are" is a mixture of graphic and editorial organizational cues supplied by the graphic design of the book, the organization of the text, and the physical sensation of the book as an object. Electronic documents provide none of the physical cues we take for granted in assessing information. When we see a Web hypertext link on the page we have few cues to where we will be led. how much information is at the other end of the link, and exactly how the linked information relates to the current page. Even the view of individual Web pages is restricted for many users. (Lyneh and Horton 2002)

On IBM's website, the most popularfeattire was the search function, because the site was difficult to navigate. The second most popular feature was the "help " button, because the search technology was so ineffective. IBM's solution was a 10-week effort to redesign the site....In the first week after the redesign, use of the "help" button decreased 84 percent, while sales increased 400 percent. (UsabilityNet2OO3)

MIS Quarterly Vol. 30 No. 3 pp. 661-678/September 2006 661

Wet)ster & Ahuja/Enhancing tiie Design of Web Navigation Systems

Introduction

A common problem users face on the Web is disorientatlon (Danielson 2003; Head et al. 2000; Wen 2003; Yu and Roh 2002) or the tendency to lose one's sense of location in a Web site. This can result in frustration., loss of interest, and a mea- surable decline in user efficiency (McDonald and Stevenson 1998). Despite its promise to produce complex, richly inter- connected, and cross-referenced bodies of multimedia infor- mation, the Web can also produce disorganized tangles of haphazardly connected documents that make locating infor- mation difficult (Utting and Yankelovich 1989). In addition, the organization ofa site is an important influence on Web- site effectiveness (Palmer 2002) and user attitudes (Chen and Wells 1999). Thus, researchers suggest that well-designed navigation systems can help to reduce disorientation (Head et a!. 2000).

Navigation systems, or systems that support users' deter- mination of paths through Web sites, may include features such as links to other pages, search facilities, directories, or site maps. According to Nielsen, users have difficulty finding infonnation and need support in the form ofa strong sense of structtire and place; these can be provided by a navigation system. Nevertheless, iack of navigation support was cited as one ofthe 10 most-common mistakes in Web design in both 1996 and 1999 (Nielsen 1996a, 1999b). Surveys indicate that over 60 percent of Web users are either "searching" or "browsing" (Pitkow et al. 1998), reinforcing the importance of presenting infonnation in an easily searchable and locat- able manner. Consequently, designers have proposed that any uscable hypertext system should include an effective mech- anism for navigation (Lightfoot 1997).

Staying oriented is frequently a problem in virtual environ- ments (Darken and Sibert 1993). Thus, the basic rationale for navigation support is to remind users where and at what level they currently are in Web sites (Spiller and Lohse 1998). In 1994, about 90 percent of commercial Web sites had poor usability: they either did not have the information users needed, or the information was not easily locatable (Nielsen and Sano 1994). By 2002, Nielsen reported that only 49 per- cent of sites followed usability guidelines (such as providing search engines). Even the highest-scoring site only followed 66 percent of the guidelines (Netsavvy Communications 2002). According to a Zona Research Inc. (1998) survey, 28 percent of Intemet-sawy users reported that it was either somewhat or extremely difficult to locate specific products, and 62 percent gave up looking for products on-line. Finally, a protocol analysis concerning the usability of a popular greeting card Web site demonstrated many navigation issues, including problems with finding desired links and informa-

tion, getting lost, and determining the correct buttons for navi- gation (Benbunan-Fich 2001). In this study, we investigate navigation support by examining the effects of simple and global navigation systems on disorientation and performance.

Reducing disorientation not only has performance benefits but also allows users to be more engaged in their Web inter- actions. Engagement, or the feeling that a system has caught, captured, and captivated user interest (Jacques et al. 1995), is an important goal for system design (Mayes 1992) and may even encourage users to revisit Web sites (Kim and Moon 1998).

Although these represent important issues faced by Web designers, little empirical research has addressed user percep- tions of disorientation (OInick Kutzschan and Webster 2006), improved navigation support (Park and Kim 2000), or increased user engagement (Mallon and Webb 2000). Thus, our study proceeds as follows, We begin by describing per- ceived disorientation, navigation, and engagement, distin- guishing between related but distinct constructs. We then present a model linking these constructs to performance and intentions by drawing on research from a variety of areas, including human factors, cognitive psychology, marketing, and information systems. We present an experiment that tests this modei. We conclude by presenting implications for practice and research.

Theoretical Model

Our model has three main constructs: disorientation, naviga- tion systems, and engagement. We develop hypotheses con- necting these constructs to user perfomiance and future intentions to use a system. For this study, we conceptualize perfonnance as both search correctness and search titne and conceptualize future intentions similar to the marketing construct of "intention to revisit" (Luna et al. 2002).

Perceived Disorientation

Disorientalion represents '^perhaps the oldest (and arguably the most devastating) problem of Web navigation" (Danielson 2003, p. 154). Disorientation has been defined as the ten- dency to lose one's sense of location and direction in a non- linear document (Ahtija and Webster 2001; Conklin 1987; Head et al. 2000). It has been called the "getting lost" phenomenon that occurs when "the user does not have a clear conception of relationships within the systetn, does not know his present location in the system relative to the display struc-

662 MIS Quarterly Vol. 30 No. 3/September 2006

Webster & Ahuja/Enhancing the Design of Web Navigation Systems

ture. and finds it difficult to decide where to look next within the system" (Woods 1984. pp. 229-230), and the "lost in hyperspace" phenomenon that occtirs when users experience "cognitive problems of finding their way in the infonnation space" (Smith 1996. p. 365).

Disorientation has led Web users to make sueh remarks as "I had specific places 1 wanted to go and couldn't understand how to find them" (Jarvenpaa and Todd 1997. p. 74). Disoriented users might reopen the same page (a single hyper- text document in the Web site) repeatedly. Disorientalion might also take different forms, sueh as those demonstrated in Web user surveys describing difficulties in finding pages known to exi.st, fmding a page already visited, and visualizing paths taken and paths that could be taken (Pitkow and Kehoe 1996).

Disorientation is related to, but conceptually distinct from, ease of use. or the "effort one experienees in the process of carrying out tasks using a given system" (Davis et al. 1992, pp. 1114-1115). Disorientation represents a feeling or state ofthe user (e.g.. "I didn't know how to get to my desired location" [Ahuja and Webster 2001, p. 18]), while ease of use refers to the user's perception of a system (e.g., "Leaming to operate [software name] would be easy for me" [Davis 1989, p. 340]). For instance, users may perceive that they learned to use a Web site easily, but can still feel disoriented. That is, they may find it easy to beeome skillful at using the Web site, but they still may beeome lost and not know how to get to their desired locations in the site. Empirical research further suggests that disorientation and ease of use are related but distinct constructs. In one study, differing menu layouts related to disorientation but not to ease of use (Bernard and Hamblin 2003). In another study, disorientation and ease of use loaded on separate factors in an exploratory factor analysis; in a second study reported in the same article, two competing a priori models were compared using confirmatory factor analysis," and a %' differenee test demonstrated that the "distinct yet correlated" model fit the data better (Ahuja and Webster 2001).

Our model incorporates perceived disorientation, rather than objective measures of users' actions. Disoriented actions are typically measured through the number of pages opened and the number opened more than once (McDonald and Stevenson I99f^). However, many pages may be opened for other reasons, such as exploring a Web site; thus, employing users' actions as a proxy for disorientation can be problematic

(Stnith 1996). In this regard, some research has found weak relations between perceived disorientation and users" aetions (Ahuja and Webster 2001). Other research has demonstrated that perceived disorientation relates to aceuraey of mental models and task time, while actions do not (Otter and Johnson 2000). Thus, perceived disorientation represents a more appropriate itidicator. Tliis assertion is consistent with those made by researchers in the behavioral sciences, who recom- mend that questionnaire measures represent more suitable methods of measurement for understanding users' intemal feelings (Sandelaiids and Buckner 1989).

Navigation Systems

Navigation has been defined as "a process of tracking one's position in a physical environment to arrive at a desired des- tination" (Cutmore et al. 2000, p. 224). Navigation, although originally denoting a physical process, has been extended to inelude the proeess of determining a palh to be traveled by any object through any environment (Darken and Sibert 1993), It "has emerged as a useful metaphor for information access" (Isakowitz and Bieber 1995, p. 6).

Computer-based navigation has been defined as "the deci- sions and actions that contribute to a person's ability to find and examine data organized in the computer medium" (Watts- Perotti and Woods 1999, p. 270). More generally, it has been conceived as "the creation and interpretation of an intemal mental model" (Spence 1999, p. 921). A mental model, in tum, has been described as the organization of knowledge about how a system works or operates (Norman 1988). Mental models may be more or less correct and may have been formed spontaneously via experience or systematically via training (Santhanam and Sein 1994). Correct mental models help users complete tasks effectively. Incorrect mental models can lead to performance breakdowns. Marchionini and Shneiderman (1988) propose that effective hypertext usage depends on the quality ofthe mental models users develop for systems.

Drawing on these definitions of navigation, we describe a Web navigation system as a system that is designed to aid users in the creation and interpretation of an intemal mental model that helps them find and examine data on a Web site.' Next, we compare simple with global navigation systems.

"The first model tested the hypothesis that disorientation was distinct yet negatively correlated with ease of use; the second that disorientation and ease of use were one construct.

A Web navigation system may be as straightforward as providing navigation bars (that is, specific areas on each page with links lo other pages). Web navigation syslems are browser-independent: that is, they e.\ist in addition to any navigation buttons, such as "Back" or "Home," on common browsers such as Netscape'" or Explorer'".

MIS Quarterty Vol. 30 No. 3/September2006 663

Webster & Ahuja/Enhancing the Design of Web Navigation Systems

Simple navigation systems present local links to other pages in the site (such as "previous" and "next"). Global navigation systems provide overviews of the entire site (such as site maps) on each page (Park and Kim 2000). Consider a site with three main sections: About Us. Catalog, and Contact Us. If the site is small, a simple navigation system could be implemented by putting a link on every page back to the front page. On the main page for each of those sections, or "second-ievel pages," a navigation bar could link to the site's front page (first level) and the other two sections" front pages (second levels). In contrast, a global navigation system might include navigation bars with links to About Us, Catalog, and Contact Us on all pages, a detailed site map. a table of con- tents, and a local search engitie (i.e., one that searches only the contents of that particular Web site).*

Global navigation systems are not yet the norm. Liu et al. (1997) found that only 40 percent of company sites surveyed provided search facilities. Moreover, only 30 percent provided an index or directory. Further, Spiller and Lohse (1998) reported that only 6 percent offered product search, and only 4 percent of sites had a site index. We further investigated these statistics by examining popular Web sites.' We found that

• 34 percent had a site map

81 percent had a search facility

• 93 percent had some type of navigation (70 percent had a "bar" and 23 percent had a "tree" type of navigation)

• most had navigation links at the top ofthe screen (89 percent had navigation at the top of the screen, 5! percent had partial navigation links at the bottom ofthe screen. 34 percent had some navigation at the left ofthe screen - these tiumbers do not add up to 100 percent

Nielsen (2000) poinis out that users often examine Web content while ignoring global navigation elements. In spite of this, he cautions thai navigation elements should not be removed from Web pages, proposing instead thai users will noiice ihesc elemenfs when they need Ihem. Rather, he suggests that global navigation should be limited to tlve or six useful elements, including features such as search functions, overviews of ihe eurrent region, and links to levels above the current location.

To conduet the eoding. we hired an HCI graduate student to code the most popular U.S. Web sites from home and work listed in the JupiterReseareh repoti. "Top 50 U.S. Web and Digital Media Properties of September 2002" (http://www.jupiterresearch.conx'bin'ilem.pl/home), Ofthe 50 sites listed, three requested personal infonnation such as SIN number; thus, 47 ofthe 50 sites were eoded. This student had not been involved wiih our study, nor was she aware of Ihe purpose ofthe coding. We asked her to eode a variety of design issues, ranging from "site map" (present or absent) to "loeation of navigation" to "consistency of displays"; we eoded two sites as examples for her, and she coded the remainder independently.

because many had navigation links in more than one part ofthe screen)

• none had stationary navigation links when scrolling through a screen

• 77 percent had navigation links on all pages

• 70 percent had consistent displays

These popular sites may have been created by professional design firms, but those created in-house may have even lower statistics. For example, a study of IS4 nonprofit organiza- tions revealed that only 30 percent had a site map, only 33 percent had good or excellent search facilities, 32 percent did not provide information concerning the user's location in the site, and 40 percent did not include clear navigation links (Cukier and Middleton 2003). Thus, many Web sites do not include all ofthe functionality of global navigation systems.

Although not yet a general practice, designers have proposed that a visual representation of the entire system will aid in navigation (Nielsen 1999a), and researchers are beginning to explore this area. Researchers have examined the effects of hyperlink annotations that represent Internet connection speeds ("traffic lights" [Campbell and Maglio 1999]). the impacts of a history mechanism (Head et al. 2000), the effects of "add-on" links (such as horizontal links to neighboring pages [Kim and Yoo 2000]), and the consequences of add-on links in conjunction with a history mechanism (Park and Kim 2000). However, these studies do not exarnine the effects of other global navigation system characteristics, such as detailed site maps or search functions.

Engagement

Users are engaged in a system when it "holds their attention and they are attracted to it for intrinsic rewards" (Jacques et al. 1995, p. 58), Users deseribe their experiences when inter- acting with engaging systems as feelings that the system has caught, captured, and captivated their interest (Jacques et al, 1995).'' Engagement is similar to flow, a state representing the extent of pleasure and involvement in an activity (Csikszentmihaiyi 1975), More specifically, flow has been described as a multidimensional construct encompassing perceptions of user control, attention focus, arousal of curio-

'Like engagement, involvement may also cxhibi! focused attention; however, involvement may occur because of task demands or deadlines and Ihus may not be enjoyable (Sandelands and Buekner 1989). In eontrast, engagement includes intrinsic interest.

664 MiS Quarteriy Vol. 30 No. 3/September 2006

Webster & Ahuja/Enhancing the Design of Web Navigation Systems

Figure 1. Model of Navigation Systems' Effects on Disorientation and Outcomes of Searching Web Sites

sity, and intrinsic interest in a computer interaction (Webster etal. 1993).

Engagement has been characterized as fiow without user control, or perceptions of attention focus, curiosity, and intrin- sic interest (Chapman etal. 1999). Engagement is a stibset of fiow and represents a more passive state. For instance, one may be highly engaged in observing (but not controlling what is happening in) a movie. Similarly, exploration-based training incorporates more user control than instruction-based training because the user sets the pace ofthe training. Thus, the former will more likely result in flow than the latter (Chapman et al. 1999). Analogously, one would expect that users exploring a Web site would experience more control (than those performing directed Web searches) because these users would be setting the pace and direction of their explora- tion. Thus, engagement appears to represent a more appro- priate construct than fiow when the user experiences less control, such as might occur in directed infomiation searches.^

Development of Hypotheses

Figure 1 shows the model to be tested. We first propose that global navigation systems will result in less disorientation

We incorporalcd engagement, rather than flow, in this model because the users in our study were directed to find specific infnrmution and were given a short period of lime lo do so. The users were not expected lo have much opportunity to e.speriencc the control aspect of flow.

than simple navigation systems. As described earlier, a common cause of disorientation is determining where a link leads or being able to retum to a document already visited (Utting and Yankelovich 1989; Wen 2003; Woods et al. 1990). This has led designers to propose that Web browsing can be improved by maintaining the context around the hypertext being viewed through the use of well-designed navigation systems (Newfield et al. 1998).

Global navigation systems provide a visual representation of the entire system. We suggest that these visual representa- tions should reduce users' cognitive overload resulting from the additional effort and concentration needed to maintain several tasks or trails at one time (Conklin 1987). We pro- pose that users of global navigation systems will rely more on recognition than recall by identifying a piece of information and referencing it to infonnation already stored in long-tcmi memory (Wickens 1992). For example, as we move through a Web site with a global navigation system, we may recognize the props that are present, such as buttons, icons, and maps, and use them with relative ease. This process is passive, and we rely on recognition memory to tell us what we are experiencing. The use of anchors, or invariant features ofthe Web site, helps reduce the load on users' cognitive resources (Tognazzine 1998; Woods 1984). Therefore, global naviga- tion systems should help users reeognize their positions, thereby decreasing cognitive overload and disorientation.

In contrast, when users travel to deeper levels of a Web site using a simple navigation system, the previous links disappear

MIS Quarterly Vol. 30 No. 3/SeptBmber2006 665

Webster & Ahuja/Enhancing the Design of Web Navigation Systems

and are replaced by new links. To travel back, the user must click on a '"Back" or "Home" link (Yu and Roh 2002). This results in "volatile" interaction with the Web site, because users encounter differing navigation links over a short time period (Danieison 2003). Navigational volatility, or "the number of hyperlinks appearing on the destination page that did not appear in the same screen location as on the source page" correlates with perceived disorientation (Danielson 2003, p. 141). Further, because simple navigation systems provide fewer site representations, they result in higher reliance on recall than recognition. Recall involves active, deliberate retrieval of information from long-term to short- term memory (Trumbo 1998). Short-tenn memory, however, has limited capacity (Head et al. 2000) Cognitive resources used for planning where to go and keeping track of position can become overloaded (Morris and Hinrichs 1996). As a result, the cognitive resources available for comprehension and understanding Web site material will be reduced (Thuring et ai. 1995; Yu and Roh 2002). This will result in higher cognitive loads for simple navigation systems.

We propose that global navigation systems help create more- aecurate mental models by giving users a feel for the structure of Web sites as they move through them. This is analogous to becoming increasingly familiar with a geographical environment, such as a city where we newly reside (Thorn- dyke 1981). Through accurate mental models, users should be able to determine their locations and what alternatives they have to progress to desired pages. In short, a well-designed conceptual model should reduce users" memory loads (Otter and Johnson 2000) because it provides a blueprint for the Web site (Yu and Roh 2002).

A global navigation system could also add to the user's cognitive load because it contributes to the amount of infor- mation presented to users (Yu and Roh 2002). On balance, however, we propose that cognitive load and disorientation decrease with well-designed global navigation systems. Therefore, we propose

Hypothesis 1: Global Weh navigation systems result in lower perceived disorientation than sitnple navi- gation systems.

Compared to global navigation systems, simple navigation systems can result in reduced performance because they re- duce the speed of completing a given task or increase the time to find information (Conklin 1987; Fleming 1998; Nielsen 1996a, 1999a). These outcomes occur because simple naviga- tion systems do not have a fiexible path mechanism: users must pass through intermediate pages to locate infonnation (Yu and Roh 2002). In contrast, global navigation systems

help orient users and increase their efficiency. Users can travel to and find desired information more quickly (Darken and Sibert 1993; Yu and Roh 2002). Users become impatient with inefficient Web sites and are inclined to go elsewhere (Nielsen and Wagner 1996). Research has also shown that users browse more quickly with a global than a simple navigation system (Yu and Roh 2002). Moreover, they are less likely to abandon information-seeking tasks when a site map is present (Danielson 2002). Thus, we propose

Hypothesis 2: Global Web navigation systems result in higher user performance than simple navigation systems.

Disorientation should relate to users' perceptions of engage- ment and ultimately their perfonnance and intentions to use a Web site. Although little empirical research has addressed engagement in computer interactions, we can draw on related research to propose how engagement relates to other con- structs in our model. As described above, engagement is made up of three dimensions: attention focus, arousal of curiosity, and intrinsic interest. Research examining these dimensions should help to explain how engagement might relate to other construets in the model.

Users who feel lost will experience less attention focus while searching the Web. Practitioners propose that engagement with a Web site will be linked to how successfully the user can move across the pages and through the space (Fleming 1998). Further, they suggest that disoriented users who are unable to find the infonnation they want eannot become engrossed in what they are doing (e.g.. Spool et al. 1999). Thus, higher disorientation should lead to lower perceptions of engagement.

Hypothesis 3: Higher perceived di.sorientation results in lower engagement with a Web site.

We propose that engagement will relate positively to perfor- mance because the increased focus of attention should lead to more effective Web searches. Research on fiow and related constructs supports this assertion. For example, researchers have proposed that Web fiow should be a central construct in explaining users' Web shopping experiences (Oinas- Kukkonen 2000). Research has shown that higher flow relates to outcomes like extent of use and expected use ofthe Web, quantity of communication, and training outcomes (e.g., Novak et al. 2000: Trevino and Webster i 992; Webster and Martocchio 1995). Research has also found positive relation- ships between (1) Web enjoyment and frequency of Web use for entertainment (Atkinson and Kydd 1997) and (2) intrinsic motivation and quality of output (Davis et al. 1992). Thus, we propose

666 M/S Quarterly Vol. 30 No. 3/September 2006

Webster & Ahuja/Enhancing the Design of Web Navigation Systems

Hypothesis 4: Higher engagement in a Web site resulis in higher perfonnance.

Engagement should also positively affect intentions to use a Web site in ioilure because of users' intrinsic interest, Engaged users enjoy the activity or product, which may make them want to prolong the activity (Sandelands 1988) or use the product again (Jordan 1998). As with flow, the positive subjective experience becomes an important reason for performing the activity (Csikszentmihalyi 1975) and intending to revisit a site (Koufaris 2002; Luna et al. 2002). Thus. higher engagement should result in a more positive view of the computer interaction and higher motivation to interact with the software in the future (Webster et al. 1993).

Related empirical research helps to explain the expected positive relationship between engagement and intentions. Users visit Web sites that facilitate higher tlow more regularly and for longer periods of time (Nel et al. 1999) and report higher intentions to revisit the sites (Luna et al. 2002). Eurther. higher How relates to higher voluntary use (Webster ct al. 1993). Trainees who experience higher flow report higher transfer of training to the workplace (Webster and Martocchio 1995). Similarly, users who enjoy a computer activity report higher intentions to use it in the future (Davis et al. 1992), those with more positive attitudes toward a Web site arc more likely to use it (Lederer et al. 1998). and those with higher intrinsic enjoyment exhibit higher intentions to return to the Web site (Koufaris 2002). Thus, we propose

Hypothesis 5: Higher engagement in a Web site leads to higher intentions to use the Web site in the future.

Users who perform better while searching a Web site should be more likely to want to revisit the Web site in the fiiture. Thus, we propose

Hypothesis 6: Higher performance in using a Web site leads to higher intentions to use that Web site in the future.

Finally, users will lose interest in Web sites when they experience disorientation (McDonald and Stevenson 1998) because they become frustrated and cannot accomplish their goals (Bessiere et al. 2003). Given that frustration results in outcomes such as withdrawal, we suggest that more dis- oriented users will be less likely to want to use the Web site in the future. Thus, we propose

Hypothesis 7: Higher perceived disorientation results in lower intentions to use the Web site in the ftiture.

Methodology

The hypotheses were tested in an experiment that compared three versions of the same on-line leaming Web site that differed in the navigation system used but were consistent in content. Participants were assigned randomly to one of three experimental conditions using a pretest-posttest control group design. This design allowed us to control for other unwanted influences. Before conducting the main experiment, the experimental manipulations and measures were refined atlcr two pilot experiments. In the main experiment, participants searched for specific infomiation on the Web sites. The three systems were placed on a local server, reducing variance in Web page retrieval times. Each session lasted almost I hour. Two graduate research assistants conducted all experimental sessions. A script was used by the research assistants to give instructions to the partieipants regarding the purpose ofthe study and the details of the task. Data were collected by (1) interaction logs recorded by server software. (2) perfor- mance on a formal search, and (3) preexperimental and post- experimental questionnaires. (Appendix B outlines the measures and their correlations.)

Experimental Manipulations

The Web site used for the experiment was adapted from an ongoing Technical Writing course offered on the Web by a North American university. This course, designed to teach principles of technical writing to nontechnical students, included complete books on various aspects of technical writing. Eor the experiment, the amount of content was reduced., resulting in three hierarchical levels; nevertheless, this had no impact on the look and feel ofthe site." In the course, users were presented with an opening graphical view of their virtual office and various options such as Communi- cation, Assignments. Bookshelf. Information, and Calendar.

We compared a simple navigation system and two global navigation systems (see Figure 2, which diagrams the navigation systems found at a second level for "Communi- cation." and Table 1, which compares the features in these navigation systems). All systems used the course's original mixed topology structure and some type of navigation system. The first navigation system consisted ofthe existing navi- gation system ofthe Web course. It contained simple links; that is. links to the following page and to lower-level sections. Both global navigation systems allowed participants to move

A study of on-line stores revealed that most had three or fewer levels (Spilter and Lohse 1998).

MIS Quarterly Vol. 30 No. 3/September2006 667

Webster & Ahuja/Enhancing the Design of Web Navigation Systems

Simple Navigation System (Control):

Communication

Newsgroup Mail List Chat Suggestions Instructor's Comments

Basic Global Navigation System (Condition #1):

Home Site Map Communication Calendar Assignments Instructor's Comments Bookshelf Course Info

Newsgroup Mail List Chat Suggestions Instructor's Comments

Site Map

Search this Site

Home Communication Calendar Assignments Instructor's Comments Bookshelf Course Info

Enhanced Global Navigation System (Condition #2).

Virtual office +Communication

• Newsgroup • Mail List • Chat " Suggestions • Instructor's Comments

+ Calendar + Assignments + Instructor's Comments + Bookshelf + Course Info. Site Map

Search this Site

Figure 2. Second-Level Screen Layouts of Navigation Systems

668 M/S Quarteriy Vol. 30 No. 3/September 2006

Webster & Ahuja/Enhancing the Design of Web Navigation Systems

Table 1. Comparison of Navigation Conditions

Simple Navigation System (Control)

• Hyperlinks only

Navigation disappears when scrolling {i.e., hyperlinks for previous page disappear)

Basic Global Navigation System (Condition #1)

• Navigation bars (top = current and parent level, left = child level, and bottom = top level)

• Site-map (in top & left) • Search form (on left) • Navigation can (partially) disappear

when scrolling (site-map, search form and navigation bars on left and top; bottom bars appear when scrolling to bottom)

Enhanced Global Navigation System (Condition #2)

• Navigation tree (Explorer" view in left frame)

• Site-map (in left frame) • Search form (in left frame) • Navigation always viewable when

scrolling (site-map, search form and navigation tree always in left frame)

through documents by clicking on links and navigating in a hierarchy. Both were text-based and were available on all pages." Additionally, both systems included backtrack facilities on all pages, cross-linking in certain locations, and search fonns and site maps.'"

Both global navigation systems were evaluated by the university's local Association for Computing Machinery CHI (Computer-Human Interaction) Group for consistency with usability guidelines. More specifically, the CHI Group compared the sites with commonly accepted guidelines developed by the HCI community (e.g., Fleming 1998; Fumas 1999; Nielsen 1999b; Tognazzine 1998; Woods 1984). These guidelines included (1) providing a visual representation of the system; (2) having a broad rather than a deep network of menus; (3) providing invariant features; (4) having facilities to backtrack and return to the parent level; (5) providing self-explanatory labels on the navigation system; (6) indicating the user's position in the Web site;

Graphics were incorporated in the systems, but not tor the navigation systotns. Text was selected over graphics for the navigation systems, because text is recommended for persons with disabilities and provides faster loading times (Lazaret al. 2003). A graphical navigation system may look attractive, but an appropriate one can only be created by a competent graphic designer. As noted by Nielsen (1999a), bloated graphic design is the worst offender in terms of response times, and graphics should not be used if not needed.

We chose to use site maps because some research comparing map displays with site maps (also called route lists) shows that site maps result in less confusion. Display maps generally are graphical representations of a space (like a map of a university campus), while sile maps generally are hierarehieal representations of a space (like an organization chan). Although Thomdyke and Hayes-Rolh (1982) expecled that the map display would be superior to a sitemap, they found the opposite: following a route helped reorient users when lost, whereas using a map display caused more confusion.

(7) making transitions to pages "graceful"; and (8) using a consistent display representation. Any changes recom- mended by the CHI Group were Implemented before conducting the pilot experiments described below.

The two global navigation systems, which we label basic and enhanced., differed mainly with respect to the position and design of the global navigation systetn. The basic global navigation system (condition #1) included navigation bars. It provided navigation features on the top, left, and bottom of all pages. Thus, the navigation system was present on every page. Nonetheless, the system might not always be totally visible to a user (e,g,, when scrolling down to the middle of a long page). For the enhanced global system (condition #2), the navigation system was available on every page (on the left, using frames) in the form of a collapsible tree structure (similar to the Explorer''"'̂ view in Windows^")., rather than navigation bars. Because frames were used, the enhanced global navigation system was always visible when scrolling down a long page. As described below, the design of these global systems was based on the results of two pilot experiments, which found that the basic global navigation system could have negative effects compared with a simple navigation system.

Pilot Experiments

Two pilot experiments were conducted. The first incor- porated two conditions: the Web site with a simple navi- gation system (the control group) and the same Web site with a basic global navigation system (condition #1). Participants were 21 graduate students from one department. Although we (and our local CHI Group) believed ihat we had designed the navigation system to be consistent with

MIS Quarterly Vol. 30 No. 3/September2006 669

Webster & Ahuja/Enhancing the Design of Web Navigation Systems

usability guidelines, we found a drop in performance and an increase in disorientation with the global navigation system (condition #1)."

During interviews, some condition #1 pilot participants observed that the global navigation system was complicated and difficult to use. For example, they felt that having to scroll to the top or the bottom o f a page to navigate was undesirable. They suggested that this resulted in forgetting current positions and the various options available. Our global navigation system apparently introduced more navi- gation volatility into the users* interactions (Danieison 2003). To address these problems, another condition was designed with a navigation tree (rather than a navigation bar) that was always stationary on the left ofthe screen (condition #2).

In the second pilot experiment, six graduate students from the same department pretested the experiment again with condition #2. Results for this experiment indicated that participants using the enhanced global navigation system found the site less disorienting. They achieved higher per- formance than those using either the simple or the basic global navigation systems.

Based on these pilot results, we decided to include ail three navigation systems in the main experiment. This allowed us to explore whether some global navigation systems (such as condition #2) might positively affect outcomes, while others (such as condition #1) might undermine performance more than even simple navigation systems.

Participants

Graduate and undergraduate students from various depart- ments at a North American university acted as participants. The only prerequisite for participation was that the students be knowledgeable about using computers and familiar with the World Wide Web. Participants were recruited by making brief presentations in courses with more than 100 students. Ofthe approximately 1,200 students approached in class- rooms. 275 signed up, and 207 participated in the experi- ment. Participants were paid $10 each. We chose to pay participants rather than offer incentives based on perfor- mance. We felt that offering performance incentives would be inconsistent with students learning from a Web-based system or customers looking for merchandise on a Web site.

None of the students had seen the Web site prior to the experiment.

Approximately 56 percent ofthe partieipants were in the age group of 17 to 20, and approximately 90 pereent were 24 or younger. Most were female (60 percent), undergraduate stu- dents (80 percent), from the Arts school (36 percent), with others from Applied Health Sciences, Engineering, Math, and Sciences. Most (88 percent) had spent at least 1 year using the Internet, which is nonnal for their age group (Pitkow et al. 1998). Over 80 percent had more than 9 years of education in the English language. Approximately 57 percent ranked themselves as novices with respect to tbeir knowledge of Web design.

Results I

This finding is not uncommon. Woods (1984) also noted previous research ihai has found an unexpected increase in disorientation wiih system.»i designed lo follow usability guidelines.

Hypotheses were tested with structural equation modeling (SEM) using AMOS (Arbuckle and Wothke 1999). The navigation dummy variables, Cl and C2 (described in Appendix A), were allowed to relate because, by design, dummy variables are partly redundant (Cohen and Cohen 1983, pp. 189-190).

The AMOS analysis for the hypothesized model resulted in a good model fit (Hair et al. 1995): a x' to degrees of free- dom ratio of 1.94, a goodness of fit index of .97, an adjusted goodness of fit index of .92, and a root mean square residual of .07). Figure 3 presents the path coefficients for the model. The hypotheses were tested by examining the significance of the standardized path coefficients shown in Figure 3.

HI, which predicted that users ofa global navigation system would report lower disorientation than those interacting with a simple navigation system, was supported. As shown in Table 2, those interacting with the enhanced global naviga- tion system reported the lowest disorientation. Interestingly, the path coefficients in Figure 3 show that participants inter- acting with the basic global navigation system did not report lower disorientation than those interacting with the simple system. Cl did not relate to disorientation (p = -.06) while C2 did (P - -.29).

H2 predicted that global navigation systems would out- perfonn simple navigation systems. The enhanced global navigation system demonstrated the highest performance (the highest score on the quiz and the lowest time), w hile the basic global navigation system demonstrated the lowest performance (Table 2). Path coefficients presented in Figure 3 support this conclusion: Cl related negatively to perfonnance (p = -.33), while C2 did not (p - .16).

670 MIS Quarterly Vol. 30 No. 3/September 2006

Webster & Ahuja/Enhancing the Design of Web Navigation Systems

. 1 7

.00

Internet Experience

Navigation Systems:

Ci

C2

.47 .

C 1 : -.33 C2: .16

User Performance

C1: -.06 C2: -.29

. 5 0 Engagement

-,02

*• Future Intentions

Perceived Disorientation

-.27 to Use

Notes: Bolded path coefficients significant at p < .05. C1 and C2 are two dummy variables representing the three navigation systems

Figure 3. Navigation Systems' Effects on Disorientation and Outcomes of Web Site Searching

Table 2. Comparison of Means across Navigation Conditions

Means (Controlling for Internet

Experience)

Perceived Disorientation

User Performance: Number of Correct Answers

User Performance: Time in Seconds

Simple Navigation System

(Control)

3.39

6.49

1199.38

Basic Global Navigation System

(Condition #1)

3.24

6.14

1359.53

Enhanced Globai Navigation System

(Condition #2)

2.65

6.89

1095.33

Four of the remaining tlve hypotheses were supported. H3, which predicted that lower disorientation would relate to higher user engagement., was supported (P ̂ -.50). H4, which predicted ihat engagement would relate positively to performance, was supported (p = .35). H5, which predicted that engagement would relate positively to intentions to use the Web site in the future, was supported (P = .65). H6, which predicted that performance would relate positively to intentions to use the Web in the future, was not supported (p ^ -.02). Finally, H7, which predicted that higher disorienta- tion would result in lower intentions was supported (P ̂ -.27).

Discussion and Conclusions

This study responds to calls for more research aimed at understanding users' Web behaviors (e.g., Pereira 2000) and specifically the effects of Web system designs on disorien-

tation (Otter and Johnson 2000; Park and Kim 2000). Our results indicate that the design of navigation systems ean affect user performance and disorientation. Moreover, they indicate that strong relationships exist between disorientation, engagement, perfonnance, and intentions to use a Web site.

Implications for Practice

Our study results demonstrate that simple navigation systems ean be better than some global navigations systems. We were initially surprised to fmd in our pilot experiments that adhering to usability guidelines for a global Web navigation system docs not necessarily result in less disorientation and higher perfonnance than a simple navigation system. Similarly, in our main experiment, we found no difference in disorientation between the simple navigation system and the basic globai navigation system. Moreover, we found better

MIS Quarterly Vol. 30 No. 3/September2006 671

Webster & Ahuja/Enhancing the Design of Web Navigation Systems

performance (task time) for the simple navigation system. Nonetheless, our enhanced global navigation system resulted in lower disorientation than the simple navigation system and the basic global system.'- Recall, our basic and global navi- gation systems differed in two ways: the type of navigation feature (bars for the basic and trees for enhanced), and the placement ofthe navigation features. Additionally, the navi- gation features could disappear when scrolling through a long page in our basic global navigation system, while they were always visible in our enhanced global navigation system. It seems that navigational volatility was reduced in the enhanced system. Thus, it appears that using a navigation tree rather than navigation bars and keeping the navigation features con- tinuously in view are important ways of reducing disorien- tation and increasing performance.

To keep the navigation features continuously in view, our enhanced global navigation system used frames. Neverthe- less, there are some problems with frames, such as book- marking, printing, and the accuracy of URLs" (Nielsen 1996b). In the short run, designers will need to weigh the reduced disorientalion benefits of frames against problems that can arise with the use of frames. In addition, our research suggests that designers should be cautious when designing navigation systems: they should not implement more com- plex global navigation systems without first comparing users' disorientation and performance when using simpler systems. Doing so may inadvertently result in negative outcomes for users.

We recommend that designers measure perceived disorien- tation before and after modifying Web designs. Perceived disorientation instruments like Ahuja and Webster's (2001) can be completed in a few minutes and are simpler to collect and analyze than users' actions, verbal protocols, or videos. In spite of this, designers will still want to collect users' actions for certain time-critical tasks (event-driven, high-con- sequence domains, such as nuclear power plant operations, patient monitoring systems, and space mission operations tWoods and Watts 1997]).

This study emphasizes the design of navigation systems to enhance user searching efficiency. On the other hand, mar- keters may have goals other than user efficiency—^they may want to create designs that encourage users to stay and ex-

"Panieipants" familiarity with an enhanced global navigation system through their university Web site is not an altemaiivc explanation for ihe findings; the university did not have an enhaneed system when the study was conducted.

Our participants did not need to bookmark, print, or reference URLs, and therefore these were not issues in the current experiment.

plore pages, especially those with infonnation about multiple related products. Further, we did not examine the effects of site content or graphics on facilitating user attraction. For sites designed to encourage users' exploratory behaviors, we suggest that designers measure users' perceptions of flow (which includes user control) rather than their engagement.

Limitations and Suggestions for Future Research

We studied university students interacting with an on-line learning Web site. Although we were able to control for Internet experience in our analyses, students may be younger and more skilled with the Web than employees interacting with intranets, job applicants submitting their resumes to organizations" Web sites, or customers ordering products or services via the Web. Nevertheless, we believe that our research results have implications not only for the design of educational and training sites but for job-applicant and customer sites.

We compared one simple with two global navigation systems. Because both of our global navigation systems consisted of search engines, site maps, and navigation bars or trees, we could not determine the relative importance of each feature. Future research should examine the relative importance of these navigation features to reduced disorientation and increased performance. Further, we did not study the organi- zation ofthe navigation system's content. Researchers are beginning to examine users' personal ontologies for Web navigation (e.g.. Chaffee and Gauch 2000) and this represents another promising direction for future research (Gruninger and Lee 2002).

We were intrigued by the fact that a global navigation system that adheres to usability guidelines might not be superior to a simple one. As indicated earlier, other research supports this finding (Woods 1984). This points to the need for further development of theory-based usability guidelines. Based on our smdy results, we suggest two guidelines for investigation. First, we propose that the entire hierarchy ofa site be present on every page with expandable and collapsible sections or branches ofthe hierarchy (as with navigation trees). This will act as a conceptual model to allow users to travel forward or backward to any level and thereby help to increase the accuracy of their mental models. Second, we propose that navigation features be available in fixed positions on every page (as with frames). Doing so should reduce users* cogni- tive loads. These two guidelines are exploratory and emerge from our study results. Future research needs to study their efficacy for Web design.

672 MIS Quarterly Vol. 30 No. 3/September 2006

Webster & Ahuja/Enhancing the Design of Web Navigation Systems

A strength of this study's design v̂ as the consistent content, task, and struelure ot the Web sites. Nevertheless, we focused only on fonnal searching, a relatively intensive form of extracting infonnation on a topic, because this best represents users' aetions during focused searehing on electronic com- tnerce sites (Park and Kim 2000). However, informal searehing is the most frequent seareh mode for employees using ihc Web in their jobs (Choo et al. 2000). For informal searching., we expect that the relation between users' actions and perceived disorientation will be even weaker than for fonnal searching. In support of this claim. Head et al. (2000) compared users" actions in general information retrieval (exploratory searehes) with specific information retrieval. They found that users delved more deeply into Web sites and revisited pages more often in general information retrieval. Thus, perceived disorientation should represent a useful tool for assessing Web sites for both fonnal and informal searches. Further, because the task used in this study gave users little chance to exercise control, we assessed engagement rather than flow. On the other hand, we would expect users to exhi- bit many more exploratory behaviors and to have many more chanees to exercise control during informal searching. In such cases, flow would represent a more appropriate con- struct. Thus., future studies should examine the relation between disorientation and flow during informal searching.

In summary, disorientation represents a common problem faced by Web users, and Web designs can have powerfiil effects on disorientation. With increasing user sophistication and advancing technologies, some argue that disorientation will lessen over time. Conversely, others propose that dis- orientation will remain a eritieal issue: it will become trans- formed from how to get to a location to how to retum to a location (Wen 2003). To reduce disorientation and enhance perfonnance, we have proposed two theoretically driven design guidelines, but many opportunities exist for extending this research. Future research should continue to develop and test design guidelines based on theory.

Acknowledgments

EvNet, the Network for the Evaluation of Education and Training Teehnologies. provided fmancial support for this research. We would like to thank Paul Beam for providing access to his course Web site; Saibal Ray. Tracy Chen, and Adrienne Olnick Kutzschan for their researeh assistance; and Derek Chapman. Ji-Ye Mao, David Zweig, and three anony- mous reviewers for their helpful comjnents on earlier versions of this paper.

References

Ahuja, J. S., and Webster, J. "Perceived Disorietitation: An Examination of a New Measure to Assess Web Design ElTcctive- ness," Interacting with Computers (14:1). 2001. pp. 15-29.

Arbuckle, J. L.. and Wothke, W. AMOS 4.0 User's Guide. SPSS Itic. Chicago. IL, 1999.

Atkinson. M.. and Kydd, C. 'individual Characteristics Associated With World Wide Web Use: An Empirical Study of Playfulness and Molivation." The DATA BASE for Advances in Information Systems (28:2). 1997, pp. 53-62.

Betibunan-Fich. R. "Using Protocol Analysis to Evaluate the Usability of a Commercial Web S\\c" Information & Manage- ment 09), 2001. pp. 151-163.

Bernard, M., and Hamblin. C. "Cascading versus Indexed Menu Design," U.sability News (5:1). 2003 (available online at http://psychology.wichita.edu/surl/usabilitynews/5l/menu.htm).

Bcssiere, K., Ceaparu, 1.. Lazar. J.. Robinson, J., and Shneiderman, B. "Social and Psychological Influences on Computer User Frustration," in A/ft//a^ccei-i.' Social and Psychological Dimen- sions of New Technology Use, E. Bucy and J. Newhagen (eds.), Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 2003, pp. 169-192.

Campbell. C . and Maglio, P. P. "Facilitating Navigation in Infor- mation Spaces: Road-Signs on the World Wide Web." Inter- mitional Journal of Human-Computer Studies (50), 1999, pp. 309-327.

Chaffee. J.. and Gauch. S. "Personal Ontologies for Web Naviga- tion." in Proceedings ofthe Ninth International Conference on Information and Knowledge Management, ACM Press, New York. November 6-11. 2000. pp. 227-234.

Chapman, P., Selvarajah. S.. and Webster, J. "Engagement in Multimedia Training Systems," In Proceedings of (he 32"^ Hawaii International Conference on System Science.s, R. H. Sprague, Jr. (cd.). IEEE Computer Society Press, Los Alamitos, CA, 1999 (CD-ROM).

Chen. Q., and Wells, W. D. "Attitude Toward the Site."./owmj/;)/' Advertising Research, September-October 1999, pp. 27-37.

Choo, C. W., Detlor, B., and Tumbull, D. "Working the Web: An Empirical Model of Web Use." in Proceedings ofthe 3.V'' Hawaii International Conference on System Sciences, R. Sprague (ed.), IEEE Computer Society Press, Los Alatnitos, CA, January 4-7, 2000.

Cohen, J., and Cohen. P. .AppliedMultiple Regression/Correlation Analysis for the Behavioural Sciences, Erlbaum, Hillsdale. NJ, 1983.

Conklin. J. "Hypertext: An Introduction and Survey," IEEE Computer (20:9), 1987, pp. 17-41.

Csikszentmihalyi. M. Beyond Boredom and Anxiety. Jossey-Bass. San Francisco. CA. 1975.

Cukier. W., and Middlcton. C. A. "Evaluating the Wob Presence of Voluntary Sector Organizations: An Assessment of Canadian Web Sites," IT & Society {1:3), Winter 2003, pp. 102-130.

MIS Quarterty Vol. 30 No. 3/September2006 673

Webster & Ahuja/Enhancing the Design of Web Navigation Systems

Cutmore. T. R. H.. Hine. T. J., Maberiy. K. J., Langford. N. M.. and Hawgood, G. "Cognitive and Gender Factors Influencing Navigation in a Virtual Environment," IntemationalJournal of Human-Computer Studies (53), 2000, pp. 223-249.

Danielson, D, R. "Transitional Volatility in Web Navigation," tT& Society (1:3), Winter 2003. pp. 131 -158.

Danielson, D. R. "Web Navigation and the Behavioral Effects of Constantly Visible Site Maps," Interacting with Computers (14:5). October 2002, pp. 601 -618.

Darken. R. P.,andSiberi, J. L. "A Toolset for Navigation in Virtual Environments," in Proceedings of the Sixth Annual ACM Symposium on User Interface Software and Technology^ S. E. Hudson. R. Pausch, B. V. Zanden, and J. D. Foley (eds.). ACM Press, New York, 1993. pp. 157-165.

Davis, F. D. "Perceived Usefulness, Perceived Ease of Use, and User Acceptance of information Technology," MIS Quarterly (13:3), 1989, pp. 319-339.

Davis, F. D., Bagozzi, R. P., and Warshaw. P. R. "Extrinsic and Intrinsic Motivation to Use Computers in the Workplace," Journal ofApplied Social Psychology (22), 1992. pp. 1111-1132.

Fleming, J. IVeh Navigation: Designing the User Experience., O'Reilly & Associates, Sebastopol, CA. 1998.

Furnas. G. W. "Effective View Navigation," in Readings in Information Visualization: Using Vision to Think., S. K. Card, J. D. Mackinlay. and B. Shneiderman (eds), Morgan Kaufmann Publishers, Inc., San Francisco. 1999. pp. 589-596.

Gruninger, M., and Lee. J. "Ontology: Applications ant/Design," Communications of the ACM (45:2). February 2002. pp. 39-41.

Hair. J. F., Anderson. R. E.. Tatham, R. L., and Black, W. C. Multi- variate Data Analysis with Readings (4''' ed.). Prentice-Hall. Englewood Cliffs, NJ. 1995.

Head, M., Areher, N.., and Yuan. Y. "World Wide Web Navigation Aid," International Journal of Human-Computer Studies (53), 2000, pp. 301-330.

Isakowitz, T., and Bieber, M. "Special Section: Navigation in Information-Intensive Environments," Journal of Management Information Systems (11:4). 1995, pp. 5-8.

Jackson, C. M.. Chow, S.. and Leitch, R. A. "Towards an Understanding ofthe Behavioral Intention to Use an Infomiaiion System," Decision Sciences (28:2), 1997, pp. 357-389.

Jacques. R.. Precce. J.. and Carey. J. T. "Engagement as a Design Concept for Hypermedia. Canadian Journal of Educational Communications. Spring. 1995, pp. 49-59.

Jarvenpaa, S. L., and Todd. P. A. "Consumer Reactions to Elec- tronic Shopping on the World Wide Web," Ititeniational Journal of Electronic Commerce (1:2), 1997. pp. 59-88.

Jordan. P. W. "Human Factors for Pleasure in Product Use," Applied Ergonomics (29:1), 1998, pp. 25-33.

Joreskog, K. G.. and Sorbom, D. LISREL 7: A Guide to the Program and .Applications (2"*̂ ed.), SPSS. Chicago, IL, 1989.

Kelly, H. H. "The Process of Causal Attribution," American Psychologist (2S). 1973, pp. 107-128.

Kim, J.. and Moon. J. Y. "Designing Towards Emotional Usability in Customer Interfaces: Trustworthiness of Cyber-Banking System Interfaces," Interacting with Computers (10), 1998, pp. 1-29.

Kim, J., and Yoo. B. "Toward the Optimal Link Structure of the Cyber Shopping Mall." International Journal of Human- Computer Studies (52), 2000, pp. 531-551.

Koufaris, M. "Applying the Technology Acceptance Model and Flow Theory to Online Consumer Behavior," Information Systems Research (13:2), June 2002, pp. 205-223.

Lazar, J., Bessiere, K., Ceaparu, I., Robinson, J., and Shneiderman. B. "Help! PmLost: User Frustration in Web Navigation,"/r (4 Society (1:3). Winter 2003, pp. 18-26.

Lederer, A. L., Maupin, D. J., Sena, M. P.. and Zhuang, Y. "The Role of Ease of Use, Usefulness and Attitude in the Prediction of World Wide Web Usage." in Proceedings ofthe I99H ACM SIGCPR Conference. Ritu Agarwal (ed.), ACM Press, New York, 1998, pp. 195-204.

Lightfoot, J. M. "A New Technique for Website Design Using an Interactive Visualization Hypertext Tool." Proceedings of the 1997 ACM SIGCPR Conference, ACM Press, New York, 1997, pp. 14-19.

Liu, C. L., Amett, K. P., Capeila, L. M., and Beatty. R. M. "Web Sites ofthe Fortune 500 Companies: Facing Customers Through Home Pages," Information and Management (31), 1997, pp. 335-345.

Luna. D.. Peraechio, L. A., and de Juan. M. D. "Cross-Cultural and Cognitive Aspects of Web Site Navigation," Journal of the Academy of Marketing Science OQ.A). 2002, pp. 397-410.

Lynch, P. J., and Horton, S. Web Style Guide: Basic Design Prin- ciples/or Creating Web Sites (2"'' ed.), Yale University Press, New Haven. CT. 2002 (available online at http://www. webstyleguide.com/index.html).

MacCullum. R. C , and Austin. J. T. "Applications of Stmctural Equation Modeling in Psychological Research." in Annual Review of Psychology (51), S. T. Fiske, D. L. Schacter. C. Zahn- Waxler (eds.). Annual Reviews, Palo Alto. CA, 2000, pp. 201-226.

Mallon, B., and Webb, B. "Structure, Causality, Visibility and Interaction: Propositions for Evaluating Engagemenl in Narra- tive Multimedia," International Journal of Human-Computer Studies (53), 2000, pp. 269-287.

Marchionini, G., and Shneiderman. B. "Finding Faets vs. Browsing Knowledge in Hypertext Systems." Computer (21), 1988, pp. 70-79.

Mayes. J. T. "Tlie 'M-Word': Multimedia Interfaces and Their Role in Interactive Leaming Systems," in Multimedia Interface Design in Education, A. D. N. Edwards and S. Holland (eds.), Springer-Verlag. Berlin, 1992, pp. 1-22.

McDonald. S., and Stevenson, R. J. "Effects of Text Structure and Prior Knowledge of the Learner on Navigation in Hypertext," Human Factors (40:1), 1998, pp. 18-27.

Morris, M.. and Henrichs, R. Web Page Design: A Different Multimedia. SunSoft Press/Sun Microsystems, Mountain View, CA, 1996.

Nel, D., van Niekerk, R., Berthon, J.-P., and Davies, T. "Going with the EIow: Web Sites and Customer Involvement." Internet Research: Electronic Networking Applications and Pal icy (9:2), 1999, pp. 109-116.

674 MIS Quarterly Vol. 30 No. 3/September 2006

Webster & Ahuja/Enhancing the Design of Web Navigation Systems

Netsavvy Communications. "E-Commerce Web Sites are Getting (Slowly) Better...," June 10, 2002 {available online al http:/''scllitonlheweb.coni/ezine/news0576.shtml).

Ncwfield, D., Selhi. B. S., and Ryall, K. "Scatchpad: Mechanisms for Better Navigation in Directed Web Searching," in Pro- ceedings ofthe 11"' Annual .4CM Symposium on User Interface Software and Technolog\\ ACM Press, New York., 1998, pp. I -8.

Nielsen, J. '̂ 'ls Navigation Useful?/' Jakob Nielsen's Alertbox. January 9.2000 ( http://www.useil.com/alertbox/20000109.html).

Nielsen, J. "Top Ten Mistakes in Web Design," Jakob Nielsen's Alertbox, 1996a (http://www.useit.com/ alertbox/9605.html).

Nielsen, J. "The Top Ten New Mistakes of Web Design," Jakob Nielsen's Alertbox. May 30, 1999a (http://www,useit.com/ alertbox/990530.himl).

Nielsen, J. "'Top Ten Mistakes" Revisited Three Years Later," Jakob Nielsen's Alertbox. May 2. 1999b (http://www.useit.conV alertbox/990502.htmt).

Nielsen. J. "'Why Frames Suek (Most of the Time)." Jakob Nielsen's Alertbox, December 1996b (http://www.useit.cotti/ alert box/9612.btml).

Nielsen, J., and Sano, D. "SunWeb: User Interface Design for Sun Microsystem's Internal Web," in Proceedings of the Second World Wide Web Conference: Mosaic and the Weh, Chicago, IL, October 17-20, 1994, pp. 547-557 (available a( hltp:// www. useit .eom/papers/sunweb/).

Nielsen, J., and Wagner. A. "User Interface Design for the World Wide Web." Proceedings ofthe CHI 96, ACM Press, New York, 1996, p. 330.

Norman, D. The Psychology of Everyday Things, Harper & Row, New York, 1988.

Novak, T. P., Hoffman, D. L,, and Yung, Y.-F. "Measuring the Customer Experience in Online Environments: A Structural Modeling Approach," Marketing Science (19:1), 2000, pp. 22-42.

Oinas-Kukkonen, H. "Balaneing the Vendor and Consumer Requirements for eleetronie Shopping Systems," Information Technology and Management (1), 2000, pp, 73-84.

OInick Kutzschan. A., and Webster. J. "HCI as MIS," in Human- Computer Interaction in Management Information Systems: Foundations. Advances in Management Information Systems. ?. Zhang and D. Galletta (eds.), M. B. Sharpe. Inc., Armonk, New York, 2006, pp. 32-47.

Otter, M., and Johnson, H. "Lost in Hyperspace: Metrics and Mental Models," Interacting with Computers (13), 2000, pp. 1-40.

Palmer. J, W, "Web Site Usability, Design, and Performanee Metrics." Information Systems Research (13:2), June 2002, pp. 151-167.

Park, J.. and Kim, J. "Contextual Navigation Aids for Two World Wide WebSystems." InteniationalJournal of Human-Computer Interaction (12), 2000, pp. 193-21 7.

Pereira, R. E. "Optimizing Human-Computer Interaction for the Electronic Commeree Environment," Journal of Electronic Commerce Research (1:1), 2000, pp. 23-44 (available at http:// www.esulb.edu/joumals/jecr/p i.htm).

Pitkow. J., and Kehoe, C. "Emerging Trends in the WWW User Population." Communications of the ACM (39:6), 1996, pp. 106-108.

Pitkow, J., Kehoe, C . and Rogers. J. "GVU's WWW User Survey," Georgia Tech Research. Atlanta, GA, 1998 (available online at bttp://v^Tvw.g\ai.gatech.edu/user_survcys/sLirvey-1998-04/).

Russell, D. W., Kahn, J. H.. Spoth. R.. and Altmaier, E. M. "Analyzing Data from Experimental Studies: A Latent Variable Structural Equation Modeling Approach," JoHrotf/o/Coume/mg P.sychologx-[A5). 1998, pp. 18-29.

Sandelands, L. E. "Effects of Work and Play Signals on Task E\&\uaX\or\"JournalofAppliedSociatPsycholog\(\%:\2), 1988, pp. 1032-1048.

Sandelands. L. E., and Buekner. G. C- "Of Art and Work: Aesthetic Experience and the Psychology of Work Feelings," in Research in Organizational Behaviour, L. L. Cummings and B.M. Staw(eds.). JAI Press, Greenwich, CT, 1989, pp. 105-131.

Santhanam, R., and Sein, M. K. "improving End-User Proficiency: EtTeets of Conceptual Training and Nature of Inicraetion," Information Systems Research (5), 1994, pp. 378-399.

Smith. P. A. "Towards a Practical Measure of Hypertext Usability," Interacting with Computers (4), 1996, pp. 365-381.

Spence, R. "A Framework for Navigation," International Journal of Human-Compuler Studies (51), 1999, pp. 919-945.

Spiller, P., and Lobse, G. L. "A Cla.ssification of Internet Retail Stores," International Journal of Electronic Commerce (2:2), 1998, pp. 29-56.

Spool, J. M., Scanlon, T., Schroeder, W., Snyder, C , and DeAngeto, T. Web Site Usability: /4 Designer'i Gu/iie, Morgan Kaufmann Publishers, Inc., San Francisco. CA, 1999.

Thomdyke, P. W. "Distance Estimation from Cognitive Maps," Cognitive Psychology {U), 1981, pp. 526-550.

Thomdyke, P. W., and Hayes-Roth, D. "Differences in Spatial Knowledge Acquired from Maps and Navigation." Cognitive P.sychology (14), 1982, pp. 560-589.

Thuring, M., Hannemann, J., and Haake. J. M. "Hypermedia and Cognition: Designing for Comprehension," Communications of the ACM(iS:^). 1995, pp. 57-66,

Tognazzine. B. "First Principles of Interaction Design," 1998 (available online at http://www.asktog.eom/basies/ firstPrineiples.html).

Trevino. L. K.. and Webster. J. "Flow in Computer-Mediated Communication," Communication Research (19), 1992, pp. 539-573,

Trumbo, J. "Spatial Design and Memory: A Conceptual Approach to tbe Creation of Navigable Space in Multimedia Design," Interactions (5), July-August 1998, pp. 26-34.

UsabilityNet. "Cost Justifying Usability," 2003 (available online at http://www.usabilitynet,org/management/c cost,htm),

Utting, K,. and Yankelovicb, N. "Content and Orientation in Hypermedia Networks," ACM Transactions on Information Systems (7:1), 1989, pp, 58-84.

Watts-Perotti, J.. and Woods. D. D. "How Experienced Users Avoid Getting Lost in Large Display Networksm," International Journal of Human-Computer Interaction (11:4), 1999, pp. 269-299.

M S Quarterly Vol. 30 No. 3/September2006 675

Webster & Ahuja/Enhancing the Design of Web Navigation Systems

Webster. J.. and Ho. H. "Audience Engagement in Multimedia Presentations," The Data Base for Advances in Information Systems (28:2), 1997, pp. 63-77.

Webster. J., and Martocchio. J. J. "The Differential Effects of Software Training Previews on Training Outeomes," Journal of Manugemeni (21:4), 1995, pp. 757-787.

Webster. J., and Martocchio, J. J. "Microcomputer Playfulness: Development of a Measure with Workplace Implications," MIS Quarterly (16:2). 1992. pp. 201 -226.

Webster, J., Trevino. L. K., and Ryan, L. "The Dimensionality and CorrelatesofFlow in Human-Computer Interactions." Com/w^er^ in Human Behavior C^). 1993, pp. 4i 1-426.

Wen, J. "Posl-Valued Reeall Web Pages: User Disorientation Hits the Big Time." IT & Society (1:3), 2003. pp. 184-194.

Wiekens, C. D. Engineering Psychology and Human Performance (2"-̂ ed.), Harper Collins Puhlishers, New York, 1992.

Woods. D, D. "Visual Momentum: A Concept to Improve the Cognitive Coupling of Person and Computer," Internationa! Journal of Man-Machine Studies (21), i 984, pp. 229-244.

Woods. D. D.. Roth. E. M.. Stuhler. W. F.. and Mumaw. R. J. "Navigating Through Large Display Networks in Dynainie Control Applications," Proceedings of the 34"' Annual Meeting of the Human Factors Society. Santa Monica, CA, 1990, pp. 396-399.

Woods. D. D.. and Watts, J. C. "How Not to Have to Navigate Through Too Many Displays," in Handbook of Human-Computer Interaction. M. Helander. T. K. Landauer and P. Prabhu (eds.). Elsevier. Amsterdam, 1997. pp. 617-650.

Yoo, Y., and Alavi, M. "Media and Group Cohesion: Relative Influences on Social Presence, Task Participation, and Group Consensus," MIS Quarterly (25:3), 2001, pp. 371-390.

Yu. B-M., and Roh, S-Z. "The Effects of Menu Design on Infonnalion-Seeking Performanee and User's Altitude on the World Wide Web." Journal of the American Society for Infor- mation Science and Technology (53:11), 2002, pp. 923-933.

Zona Research Ine. "1998 Holiday Shopping: Is Web-Based Buying Reaching Critieal Mass?," 1998 (htlp://www. binteractive.com/press/archive/release68.asp).

About the Authors

Jane Webster received her Ph.D. from New York University and is the E. Marie Shantz Professor of MIS in the Sehool of Business at Queen's University in Canada. She has served as a senior editor for MIS Quarterly and guest associate editor of Information Systems Research. She currently serves on the advisory board for .4/5- SIGCHI and is the viee president of Publieations for AIS. She has published in a variety of journals including the Academy of Man- agement Journal. Communication Research, Journal of Organiza- tional Behavior, MIS Quarterly, and Organization Science. Her research investigates the impacts of technologies in the support of distributed work, organizational communication, employee recruit- ment and selection, employee monitoring, training and learning, and human-computer interaetion issues.

Jaspreet Ahuja is the director of Research with Electrolinks, a pioneer in Broadband Over Powerlines (BPL). He also is a senior consultant and manages telecom projects for companies like Bell Canada. The present research is based on his Master's thesis, completed at the University of Waterloo in Canada.

676 MIS Quarterly Vol. 30 No. 3/September 2006

Webster & Ahuja/Enhancing the Design of Web Navigation Systems

Appendix A

Measures

Navigation Systems, the control condition (ihc simple navigatioti system), condition #1 (the basic global navigation system), and condition #2 (the enhanced global navigation system), were compared. They were coded into two dummy variables (Cl and C2) for analyses. That is, the control condition (the simple navigation system) was represented as 0 on both C1 and C2, condiiion #1 (the basie global navigation system) a.s 1 on Cl and 0 on C2, and condition #2 (the enhanced global navigation system) as 0 on Cl and I on C2. The use ofdummy coding of experimental conditions for SEM analyses is recommended when investigating direct rather than tnoderating effects of experimental conditions (Jiireskog and Sorbom 1989; MacCallum and Austin 2000; Russell et al. 1998) and is eonsistent with others' use ofdummy coding to represent experimental groups in SEM analyses (e.g.. Webster and Martocchio 1995; Yoo and Alavi 2001).

Perceived Disorientation was captured with Ahuja and Webster's (2001) seven-item perceived disorientation measure on seven-point scales ranging from "never" to "always" (evidence ofthe measure's construct validity is reported in Ahuja and Webster). The items were

While I was browsing: I felt lost I felt like I was going around in eircles It was difficult to fmd a page that I had previously viewed Navigating between pages was a problem 1 didn'l know how to get to my desired location ! felt disoriented After browsing for a while I had no idea where lo go next

Engagement was captured with Webster and Ho's (1997) seven-item engagement measure on seven-point seales ranging from "strongly disagree" to "strongly agree." The items were

The Web site: Kept me totally absorbed in the browsing Held my attention Excited my curiosity Aroused my imagination Was fun Was intrinsieally interesting Was engagitig

User Performance was determined by two measures: the time taken to answer the questions (with a lower time representing higher perfonnance) and the nutnber of questions answered correctly during the formal seareh. The time taken to answer the questions was determined by the baek-end program and was reverse-coded for analyses.

To determine the number of correet answers, participants were presented with a blue sheet during browsing which asked them to fmd the answers to eight questions. These questions were pretested in the pilot experiments, and answers to these questions were located in different sections and levels ofthe Web site. There were three types of questions.

Simple fact questions: When is Assignment #4 due? Who is the technical contact person? Which is the name of the first assignment that is due?

Judgment questions: From the software available for download, which one do you think is the best to eapture screenshots? Which editor would be best to use for this course (to edit SGML documents)? What is the best way to print modules ofthe course?

Comparison of facts questions: If you wanted to know what the instruetor has asked you to do, which is the best place to look; the Newsgroup or Instructor's comments? Which is the lowest Browser that is supported by this course; Netscape 3 or Netscape 4?

MIS Quarterly Vol. 30 No. 3/September2006 677

Webster & Ahuja/Enhancing the Design of Web Navigation Systems

Future Intentions to Use. Six items from Jackson et al. (1997) captured intentions on seven-point scales ranging from "strongly disagree" to "strongly agree." The items were

For leaming Technical Writing Using this web site would be a good idea I would intend to use this web site Using this web site would be a foolish idea 1 would like the idea of using this web site Using this web site would be unpleasant I would intend to use this web site very frequently

Control Variables. In addition to the variables included in the model, we incorporated 10 control variables that past researeh suggests could relate lo the outcomes of interest. More specifically, we e.xamined the Web skill variables (Koufaris 2002) of Intemet experienee and Web design experience, attributions for pcrfonnanee (Kelly 1973), computer playfulness (Webster and Martocchio 1992), ease of use (Davis 1989), and demographic variables (Kouiaris 2002) of age, gender, years of education in the English language, level of education, and subject of education. We compared these variables across the three conditions and only Intemet experience (a one-item measure of "How long have you been using the Intemet?" with three response choices, i-6 months, 7-12 months, and more than 12 months) differed; henee it was the only control variable included in the analysis.

Appendix B

Correlations Between Measures'

1. Navigation Systems: CI

C2

2. Perceived Disorientation

3. Engagement

4. User Performance: Time (reverse-coded)

Correct Answers

5. Future Intentions to Use

6. Internet Experience

1

(N/A)

-.50***

.09

.03

-.27*"

-.20"

-.05

-.00

(N/A)

-.26"-

.04

.24**'

.21**

.03

.17'

2

(.90)

-.45*"

-.19"

-.27*"

-.54**-

-.01

3

(.92)

.07

. 2 1 "

.67"*

-.07

4

(N/A)

. 2 6 " '

.24-"

.17-

(N/A)

.20"

. 3 1 " *

5

(.90)

-.05

6

(N/A)

^Internal-consistency reliabilities on the diagonal.

*p < .05 ' - p < .01 *--p < .001

678 MIS Quarterly Vol. 30 No. 3/September 2006