Complete Research Paper from attached feedback
Wang & Benbasat/Interactive Decision Aids
RESEARCH ARTICLE
INTERACTIVE DECISION AIDS FOR CONSUMER DECISION MAKING IN E-COMMERCE: THE INFLUENCE OF PERCEIVED STRATEGY RESTRICTIVENESS1
By: Weiquan Wang Department of Information Systems Faculty of Business City University of Hong Kong Tat Chee Avenue, Kowloon HONG KONG SAR weiquan@cityu.edu.hk
Izak Benbasat Sauder School of Business University of British Columbia 2053 Main Mall Vancouver, BC V6T 1Z2 CANADA benbasat@sauder.ubc.ca
Abstract
This paper extends the effort–accuracy framework of cogni- tion by taking into account the perceived strategy restric- tiveness of decision aids, and tests the extended framework in a context in which online decision aids are used to elicit consumers’ preferences, automate the processing of the pre- ferences, and provide product advice for consumers. Three types of decision aids with different decision strategy support capabilities (an additive-compensatory based aid, an elimina- tion-based aid, and a hybrid aid supporting both strategies)
1Bernard Tan was the accepting senior editor for this paper. Cheri Speier served as the associate editor.
are compared in terms of users’ perceptions of strategy restrictiveness, advice quality, and cognitive effort. These comparisons are grounded on the properties of normativeness and complementarity of decision strategies employed by the aids. A normative strategy takes into account both the users’ attribute preferences and the relative importance of such preferences, and allows for trade-offs among preferences (e.g., additive–compensatory). Strategy complementarity indicates support for decision rules based on multiple strategies (e.g., both additive–compensatory and elimination strategies).
The experimental results support the validity of the extended effort–accuracy–restrictiveness framework and the effects of strategy normativeness, but not the effects of strategy com- plementarity. In addition to the perceptions of cognitive effort and advice quality, perceived strategy restrictiveness exerts a significant influence on consumers’ intentions to use online decision aids. The additive–compensatory aid is perceived to be less restrictive, of higher quality, and less effortful than the elimination aid, whereas the hybrid aid is not perceived to be any different from the additive–compensatory aid.
Keywords: Decision aid, decision strategy, restrictiveness, cognitive effort, advice quality, strategy normativeness, strategy complementarity, explanation
Introduction
With the rapid advancement of Internet and Web-based appli- cations, interactive online decision aids are increasingly being made available to facilitate consumers’ online decision
MIS Quarterly Vol. 33 No.2, pp. 293-320/June 2009 293
Wang & Benbasat/Interactive Decision Aids
making (Grenci and Todd 2002; Häubl and Trifts 2000; Leavitt 2006; Wang and Benbasat 2005; Xiao and Benbasat 2007). These aids elicit users’ preferences, carry out a set of search and evaluation operations on behalf of users, and provide product recommendations (Maes et al. 1999). Much behavioral research on online decision aids has relied on the effort–accuracy framework of cognition to investigate the beneficial impact of decision aids on reducing the cognitive effort expended by users and increasing their decision quality (accuracy) (Häubl and Trifts 2000; Hostler et al. 2005). Nevertheless, the use of online decision aids has its limita- tions. They usually employ pre-embedded decision rules to provide product recommendations, and as a consequence restrict users to only those functionalities and decision processes that are supported by the aids. When users realize that their desired decision processes or strategies are not supported, they are likely to perceive the decision aid to be restrictive (Silver 1988, 1990, 1991a, 1991b).
This study focuses on the influence of such restrictiveness on user perceptions concerning the utilization and adoption of decision aids. A highly restrictive aid can invoke psycholo- gical reactance (Brehm 1966; Brehm and Brehm 1981), which results in two possible choices for the user. The first is not to use the aid at all if it is deemed to be too constraining (Bennet 1983; Silver 1990), but then the user has to deal with an arduous and difficult choice problem with a likely inferior outcome. Of course, the user may switch to other websites hoping for better support, especially given the low switching cost in online environments (Rust and Kannan 2003). The second choice is to use the restrictive aid that does not support the preferred decision processes of the user. In this case, the user may expect that the advice given by the aid will poorly match her preferences, hence will less likely accept the advice or to use the aid again in the future (Komiak and Benbasat 2006).
Despite the evident importance of such concerns, few studies have examined the role of users’ perceptions of the restric- tiveness of decision aids. Therefore, the key objective of this study is to extend the effort–accuracy framework by taking into account the perceived strategy restrictiveness of decision aids. Perceived strategy restrictiveness is defined as decision makers’ perceptions of the extent to which their preferred decision processes are constrained by the functionalities and support provided by a decision aid (Silver 1988). The central premise of this study is that when users’ preferred decision processes are not supported by a certain decision aid, they will perceive it to be restrictive and will less likely use it as compared to an aid that supports their preferred decision processes.
An examination of behavioral decision literature shows that two properties of decision strategy may affect the restric- tiveness of decision aids: strategy normativeness and strategy complementarity (Elrod et al. 2004; Song et al. 2007; Todd and Benbasat 1999, 2000). These two properties are iden- tified based on the fact that decision aids either employ a single strategy (that could be more or less normative) or multiple strategies that can complement each other. These two properties provide a theoretical grounding for empirical comparisons of different decision aids in this research and have theoretical and practical implications for the design of decision aids.
First, many extant online decision aids support an elimination strategy that does away with product alternatives with unacceptable attribute levels as specified by the users. However, such aids do not allow lower valued product attri- butes to be compensated by the higher valued ones. Conse- quently good alternatives risk to be prematurely eliminated, and the quality of the advice may be unsatisfactory (Ansari et al. 2000; Todd and Benbasat 2000). Therefore, the elimina- tion strategy is considered as “less” normative, and decision aids that use such a strategy may restrict users to decision processes that they do not prefer. This issue may account for negative reactions of online consumers toward using such decision aids (Burke 2002).
In contrast, a “more” normative strategy (e.g., additive– compensatory, abbreviated AC) takes into account the relative importance of a user’s attribute preferences and allows for trade-offs among these preferences, fully using all of the information on available alternatives in making choices (Song et al. 2007; Todd and Benbasat 2000). Because a key design objective of decision aids is to encourage users to make use of normative strategies (e.g., Baron 2004; Todd and Benbasat 1999, 2000), strategy normativeness is examined in this study.
Second, decision aids can alternatively employ a hybrid approach that supports both elimination and AC strategies, thus not restricting users to employ one strategy exclusively (Elrod et al. 2004). The elimination (non-compensatory) and AC (compensatory) strategies can complement each other and offer more decision support for users. However, to the best of our knowledge, no previous studies have empirically investigated these alternative designs.
To examine the role of strategy normativeness and comple- mentarity in inducing users’ perceptual differences, a labora- tory experiment is conducted to compare three decision aids. These three aids represent the normative and less normative ends of the strategy continuum (AC-based and elimination- based aids, respectively), and their combination (a hybrid
294 MIS Quarterly Vol. 33 No. 2/June 2009
Wang & Benbasat/Interactive Decision Aids
aid). Comparisons are conducted within an extended effort– accuracy framework that includes user perceptions of strategy restrictiveness, cognitive effort, and advice quality. In doing so, this study contributes to Information Systems research by advancing the behavioral decision theory with the extended framework in the context of online choice making supported by decision aids.
Moreover, the extent to which the objective capabilities of a decision aid influence a user’s perceptions may be a function of (1) the user’s understanding of the underlying reasoning used by the aid to generate product recommendations and (2) the user’s ability to fully utilize the functions provided by the aid (Gregor and Benbasat 1999; Silver 1991a). According to the theory of cognitive learning, explanation facilities can enhance users’ understanding of the inner workings of a decision aid and thus help the users to effectively utilize its capabilities (Brunswik 1952; Dhaliwal and Benbasat 1996; Wang and Benbasat 2007). Therefore, we also examine the role of the explanation facilities provided by decision aids in the user evaluations of the three aids. The empirical findings of this study will provide guidelines for practitioners on how to better design online decision aids with explanation facilities.
The remainder of this paper is organized as follows. The next section introduces the theoretical background of this study, followed by a section that develops the hypotheses to be tested. The research method is then described, and the results of the experiment are reported. A discussion of the findings, the implications for theory and practice, the limitations of the study, and suggestions for future research are provided, followed by the conculsion.
Theoretical Background
We first introduce the two key components of decision aids examined in this study, namely, decision strategy and explanation facilities, and review the theoretical and empirical work related to these two components. We then briefly review the cognitive effort perspective and the theory of strategy restrictiveness which together provide a theoretical grounding for our investigation into the impacts of these two components of decision aids on user perceptions and the use of the aids.
Decision Strategy and Explanation Facilities
Interactive decision aids help consumers in making informed purchase decisions amidst the vast availability of online
product offerings (Maes et al. 1999; Xiao and Benbasat 2007). In essence, such decision aids are utilized to help users make choices in the context of a multi-alternative, multi-attribute preferential choice task (Keeney and Raiffa 1976). To integrate different product attribute preferences and generate recommendations for users, decision aids apply certain decision strategies. The decision strategy(ies) em- ployed by a decision aid determines the quality of its recom- mendations and the decision processes it supports (e.g., Todd and Benbasat 1999). Therefore, decision strategies are regarded as an integral part of decision aids.
Users’ evaluations of a decision aid are based on their cogni- tive and behavioral assessments of the aid (Silver 1991a; Wang and Benbasat 2007; Ye and Johnson 1995). Cognitive assessment is based on users’ understanding and mental model of the aid, whereas behavioral assessment refers to their evaluations of the support features of the aid based on the usage of the features. When users cannot fully understand the inner workings of a decision aid or cannot use it properly, they may not be able to correctly identify and perceive the objective properties of the aid. An important IT-based com- ponent that can be included in a decision aid to enhance such user understanding and proper use is explanation facilities (Gregor and Benbasat 1999; Wang and Benbasat 2007). Other factors, such as users’ familiarity with the aid, do not involve an IT design aspect and are therefore not investigated in this study (see Figure 1). What follows is a brief review of these two critical components of online decision aids, namely, decision strategy and explanation facilities.
Decision Strategy
Svenson (1979) describes 12 strategies that are applicable to preferential choice problems. Among them, the additive– compensatory (AC) and elimination models (elimination-by- aspect and conjunctive strategies2 are the two most commonly
2 Both elimination by aspect (EBA) and conjunctive (CNJ) models compare
attribute levels against certain thresholds. However, the evaluation process of CNJ is by alternative: if an alternative violates a specified threshold for any of the attributes, then it is dropped from further consideration. In contrast, the evaluation process of CNJ is by attribute. The values of an attri- bute are examined across alternatives, and any alternative that does not meet the threshold for the attribute is eliminated. All the remaining alternatives are evaluated against the next attribute. Detailed explanations are provided in Todd and Benbasat (1992).
In the context of online decision aids, the EBA does not differ from the CNJ from the user’s point of view. Regardless of whether the evaluation process is by attribute or by alternatives, the user provides the thresholds of different attributes, either cumulatively or simultaneously, and the decision aid recommends alternatives that satisfy these thresholds. We, therefore, do not differentiate them in the present study.
MIS Quarterly Vol. 33 No. 2/June 2009 295
Wang & Benbasat/Interactive Decision Aids
Figure 1. User Perceptions and Evaluations of Decision Aids
studied strategies (e.g., Jarvenpaa 1989; Payne 1976; Todd and Benbasat 1999, 2000), and are applicable to interactive decision aids (e.g., Pereira 2000; Tan 2003).
AC evaluates each alternative based on all of its relevant attributes. Each attribute is assigned a weight by the user, which indicates its importance. A score for each alternative is then determined by adding up the products of each attribute’s transformed (e.g., normalized) value and weight. Once the computations are complete for all of the product alternatives, the user is provided with a list of recommended alternatives arranged by the weighted total score.
In contrast, the elimination strategy evaluates each alternative along various attributes, and any alternative that violates any threshold value specified by the user for an attribute is eliminated. Unlike with the AC strategy, users’ attribute preferences are not comprehensively processed (e.g., lower valued attributes are not compensated by higher valued attri- butes). The AC and elimination strategies, therefore, repre- sent the normative and less normative ends of the strategy continuum, respectively (e.g., Keeney and Raiffa 1976; Payne et al. 1993; Todd and Benbasat 2000).
Explanation Facilities
Wang and Benbasat (2007) investigated the explanation facilities for online decision aids and found three types of explanations deemed effective in transferring knowledge to users about decision aids and their use. The three are how explanations, why explanations, and guidance. The how and why explanations deliver knowledge about the system’s actions (e.g., how it works and why its actions are appro-
priate) and assist users in creating a valid mental model of the aid (McSherry 2005; Ye and Johnson 1995). How explana- tions reveal the logical processes and the line of reasoning used by the decision aid. Why explanations justify the importance and purpose of the questions asked by the aid in order to elicit users’ preferences, as well as the basis for the product recommendations provided.
Guidance, which is also referred to as trade-off explanations by Wang and Benbasat (2007), provide information about the various options provided by the aid for users to indicate their attribute preferences. Guidance is not only about the mech- anics of choosing and adjusting users’ preferences, but it also helps users to arrive at and express their actual preferences when using the aid (Barkhi 2001-2002; Mahoney et al. 2003; Wilson and Zigurs 1999). Due to the fact that the use of online decision aids is voluntary, guidance is often incor- porated into explanation facilities so that users can better understand the features of the aids and thus utilize them effectively.
Further details of the provision of these explanations in decision aids are provided by Wang and Benbasat (2007). Examples of the three types of explanations that were embedded in the experimental decision aids in this study are provided in Appendix A.
The Cognitive Effort Perspective and Perceived Strategy Restrictiveness
According to the effort–accuracy framework of cognition pro- posed by Payne (1982), the primary objectives of a decision
User Perceptions/ Evaluations
Understand the capabilities and use
them properly
* Explanation Facilities * Familiarity * ……
Cognitive and behavioral assessmentsDecision Aids
User Perceptions/ Evaluations
Understand the capabilities and use
them properly
* Explanation Facilities * Familiarity * ……
* Explanation Facilities * Familiarity * ……
Cognitive and behavioral assessmentsDecision Aids
296 MIS Quarterly Vol. 33 No. 2/June 2009
Wang & Benbasat/Interactive Decision Aids
maker are to maximize accuracy (decision quality) and minimize cognitive effort. As these objectives often conflict, trade-offs are made between the two. Todd and Benbasat have conducted a series of studies investigating the strategy selection and choice behavior of decision makers when they are assisted by decision support systems (DSS) (Todd and Benbasat 1991, 1992, 1994a, 1994b, 1999, 2000). The key findings of these studies state that decision makers tend to adapt their strategy selection to the type of aid available in such a way as to maintain a low overall expenditure of effort. Moreover, if a DSS provides support to make a more accurate strategy at least as easy to employ as a simpler but less accurate strategy, then the use of the DSS induces behavior that is consistent with the implementation of the more accurate strategy.
In line with previous studies on individuals’ choice behavior with the use of DSS, the recent literature on decision aids examines their impact on users’ decision quality and cognitive effort. However, when using decision aids in online environ- ments, the processing of user preferences is automated by the aids, leading to effort savings for the user. Moreover, the user’s decision quality is largely influenced by the choice strategy(ies) employed by the aid to generate product advice. Therefore, the focus has shifted from users’ trade-offs between effort and decision quality in employing different strategies to the designs of decision aids that not only save users’ effort but also improve their decision quality (Häubl and Trifts 2000; Hostler et al. 2005; Maes 1994).
Despite the aforementioned benefits of using a decision aid, relying on it restricts users to the decision processes that are embedded in the aid (Silver 1990). This is because a decision aid normally employs a particular predefined decision stra- tegy, despite the fact that people are known to utilize and prefer a variety of strategies (Svenson 1979). This concept of system restrictiveness was first proposed by Silver (1988) as an important consideration in utilizing DSS. Silver (1988, 1991b) suggests that the perceived restrictiveness of a decision aid varies between users, and therefore, what matters in users’ evaluations of the aid is its subjective restrictiveness. In this study, we therefore compare users’ perceptions of the strategy restrictiveness of different decision aids and test the effects of such perceptions on users’ behavioral intentions toward the aids.
Hypotheses Development
The primary objectives of this study are (1) to extend the effort–accuracy framework by taking into account the per-
ceived strategy restrictiveness of decision aids, and (2) to compare the elimination aid, the AC aid, and the hybrid aid in terms of perceived strategy restrictiveness, cognitive effort, and advice quality. Figure 2 shows the research model; the hypotheses are developed below.
The Structural Model
Numerous studies have confirmed that effort and quality are the two important factors influencing users’ choice behavior and their intentions to use decision aids (e.g., Payne 1982; Todd and Benbasat 1999). Moreover, when a decision aid is perceived to be restrictive in terms of choice strategy, users may feel that the aid does not represent their true preferences and thereby become less disposed to using it (Komiak and Benbasat 2006). According to the reactance theory, when a user is constrained to the decision processes supported by an aid and her desired support functionalities are not offered by the aid, then psychological reactance will be aroused, reducing her intentions to use it again (Brehm 1966; Brehm and Brehm 1981; Clee and Wicklund 1980).
The reactance theory was first proposed in a generic context of personal freedom and choices (Brehm 1966). The basic premise of reactance is that people possess certain behavioral and cognitive freedoms. If such freedoms are threatened, people tend to maintain their freedoms to think and act by regaining control over their behaviors and avoiding the constrained behaviors. Applying the reactance theory to the context of decision aids, we contend that users want to use the aids to facilitate their decision making rather than be restricted by the decision processes that they do not prefer to apply. Psychological reactance that is aroused by a restrictive aid will lead to behavioral avoidance, reducing the user’s intentions to use the aid (Edwards et al. 2002). Thus, together with the cognitive effort and advice quality considerations, we hypothesize the following:
H1(a): Greater perceived strategy restrictiveness of a decision aid will negatively influence users’ intentions to use the aid.
H1(b): Greater perceived advice quality of a deci- sion aid will positively influence users’ intentions to use the aid.
H1(c): Greater perceived cognitive effort of using a decision aid will negatively influence users’ intentions to use the aid.
To systematically investigate the effects of the aforemen- tioned three antecedents on users’ intentions to use decision
MIS Quarterly Vol. 33 No. 2/June 2009 297
Wang & Benbasat/Interactive Decision Aids
Figure 2. Research Model
aids, we also examine the mediated effect of an antecedent by other antecedents and posit the indirect effects of these ante- cedents (Baron and Kenny 1986). First, perceived strategy restrictiveness will influence the two other antecedents, thereby exerting indirect effects on intentions to use the aid. When users perceive a decision aid to be restrictive, their psychological reactance will result in a tendency to disagree with the quality of advice given by the aid (Silvia 2006). Therefore, greater perceived restrictiveness will lower the perceptions of advice quality which in turn will influence users’ intentions to use the aid. As such, perceived advice quality will negatively mediate the effects of perceived strategy restrictiveness on users’ intentions to use the aid. Moreover, such an indirect effect of perceived strategy restrictiveness is posited in addition to its direct effect on intentions to use an aid (see H1(a)). Therefore, the effect of perceived strategy restrictiveness on intentions to use the aid will be partially mediated by perceived advice quality.
H2: Perceived advice quality of a decision aid will partially negatively mediate the effects of perceived strategy restrictiveness on intentions to use the aid.
The reactance theory also suggests that users may exercise additional effort to restore their freedom if they feel restricted by the aid (Brehm 1966; Brehm and Brehm 1981). For example, users may perform multiple iterations by adjusting attribute thresholds when given a restrictive aid, which means that they expend more effort in utilizing the aid. Accordingly, perceived strategy restrictiveness will exert an indirect effect on intentions to use a decision aid by positively affecting perceived cognitive effort, in addition to its direct effect. As such, perceived cognitive effort will partially positively mediate the effects of perceived strategy restrictiveness on intentions to use the aid.
H3: Perceived cognitive effort of using a deci- sion aid will partially positively mediate the effects of perceived strategy restrictive- ness on intentions to use the aid.
Second, perceived advice quality will also exert an indirect influence on the users’ intentions to use a decision aid through perceived cognitive effort. When the quality of the aid is perceived to be low, users may want to refine their inputs (e.g., attribute preferences) to the aid, expecting that the advice from the aid may be improved, or they may need to examine more closely a larger number of recommended products to find a suitable alternative (Fasolo et al. 2005). Such activities require additional cognitive effort, thereby reducing users’ intentions to use the decision aid. With these, the following hypothesis is posited:
H4: Perceived cognitive effort of using a deci- sion aid will partially negatively mediate the effects of perceived advice quality on intentions to use the aid.
Comparisons of Decision Aids with Different Decision Strategy Support Capabilities
To investigate how users perceive decision aids in terms of strategy restrictiveness, cognitive effort, and advice quality, three types of decision aids with different decision strategy support capabilities are examined. Because the AC and elimi- nation strategies are the typical strategies used in online decision aids (e.g., Pereira 2000), one decision aid is AC- based and the other is elimination-based (hereafter referred to as the AC aid and the elimination aid, respectively). The third type of decision aid supports both AC and elimination stra-
Perceived Cognitive Effort
Perceived Restrictiveness
Intention to Use Decision Aids
Perceived Advice Quality
Explanation Facilities
Decision Strategy
H1(a)
H1(b)
H1(c)
H2
H4
H3
H5
H7
H8(b) H8(a)
H6
Perceived Cognitive Effort
Perceived Restrictiveness
Intention to Use Decision Aids
Perceived Advice Quality
Explanation Facilities
Decision Strategy
H1(a)
H1(b)
H1(c)
H2
H4
H3
H5
H7
H8(b) H8(a)
H6
298 MIS Quarterly Vol. 33 No. 2/June 2009
Wang & Benbasat/Interactive Decision Aids
tegies, and is labeled a hybrid aid. The hybrid aid allows users to choose their preferred decision strategies to process different attribute preferences, rather than being restricted to adopting a single predetermined strategy as with the AC and elimination aids (Elrod et al. 2004).
The three decision aids described above differ according to their strategy normativeness and complementarity. Norma- tiveness differentiates the AC and hybrid aids from the elimination aid, whereas complementarity differentiates the hybrid aid from the AC and elimination aids (see Table 1). As the AC and hybrid aids both support a normative strategy (i.e., AC), the elimination aid differs from them due to its nonnormativeness (e.g., Keeney and Raiffa 1976; Payne et al. 1993; Todd and Benbasat 2000). The hybrid aid is different from the AC and elimination aids because it supports two strategies (i.e., AC and elimination) that complement each other.
To understand the underlying mechanisms of the differences among the three aids, two comparisons are deemed meaning- ful. The first is a comparison of the AC aid and the elimi- nation aid, and the second is that of the AC aid and the hybrid aid. The former comparison will demonstrate the differences due to strategy normativeness, whereas the latter will exhibit the differences due to strategy complementarity. A com- parison between the elimination aid and the hybrid aid is not introduced because it would involve differences due to both strategy normativeness and strategy complementarity.
Perceived Strategy Restrictiveness of Decision Aids
We first compare the AC aid and the elimination aid. The AC aid processes users’ requirements in a more comprehensive manner than the elimination aid. In addition to eliciting users’ attribute preferences, the AC aid takes into account the weightings of such preferences in generating recommen- dations. Such support allows users to make trade-offs among their attribute preferences (Bettman et al. 1998; Keeney and Raiffa 1976; Payne et al. 1992; Song et al. 2007), which are necessary for most individuals because “hardly ever is one lucky enough to find an alternative that is superior in every way to other alternatives” (Goldstein et al. 2001, p. 175). According to Silver’s theory of system restrictiveness, users’ perceptions of restrictiveness are determined by the extent to which they perceive that their preferred decision processes are proscribed. The inability to express the relative impor- tance of different preferences may lead most users to perceive the elimination aid to be more restrictive than the AC aid. This leads us to the following hypothesis:
H5(a): The perceived strategy restrictiveness of the AC aid will be lower than that of the elimination aid.
A second way to reduce strategy restrictiveness is to support multiple decision strategies that complement one another, rather than a single strategy (e.g., Elrod et al. 2004). The objective restrictiveness of the hybrid aid is lower than the AC aid because the capabilities it supports (i.e., AC strategy only) are a subset of those supported by the hybrid aid (both AC and elimination strategies) (Silver 1988). Moreover, both the AC and elimination strategies are likely to be utilized by users because, as suggested by Todd and Benbasat (2000), “in practice, individuals typically employ hybrid strategies as opposed to pure approaches” (p. 96). The hybrid aid supports the user- preferred hybrid strategies; thus, users will perceive it to be less restrictive than the AC aid.
Prior research has also confirmed users’ ability to perceive the difference in the objective restrictiveness of decision aids. In particular, Silver (1988) investigates users’ perceptions of restrictiveness of DSS with different types and amounts of supporting operators (e.g., Data Sort), from which the users can choose and control. He compares three types of DSS, namely, Lotus 1-2-3” Multi-Attribute-Software System (MASS), and Elimination by Aspector (EBA). EBA applies a single choice strategy (elimination) that is a subset of the decision rules supported by MASS, whereas the latter applies a subset of the choice rules supported by Lotus 1-2-3. Silver found that the overall user perception of restrictiveness was lower when more decision rules and functionalities were supported by a DSS. Similarly, we posit the following hypothesis:
H5(b): The perceived strategy restrictiveness of the hybrid aid will be lower than that of the AC aid.
Perceived Advice Quality of Decision Aids
The AC aid will yield higher decision quality than the elimi- nation aid because the former utilizes all available infor- mation when making a choice (e.g., Keeney and Raiffa 1976; Payne et al. 1993). Particularly, by taking into account the relative importance of users’ attribute preferences, the AC aid can provide more accurate product recommendations that reflect the differential preference structure of users than can the elimination aid (Baron 2004; Keeney and Raiffa 1976). To perceive the advice quality of an aid, users can assess, for example, (1) whether or not the aid elicits the importance levels of their attribute preferences and (2) whether or not
MIS Quarterly Vol. 33 No. 2/June 2009 299
Wang & Benbasat/Interactive Decision Aids
Table 1. Properties of the Three Decision Aids
Property Decision Aid Normativeness? Complementarity?
Elimination aid No No
AC aid Yes No
Hybrid aid Yes Yes
their important preferences are satisfied by the top recommendations of the aid. Through these evaluations, users will perceive the advice quality of the AC aid to be higher than that of the elimination aid.
H6(a): The perceived advice quality of the AC aid will be higher than that of the elimination aid.
As the hybrid aid incorporates the AC strategy, it naturally has the same quality advantage as the AC aid. That is, the hybrid aid also takes into account the importance levels of users’ attribute preferences and makes tradeoffs by using the AC strategy when generating recommendations. Neverthe- less, users may consider some attribute levels to be unaccept- able for certain attributes. The hybrid aid also supports the elimination strategy, which helps screen out product alter- natives with such unacceptable attribute levels. This process further increases advice quality. Users may perceive the advice quality based on the support (or the lack of support) of complementary decision strategies. In addition, they could also examine the recommended products to judge the advice quality. If alternatives with unacceptable attribute levels are recommended by an aid, it is likely that the user would perceive the aid as not being responsive to her needs and, consequently, its advice quality as being low (Elrod et al. 2004; Green and Srinivasan 1990; Song et al. 2007; Tan 2003). Therefore, the complementarity of the AC and elimi- nation strategies will cause users to perceive that the advice quality of the hybrid aid is higher than that of the AC aid.
H6(b): The perceived advice quality of the hybrid aid will be higher than that of the AC aid.
Perceived Cognitive Effort of Decision Making with Decision Aids
A major objective of using decision aids in online decision making is to save cognitive effort on the part of the users (e.g., Todd and Benbasat 1992). When the processing of user attribute preferences is automated by decision aids, the dif-
ference in the user’s effort required in the stages of the elicita- tion and processing of user preferences is minimized.3
Nevertheless, consumers may need to refine their inputs to a decision aid, and thus the use of the aid also entails the necessity of user effort for iterating. The different support provided by decision aids influences the need for iterations (Fasolo et al. 2005; Song et al. 2007). Again, we first com- pare the AC aid and the elimination aid. The AC aid allows users to express the importance of their attribute preferences, whereas the elimination aid does not. In the absence of such support, users may have to adjust their preferences to better convey their requirements. In doing so, they need to use the aid more times, thereby spending more effort. Previous studies have shown that users of elimination aids experience a larger number of iterations in revising attribute preferences than those using AC aids (Fasolo et al. 2005). Song et al. (2007) compared decision aids that use a compensatory strategy (e.g., AC) and those that use a non-compensatory strategy (e.g., elimination), and found that the compensatory- based aid is perceived to require less effort than the non- compensatory-based aid. Similarly, we posit the following:
H7(a): The perceived cognitive effort of using the AC aid will be lower than that of using the elimination aid.
The above-mentioned reason for the differences between the AC aid and the elimination aid also applies to the comparison between the hybrid aid and the AC aid. The function of
3 There is some difference in the effort of inputting preferences across dif-
ferent decision aids. In particular, the AC aid requires the user to indicate the importance of each attribute preference, whereas the elimination aid does not. Furthermore, the hybrid aid requires the user to indicate whether the attribute preference should be processed in a compensatory or non-compensatory way. Moreover, the number of recommended products that users examine before they make the final choice and, accordingly, the effort in examining the pro- ducts may differ across decision aids (Häubl and Trifts 2000). However, the differential effort across decision aids in providing inputs and examining recommended products for a single iteration is dominated by the effort of additional iterations (Fasolo et al. 2005). Therefore, this study mainly focuses on the effort needed for iterations.
300 MIS Quarterly Vol. 33 No. 2/June 2009
Wang & Benbasat/Interactive Decision Aids
indicating unacceptable levels of certain product attributes provided by the hybrid aid further supports the expression of preferences. For users of the hybrid aid, the need to adjust their requirements in the user-aid dialogues will be reduced. As a result, they can save more effort than those using the AC aid. We therefore posit the following:
H7(b): The perceived cognitive effort of using the hybrid aid will be lower than that of using the AC aid.
Effects of Explanation Facilities
According to cognitive learning theories, explanation faci- lities assist users to appropriately evaluate a decision aid for two reasons (Balzer et al. 1989; Brunswik 1952; Dhaliwal and Benbasat 1996). First, explanations promote the cognitive assessment of decision aids by increasing the understand- ability of the aids (e.g., Arnold et al. 2006; Gregor and Benbasat 1999). To be able to judge if their preferred deci- sion processes are supported by an aid (i.e., restrictiveness) and if the aid is able to produce high quality advice, users need to understand the strategy(ies) employed by the aid and how it works (Beaulieu and Jones 1998; Dhaliwal and Benbasat 1996; Wang and Benbasat 2007). When the mental model of a decision aid is in line with the preferred decision processes of the user, she will then perceive the aid to be less restrictive.
In particular, the how and why explanations inform the rationale of eliciting requirements from users and the inner workings of a decision aid (Wang and Benbasat 2007). Such explanations can assist users to build a mental model of the aid, and thus allow their judgments about the aid to better reflect its actual capabilities. It is more likely that after viewing the explanations, users would perceive a decision aid with low restrictiveness as the one that supports the decision processes they want to employ. Similarly, for a decision aid that provides high-quality advice, explanation facilities will make users feel more comfortable and confident in the advice given. As a result, through explanations, users can perceive the restrictiveness and advice quality of decision aids more accurately, and the comparisons between different aids will be more salient.
The second reason why explanations help users evaluate a decision aid is that they facilitate behavioral assessment of the aid by helping them properly utilize its capabilities (Silver 1991a). In particular, the guidance explanation is intended to help users make informed choices when interacting with the aid. By accomplishing this, users will more likely realize that
they are indeed supported by a less restrictive aid. In addition, the advice quality of a decision aid is influenced by inputs from the user (e.g., attribute preferences) (Wang and Benbasat 2007). If users are unable to make informed choices when interacting with a decision aid, the aid may in turn be unable to provide high-quality advice. Consequently, the dif- ference in advice quality among the aids will not be salient. To summarize, explanations will help the user better realize the differences in perceived strategy restrictiveness and advice quality between different decision aids.
H8(a): The differences in users’ perceptions of strategy restrictiveness between the AC aid and the elimination aid and those between the hybrid aid and the AC aid will be larger when explanations are provided than when explanations are not provided.
H8(b): The differences in users’ perceptions of advice quality between the AC aid and the elimination aid and those between the hybrid aid and the AC aid will be larger when explanations are provided than when explanations are not provided.
No hypotheses are stated concerning the differences in perceived cognitive effort in the presence and absence of explanations. The reason is that, as discussed earlier, the differences in cognitive effort required by different types of decision aids are mainly induced by the support for the expression of user requirements and filtering products with unacceptable attribute levels, neither of which are directly influenced by explanations.
Research Methodology
To compare the decision aids and examine the impacts of perceived strategy restrictiveness, advice quality, and cogni- tive effort on intentions to use online decision aids, a 3 × 2 between-subject experimental design was employed (Table 2). Descriptions of the experimental decision aids and the participants are provided next. The experimental tasks and procedures are then presented, followed by a description of the dependent variables and their measures.
Decision Aids
The experimental decision aids were designed to help the participants choose a digital camera. This product was chosen
MIS Quarterly Vol. 33 No. 2/June 2009 301
Wang & Benbasat/Interactive Decision Aids
Table 2. The 3 × 2 Full Factorial Experimental Design
Decision Aid (Different Decision Strategy Support Capabilities)
Hybrid Aid AC Aid Elimination Aid
Explanations With Group 1 Group 2 Group 3
Without Group 4 Group 5 Group 6
for two reasons. First, digital cameras are suitable for a multi- attribute, multi-alternative choice task because they bear a variety of product attributes and there are hundreds of different models. Indeed, many of the decision aids already available have been developed for the marketing and sale of digital cameras. Second, an informal survey that we con- ducted revealed that many undergraduates had considerable interest in them, which meant that the student participants in the experiment would be motivated regarding the topic of digital cameras (Wang and Benbasat 2007, 2008).
The experimental decision aids simulated well-known operational aids available from leading online decision aid providers (e.g., www.MyProductAdvisor.com). To elicit users’ preferences, a user-aid dialogue was used to simulate the dialogues presented in other studies (e.g., Komiak and Benbasat 2006; Russo 2002; Wang and Benbasat 2007, 2008) and in commercial applications. The user-aid dialogue in- cluded 11 questions that corresponded to 11 product attributes and features (price, brand, zoom, resolution, camera size, screen size, flash, rapid-fire shot, image processing time, additional features, and advanced features). For most of the attributes, three nominal or ordinal options (levels) were provided from which users could choose.4 These attributes and options were drawn from the aforementioned decision aids available in commercial websites. We also validated the user-aid dialogue with the assistance of several digital camera experts and incorporated their comments into the design of the user-aid dialogue. Screenshots of the dialogue interface of the three decision aids and a sample question from the user-aid dialogue are provided in Appendix A.
The decision aids first use embedded decision rules to deter- mine the attribute levels for products (e.g., a digital camera with an LCD screen size larger than 3 inches) based on pre-
ferences elicited from users (e.g., a digital camera with a large LCD screen). After that, the attributes of available alternatives are compared with the user’s preferred product features or attribute levels (Russo 2002). Next, when a pro- duct alternative does not have an attribute that fits the user’s preferred attribute level, in the case of the AC aid, the fit score of the alternative is reduced, whereas the elimination aid removes the alternative from further consideration.
As opposed to the typical implementation of offline AC aids, the implementation of the AC aid in this study does not assume that the desirability of an attribute is a monotonically increasing or decreasing function of the magnitude of the attribute. For example, if a user prefers a digital camera with a medium-sized LCD screen, neither an alternative with a large-size screen nor one with a small-sized screen will exactly match user preferences, and the fit scores of such products will decline accordingly. We elicited the user pre- ferences using an ideal-point oriented approach (Russo 2002), in line with the traditional concept of the attractiveness asso- ciated with product attributes (Montgomery and Svenson 1976; Svenson 1979) and the implementation of online decision aids in recent studies (Komiak and Benbasat 2006; Wang and Benbasat 2007).5
With the hybrid aid, users can indicate whether or not the preference can be traded off with other preferences when answering questions about their attribute preferences in the user-aid dialogue. This was implemented by asking users if their preference was essential or nonessential. For the set of essential attribute preferences, the elimination strategy was applied (i.e., it filtered product alternatives that did not satisfy any of the essential attribute preferences), whereas for the set of nonessential preferences, the AC strategy was applied (i.e., it made trade-offs among the nonessential attribute prefer- ences). In doing so, the hybrid aid allows users to use dif- ferent strategies across product attributes.
4 The magnitude of the differences between options was determined in two
steps. We first determined the whole range of the value of an attribute by examining the products available in the market and allocated the differences evenly among the three options. We then adjusted the differences by taking into account the number of product alternatives satisfying each option. For additional and advanced features (e.g., remote control), check boxes were provided to the participants to indicate whether or not they wanted those features.
5 Given that the essential feature of the AC of interest in the study (i.e., its
normativeness) remains, regardless of its implementation, we believe that the differences in implementation will not impose a threat to the validity of the findings of this study. Nevertheless, there is a need in future research to empirically test the generalizability of the findings to online decision aids that are implemented in different ways.
302 MIS Quarterly Vol. 33 No. 2/June 2009
Wang & Benbasat/Interactive Decision Aids
In the user-aid dialogue, the participants could choose any questions in any order to answer. After the decision aids had elicited participants’ requirements in the user-aid dialogue, most of the participants received several pages of recommen- dations, with each page containing five products.6
The recommended products returned by all three decision aids were ranked and ordered. The hybrid aid and the AC aid ranked the recommended products based on their fit scores (from high to low). For the elimination aid, the recommen- dations were ranked by price (from low to high), which followed the practice of search functions available at many websites. From the perspective of the elimination decision aid, it is reasonable to recommend products with lower cost first when all the recommended products fit all the require- ments of the user. A screenshot of the recommendation page as well as many operational details of the experimental decision aid are presented in Appendix A.
Participants
The participants were comprised of 156 undergraduate and graduate students from a large North American university. The average age of the participants was 21.4 years; 50 of the participants were males. Half of the participants utilized the Internet for more than 2 hours a day, 48 percent of them used it between 30 minutes and 2 hours, and 2 percent used it for less than 30 minutes a day. In addition, most of them (97 percent) had used the Internet for more than 3 years.
The validity of the choice of participants in the context of the study is justified by the fact that they matched the Internet user demographic data (Johnson 2005). People with similar backgrounds and profiles as our participants comprise a major segment of Internet users (Burns 2006). Furthermore, this study was conducted in the context of using online decision aids for buying digital cameras; college students are generally a group of people interested in such tasks as revealed in the informal survey reported in the previous section. Hence, college students are a relevant subject group.
The use of student participants also helped control for factors external to this research that may influence a buyer’s use of online decision aids. Those factors would make it difficult to isolate the effects of variables critical to this research (Ben-
basat 1989). Moreover, because the domain knowledge of the participants may influence the usefulness of the explanations and thus the use and effects of the explanations (Arnold et al. 2006; Ye and Johnson 1995), we controlled for the product knowledge of the participants. When the potential partici- pants signed up online for the experiment, they were asked to report their experience with digital cameras and their knowl- edge of digital cameras using a five-point scale. We only invited as participants those registrants who had yet to own or buy a digital camera and whose domain knowledge was below the medium level on the scale. This use of filtering, which has also been adopted in previous research (e.g., Wang and Benbasat 2007), helped mitigate the confounding factor of domain knowledge.7
All of the participants were randomly assigned to one of the treatment groups. Before the experimental tasks, their pro- duct knowledge and comfort levels with the Internet and online shopping were surveyed. No significant differences were found between the participants in the different experi- mental conditions, and no significant differences were found between the groups in terms of their age, gender, Internet usage, and Internet experience.
Experimental Tasks and Procedures
The participants were asked to complete two tasks. One was to choose a digital camera as a wedding gift for a good friend who likes photography very much, with the cost to be shared with four other friends. The other task was to select a digital camera for a close family member who is an amateur photo- grapher. The participants were informed that the tasks were flexible, and that they could make as many assumptions as they wished. Two tasks were used instead of one to ensure that the participants had sufficient interaction with the aid to evaluate it (Wang and Benbasat 2007). The participants were directed to a Web vendor to shop for the two tasks and the Web vendor provided a decision aid to help consumers make choices.
6 The product database used in the decision aids contained 132 products. Most
of the product models available in the market at the time of the experiment were included. The number of recommendations depended on the require- ments elicited from the participants. In the most reduced set, no product recommendations could be provided by the aid.
7 In addition, previous research has shown that consumers who already owned
or had purchased digital cameras in the past based their evaluations primarily on whether or not the decision aid recommended the model that they owned (Wang and Benbasat 2007), and hence their evaluations might be biased. The sampling employed here helped reduce such potential bias. The filtering- based sampling method was also justified because most consumers need extra shopping advice when they buy a product such as a digital camera,for the first time. In such case, consumers may not have sufficient knowledge and experience to make an informed purchase decision. In fact, online decision aids are mainly provided for consumers who lack product expertise and do not know which product is most suitable for them (Wang and Benbasat 2007).
MIS Quarterly Vol. 33 No. 2/June 2009 303
Wang & Benbasat/Interactive Decision Aids
Participation was voluntary, and each participant was given monetary compensation (CAD $15) in exchange for his/her participation. To motivate the participants to view the experi- ment as a serious online shopping session and to increase their involvement, they were informed before the experimental tasks that 25 percent of them would get an additional award from $5 to $100 based on their performance. They were also informed prior to the experiment that they would be asked to provide their justifications for their shopping choices, and that their performance would be judged based on how convincing their justifications were in supporting their shopping choices.8
The experimental procedures were as follows. Upon arrival, the participants completed a background questionnaire to collect their demographic information. A research assistant first trained the participants on how to use and navigate the assigned Web interface using recorded videos that showed a tutorial decision aid with the same features as the experi- mental aid for that condition. Each participant was then asked to complete the two experimental tasks, the order of which was systematically reversed from participant to participant. The tasks had no time limit, and the participants could iterate through their replies to the user-aid dialogue as many times as they wished. After each task, the participants were directed to an online form to write down their choice and their justifi- cations for such choice. They were required to use the decision aid first in order to evaluate it for experimental purposes, but were also allowed to use a simple search query (search by brand name) available on the homepage of the experimental website if they were not satisfied with the aid. Finally, after performing the two tasks, the participants were asked to complete a questionnaire that included the measures of the dependent variables and the manipulation checks.
Dependent Variables
Four dependent variables were assessed in this study. There are well-established multi-item measures in the literature for perceived cognitive effort and intentions to use a decision aid. Those for perceived cognitive effort were taken from Pereira
(2000),9 and those for intentions to use a decision aid were adopted from Venkatesh et al. (2003).
The direct measure of perceived advice quality is whether or not the participants followed the top recommendations pro- vided by the decision aid. After undergoing the user-aid dialogues, the participants received the top five recommen- dations and could then navigate to the next page to obtain the next five recommendations, if available. We coded the per- ceived advice quality into three levels. If a participant followed the top recommendations from the aid by choosing one of those, which was shown on the first page of the recommendations, we infer that she perceived the advice quality to be high, and it was therefore coded “2.” If the participant checked more pages of recommendations from the aid and did not follow the top ones (i.e., the product she chose was not among the top five shown on the first page), we infer that she perceived the advice quality to be lower and it was therefore coded “1.” Otherwise, if the participants selected a product on their own using the simple search query available on the homepage of the experimental website, we concluded that they did not perceive the recommendations from the aid to be of satisfactory quality, and it was therefore coded “0.”10
We developed new instruments for perceived strategy restrictiveness following the guidelines in Moore and Ben- basat (1991), because there are no validated measures available for this variable in the literature. We generated candidate measurement items with the help of four parti- cipants (two graduate students, one from the Department of Psychology and the other from the Faculty of Science, and two assistant professors in MIS). To further check content validity, the instrument was submitted to a panel of graduate students majoring in MIS to obtain their views on appropriate items for inclusion. This procedure generated seven items for perceived strategy restrictiveness.
8 A research assistant who was very knowledgeable about digital cameras
helped judge the justifications. The research assistant counted the number of supporting arguments weighted by supporting strengths (level of persuasion). Three levels of supporting strengths were used: 0.5, 1, and 2, representing weak, normal, and strong, respectively. The sum of the weighted argument numbers was then used as an indicator of performance.
9 We also captured the decision times of the participants as another indicator
of cognitive effort. The experimental data show that the decision times were correlated significantly with perceived cognitive effort, but the correlations were not very high (Pearson correlation coefficients of 0.22~0.26). We also checked the correlations between decision times and intentions to use deci- sion aids, and found that the correlations were not significant. To be con- sistent with the two other perception-based factors leading to intentions to use decision aids, we used a multi-item scale for perceived cognitive effort, rather than decision times.
10 The appropriateness of the operationalization of perceived advice quality
will be further confirmed in the “Discussion” section. In addition, we mea- sured perceived advice quality with multi-item scales, including perceived competence and perceived usefulness of decision aids. The results of the hypothesis tests were the same regardless of the scale used. Nevertheless, the measure of perceived advice quality is somewhat coarse, and future research is therefore needed to refine it.
304 MIS Quarterly Vol. 33 No. 2/June 2009
Wang & Benbasat/Interactive Decision Aids
Next, a card-sorting exercise was used in the new scale devel- opment process (Moore and Benbasat 1991). The newly created measures, together with items for the other constructs, were shuffled into a random order and were presented to the judges. Two rounds of this exercise were conducted. In the first round, four master’s and doctoral students were asked to sort the items into separate categories based on their simi- larities and differences, and to label the underlying constructs for each of the categories. The average inter-judge raw agree- ment rate was 73 percent. The judges also provided com- ments on ambiguous or unclear items. In this process, only a single item of perceived strategy restrictiveness was identified to be too ambiguous. The rest of the items were refined ac- cording to the comments of the judges and were retained for the next sorting round.
In the second round, another four master’s and doctoral students were asked to sort the refined and retained items based on the construct definitions. A “too ambiguous/doesn’t fit” category was also included to ensure that the judges were not forced to fit any item into a predefined category. This round of sorting ended with an average raw agreement rate of 95 percent, indicating a very high reliability (Moore and Benbasat 1991). This process also helped establish the discriminant validity of the items. The resulting measures for perceived strategy restrictiveness contained six items (see Appendix B). These measures were further validated in the main experiment.
Data Analysis and Results
We first conducted manipulation checks and validated the measurement. Then, to test the extended effort–accuracy– restrictiveness framework and the relationships among the three antecedents, a partial least squares (PLS) analysis was employed. A set of planned a priori contrasts using Bon- ferroni tests was applied to examine the differences in the participants’ perceptions of strategy restrictiveness, advice quality, and cognitive effort of the different decision aids.
Manipulation Checks
Manipulation checks were conducted for the two treatments. When interacting with the hybrid aid, the participants were required to choose either the “nonessential preference” (i.e., AC) or the “essential preference” (i.e., elimination) for each of their attribute preferences. Table 3 reports the distribution of the strategy choices of the participants in the hybrid aid conditions. On the average, about 40 percent of the partici-
pants’ choices were essential preferences, and the remaining choices were nonessential. The results show that most participants utilized both AC and elimination strategies. The implementation of the hybrid aid was, therefore, considered to be successful.
The use of explanations was counted by watching videos that were captured unobtrusively during the experiment through a screen capture software (Camtasia Recorder 3.0) (Wang and Benbasat 2007). On the average, 31 percent of the how explanations, 22 percent of the why explanations, and 25 percent of the guidance were viewed by the participants (the detailed frequency distributions of explanation use are reported in Table 4). The average usage rates were similar to those in previous empirical studies (e.g., Dhaliwal 1993; Wang and Benbasat 2007),11 and as such the manipulation of the explanation facilities seems to be successful.
In addition to the objective measure of explanation usages, a perceptual measure was also used for the manipulation check. After the experimental tasks, the participants were asked to report their level of understanding of the logic used by the decision aid to provide shopping advice on a seven-point Likert scale. The participants in the “with explanations” group reported a significantly higher level of understanding than those in the “without explanations” group: with explana- tions group (mean = 5.3, SD = 1.5), without explanations group (mean = 4.1, SD = 1.2), t(154) = 5.08, p < 0.001. The subjective measure also shows that the manipulation of explanations was successful.
Measurement Characteristics
The descriptive data of the dependent variables are presented in Table 5. Following extensive pretests in the instrument development stage, we examined the reliability and validity of the constructs. The reliability indicators that were mea- sured by Cronbach’s α (Table 6) were all above the cited minimum value of 0.7, indicating that each set of mea- surement items was consistent in what it intended to measure
11 For example, in Dhaliwal’s study, the average usage rates for the how and
why explanations were 20.3% and 17.9%, respectively, whereas guidance was not examined. The usage rates were not very high for two reasons. First, the calculation of rates was based on all available explanations. How, why, and guidance explanations were provided for each question in the user-aid dialogue, but the participants did not need to answer all of the questions in the dialogue. Therefore, it was not surprising that the usage rates based on all available explanations were not high. Second, theoretically, it is consis- tent with the prediction based on the production paradox that the actual usage rates would not be high because the participants focused more on solving a problem than on the sole purpose of learning (Carroll and McKendree 1987).
MIS Quarterly Vol. 33 No. 2/June 2009 305
Wang & Benbasat/Interactive Decision Aids
Table 3. Distribution of Strategy Choice for the Hybrid Aid
Percent of “Essential Preference” chosen (% of “Non-essential Preference” chosen)
Number of the participants using the hybrid aid in the “with” explanation group
(%)
Number of the participants using the hybrid aid in the “without” explanation group
(%)
Based on Initial Choice
Based on Final Choice
Based on Initial Choice
Based on Final Choice
100% (0%)
2 (7.6%)
2 (7.6%)
1 (3.8%)
0 (0%)
75.0% – 99.9% (0.1% – 25.0%)
0 (0%)
0 (0%)
1 (3.8%)
1 (3.8%)
50.0% – 74.9% (25.1% – 50.0%)
6 (23.1%)
5 (19.2%)
8 (30.8%)
9 (34.6%)
25.0% – 49.9% (50.1% – 75.0%)
10 (38.5%)
8 (30.8%)
11 (42.3%)
11 (42.3%)
0.1% – 24.9% (75.1% – 99.9%)
5 (19.2%)
8 (30.8%)
4 (15.4%)
4 (15.4%)
0% (100%)
3 (11.5%)
3 (11.5%)
1 (3.8%)
1 (3.8%)
Average % of “Essential Preference” chosen
40.1% 35.7% 42.1% 40.8%
Average % of “Non-Essential
Preference” chosen 59.9% 64.3% 57.9% 59.2%
Note: Because the participants could change and refine their choices, the distributions of strategy choice that were based on initial choices and
final choices are summarized separately.
Table 4. Frequency Distributions of Explanation Use
How explanations Why explanations Guidance
% of how explanations used (pc: percentage)
Number of subjects
% of why explana- tions used (pc)
Number of subjects
% of guidance used (pc)
Number of subjects
pc = 0† 5 pc = 0 5 pc = 0 9
0 < pc < 10% 17 0 < pc < 10% 25 0 < pc < 10% 26
10% < pc < 30% 21 10% < pc < 30% 20 10% < pc < 30% 12
30% < pc < 50% 19 30% < pc < 50% 16 30% < pc < 50% 16
50% < pc < 70% 13 50% < pc < 70% 4 50% < pc < 70% 4
pc > 70% 2 pc > 70% 2 pc > 70% 5
Total 72‡ Total 72 Total 72
†Several participants did not view any explanations for a particular type of explanation, and two did not view any explanations at all. Dropping these
two participants did not change the hypothesis testing results. Therefore, in the analyses all of the participants were retained.
‡In total, 78 participants were assigned to the “with” explanations condition. Among these, the navigations and screens of six were not successfully
recorded, and therefore the explanation use rates reported here are based on 72 participants.
306 MIS Quarterly Vol. 33 No. 2/June 2009
Wang & Benbasat/Interactive Decision Aids
Table 5. Descriptive Statistics: Mean (Std. Deviation) and Correlation
Means (Std. Deviation) Correlations
With Explanations Without Explanations
H yb
ri d
A C
E lim
in a tio
n
T o ta
l
1 2 3 4H yb
ri d
A C
E lim
i- n a tio
n
S u b -
to ta
l
H yb
ri d
A C
E lim
i- n a tio
n
S u b -
to ta
l
1. Perceived
Cognitive Effort
2.56
(1.23)
2.38
(1.09)
3.00
(1.26)
2.65
(1.21)
3.09
(1.41)
2.29
(1.08)
2.96
(1.29)
2.78
(1.30)
2.83
(1.34)
2.34
(1.08)
2.98
(1.27)
2.72
(1.25) —
2. Perceived
Advice Quality
1.71
(0.57)
1.69
(0.43)
1.02
(0.71)
1.48
(0.66)
1.44
(0.75)
1.40
(0.58)
1.23
(0.81)
1.36
(0.71)
1.58
(0.67)
1.55
(0.53)
1.12
(0.76)
1.42
(0.68) -0.23 —
3. Perceived
Strategy
Restrictiveness
3.82
(1.21)
3.95
(0.99)
4.82
(0.91)
4.19
(1.12)
4.43
(1.01)
4.14
(0.93)
4.58
(0.93)
4.38
(0.96)
4.12
(1.14)
4.04
(0.96)
4.70
(0.92)
4.29
(1.05) 0.34 -0.30 —
4. Intentions to
Use
4.59
(1.77)
4.56
(1.47)
3.51
(1.96)
4.23
(1.79)
3.78
(1.83)
4.69
(1.60)
3.51
(1.69)
4.00
(1.76)
4.19
(1.83)
4.63
(1.52)
3.51
(1.81)
4.11
(1.77) -0.43 0.36 -0.51 —
Table 6. Reliability and Factor Analysis
Construct Cronbach’s α Item Factor 1 Factor 2 Factor 3
Intentions to use 0.98 INT1 0.96 0.04 0.03
INT2 0.98 0.03 0.00
INT3 0.98 0.06 0.02
Perceived Cognitive Effort
0.87 0.84*
PCE1 (dropped) 0.44 0.41 0.19
PCE2 (dropped) 0.63 0.25 0.05
PCE3 0.02 0.84 0.03
PCE4 (dropped) 0.43 0.48 0.01
PCE5 0.07 0.88 0.07
PCE6 0.10 0.86 0.03
Perceived Strategy Restrictiveness
0.74 0.73*
PSR1 (dropped) 0.22 0.19 0.29
PSR2 0.24 0.09 0.47
PSR3 0.28 0.00 0.50
PSR4 0.10 0.04 0.70
PSR5 0.15 0.02 0.81
PSR6 0.10 0.14 0.77
Eigenvalue 6.59 1.72 1.36
Percentage of Explained Variance 43.9 11.4 9.0
Extraction Method: Principal Component Analysis. Rotation Method: Oblimin with Kaiser Normalization.
*indicates reliability after some items were dropped.
MIS Quarterly Vol. 33 No. 2/June 2009 307
Wang & Benbasat/Interactive Decision Aids
(Nunnally and Bernstein 1994). The discriminant validity of the constructs was assessed through an exploratory, principal components factor analysis (PCA) with direct oblimin rotation, using SPSS for Windows Version 13.0. The results of factor analysis are reported in Table 6. Except for three items for perceived cognitive effort (i.e., PCE1, PCE2, and PCE4) and one item for perceived strategy restrictiveness (i.e., PSR1), all other items had loadings above the commonly specified minimum value of 0.4 on the intended construct, and did not have cross loadings above 0.4 on the unintended con- structs (Hair et al. 1998). The four exceptions were dropped from later analyses, whereas all others items were retained.
PLS Analysis Results
The results of the structural model are shown in Figure 3. As predicted, all the three factors exerted significant effects on intentions to use decision aids. Congruent with the effort– accuracy framework, the participants focused more on effort reduction than advice quality when using the decision aids (standardized path coefficient: 0.26 versus 0.19). More importantly, we found support for the central premise of this study. The perceived strategy restrictiveness of decision aids exerted a significant (and the highest) impact on intentions to use the aids (standardized path coefficient 0.37). Therefore, H1(a), (b), and (c) were all supported.
The relationships among the perceptions of strategy restric- tiveness, advice quality, and cognitive effort were also con- firmed, lending support to the indirect effects of perceived strategy restrictiveness and advice quality on intentions to use a decision aid. Perceived strategy restrictiveness significantly influenced perceived advice quality and perceived cognitive effort, and perceived advice quality significantly influenced perceived cognitive effort. A further mediation test based on Baron and Kenny (1986) confirmed that perceived advice quality and cognitive effort partially mediated the rela- tionships between perceived strategy restrictiveness and intentions to use a decision aid. Therefore, H2 and H3 were supported. Similarly, perceived cognitive effort partially mediated the relationships between perceived advice quality and intentions to use a decision aid, lending support to H4.
Group Comparisons with Bonferroni Tests
To test the hypotheses on the differences among the three decision aids, we conducted a series of group contrasts using Bonferroni tests. In this study, we are interested in two non- orthogonal a priori contrasts (the AC aid versus the elimi- nation aid on the one hand, and the hybrid aid versus the AC
aid on the other) for which the Bonferroni tests are considered suitable (Kirk 1995).
The results of the Bonferroni tests are reported in Table 7. As predicted, the participants perceived the AC aid to be significantly different from the elimination aid. The percep- tions of strategy restrictiveness and cognitive effort of the AC aid were significantly lower than those of the elimination aid, and the perceived advice quality of the AC aid was signi- ficantly higher than that of the elimination aid. Therefore, H5(a), H6(a), and H7(a) were all supported. However, the differences between the AC aid and the hybrid aid were not significant in terms of perceived strategy restrictiveness, cognitive effort, and advice quality. Thus, H5(b), H6(b), and H7(b) were not supported, indicating that the evidence for the significant effects of strategy complementarity was not found. Table 8 summarizes the results.
Hypotheses H8(a) and (b) focus on the moderating effects of explanations facilities, so the contrasts were conducted under two subgroups. In one subgroup, the participants interacted with decision aids that provided explanations, whereas ex- planations were not provided in the other group. The results are presented in Table 9. In terms of perceived strategy restrictiveness and advice quality, the differences between the AC aid and the elimination aid were significant in the “with explanations” condition, but not in the “without explanations” condition. Therefore, H8(a) and H8(b) were partially supported.
Discussion, Contributions, Implications, and Limitations
Discussion of the Findings
This study examines an extended effort–accuracy–restrictive- ness framework of decision making supported by online deci- sion aids, and compares the perceptual differences between users of different decision aids that are characterized by their levels of strategy normativeness and complementarity. As predicted, in addition to perceived advice quality and per- ceived cognitive effort, perceived strategy restrictiveness exerts a significant impact on intentions to use decision aids. Users desire to be less restricted by the decision strategy(ies) employed by the aids.
From a broader perspective, this research can be viewed as an extension of the theories of cognitive fit and task–technology fit to the context of using online decision aids for multi- attribute, multi-alternative decision tasks (e.g., Goodhue and
308 MIS Quarterly Vol. 33 No. 2/June 2009
Wang & Benbasat/Interactive Decision Aids
Figure 3. Results of the PLS Analysis
Table 7. Multiple Comparison Results (Bonferroni Tests)
Group A Group B Mean Difference
(A – B) Significance
Perceived Strategy Restrictiveness
AC Aid (mean: 4.04, sd: 0.96)
Elimination Aid (mean: 4.70, sd: 0.92)
-0.66 0.003
Hybrid Aid (mean: 4.12, sd: 1.14)
AC Aid (mean: 4.04, sd: 0.96)
0.08 1.00
Perceived Advice Quality
AC Aid (mean: 1.55, sd: 0.53)
Elimination Aid (mean: 1.12, sd: 0.76)
0.43 0.004
Hybrid Aid (mean: 1.58, sd: 0.67)
AC Aid (mean: 1.55, sd: 0.53)
0.03 1.00
Perceived Cognitive Effort
AC Aid (mean: 2.34, sd: 1.08)
Elimination-Aid (mean: 2.98, sd: 1.27)
-0.64 0.03
Hybrid Aid (mean: 2.83, sd: 1.34)
AC Aid (mean: 2.34, sd: 1.08)
0.49 0.14
Table 8. Summary of Contrast Results
Hypotheses Results Supported?
Perceived strategy restrictiveness
Normativeness H5(a): Elimination > AC Elimination > AC Yes
Complementarity H5(b): AC > Hybrid AC = Hybrid No
Perceived advice quality
Normativeness H6(a): Elimination < AC Elimination < AC Yes
Complementarity H6(b): AC < Hybrid AC = Hybrid No
Perceived cognitive effort
Normativeness H7(a): Elimination > AC Elimination > AC Yes
Complementarity H7(b): AC > Hybrid AC = Hybrid No
Perceived Advice Quality
(R2 = 0.09)
Perceived Restrictiveness
Perceived Cognitive Effort
(R2 = 0.14)
Intention to Use Decision Aids
(R2 = 0.37)
*p < 0.05
0.19*
-0.26*
-0.37*
0.30*
-0.30*
-0.14*
Perceived Advice Quality
(R2 = 0.09)
Perceived Restrictiveness
Perceived Cognitive Effort
(R2 = 0.14)
Intention to Use Decision Aids
(R2 = 0.37)
*p < 0.05
0.19*
-0.26*
-0.37*
0.30*
-0.30*
-0.14*
MIS Quarterly Vol. 33 No. 2/June 2009 309
Wang & Benbasat/Interactive Decision Aids
Table 9. Multiple Comparison Results: With Versus Without Explanations (Bonferroni Test)
Group A Group B
With Explanations (n = 78)
Without Explanations (n = 78)
Mean Difference (A – B) Significance
Mean Difference (A – B) Significance
Perceived Strategy Restrictiveness
AC Aid Elimination Aid -0.87 0.01 -0.44 0.31
Hybrid Aid AC Aid -0.13 1.00 0.29 0.82
Perceived Advice Quality
AC Aid Elimination Aid 0.67 < 0.001 0.18 1.00
Hybrid Aid AC Aid 0.02 1.00 0.04 1.00
Thompson 1995; Vessey and Galletta 1991). Perceptions of strategy restrictiveness can be considered as the perceptual fit between a decision aid (i.e., a problem-solving tool) and the user’s referred decision processes in solving a decision task. Our findings are generally in line with the cognitive fit theories, which maintain that decision aids should “support the strategies (methods or processes) required to perform that task” (Vessey and Galletta 1991, p. 64).
Regarding the comparisons of decision aids, the AC aid was perceived to be significantly different from the elimination aid. Due to the fact that the AC and elimination aids each support a single strategy, one cannot say that one of the two aids has greater objective strategy restrictiveness than the other. However, as AC is more normative than elimination, users perceive the AC aid to be less restrictive than the elimination aid. The desirability of the AC aid demonstrates that the prescriptive, normative rules used by AC to produce accurate choices are not merely proposed theoretically. More so, these rules are in line with the fact that for multi- attribute, multi-alternative choice problems, consumers accord different importance levels across attribute preferences and need to make trade-offs among them (Bettman et al. 1998; Song et al. 2007).
The perceptual differences across decision aids are particu- larly marked when explanation facilities are provided by the aids. We conducted a further analysis on the possible direct effects of explanation facilities on perceived restrictiveness. We compared the differences between the perceived restric- tiveness for a given decision aid with and without explana- tions. We found that for the hybrid aid, explanation facilities significantly reduced users perceptions of its restrictiveness (mean = 3.82 versus 4.43, p < 0.05). However, the results for the AC and elimination aids were not significant. This result is consistent with Silver’s (1991a) prediction that when a user has more discretion in using a decision aid, guidance and
explanations will be more useful. The user of a hybrid aid has the discretion to decide whether each attribute preference is essential or nonessential. This renders the effects of explanation facilities provided by the hybrid aid as more salient than those provided by the AC or elimination aids.
Our prediction about the differences between the hybrid and AC aids was not supported. The participants perceived the two aids to be equivalent in terms of strategy restrictiveness, advice quality, and cognitive effort, thus showing that the strategy complementarity supported by the hybrid aid did not produce the hypothesized effects. In terms of perceived stra- tegy restrictiveness, a possible explanation is that perceptions of restrictiveness are mainly influenced by the lack of support of decision processes most favored by the user, rather than by the provision of additional support of decision processes deemed to be nonessential by the user. In this study, the lack of support for AC (i.e., the elimination aid) increased the restrictiveness perception, but the provision of elimination in addition to AC (i.e., the hybrid) did not reduce this perception.
Nevertheless, it must be noted that this study was conducted in the particular context of using decision aids for online decision making, in which the processing of user preferences along with product alternatives was automated by the aids. If the preference processing had not been automated, users would have experienced a higher cognitive load due to the need to evaluate a large number of product alternatives available online. In such a context, the strategy complemen- tarity of the hybrid aid might generate a stronger impact because it not only reduces the amount of alternatives to be evaluated but also retains those high quality alternatives.
A possible explanation of the equivalence of the AC and hybrid aids in terms of perceived advice quality is that the AC aid does not normally recommend unacceptable products to
310 MIS Quarterly Vol. 33 No. 2/June 2009
Wang & Benbasat/Interactive Decision Aids
the user among its top recommendations because such products normally have lower fit scores. Instead, the less desirable products are listed lower in the set of recommended products. As a result, although the AC aid provided the parti- cipants with some product recommendations with unaccep- table attribute values, which would be evaluated as having low quality, the participants might not have noticed or examined these products. The reason, as the well-known “cognitive miser” model suggests, is that to save effort, users will not examine all recommended products in detail (Beach 1993). We checked the choices of the participants assigned to the AC aid treatment and found that most participants (82.3 percent) chose a product from the first two pages of recom- mendations (which have relatively high fit scores), while few chose those products listed lower in the set of recommen- dations. Therefore, the AC aid to some extent does some of what the elimination does (albeit not via elimination) by treating product alternatives with unacceptable attribute values as low-level recommendations that are less likely to be examined. Consequently, the AC aid was perceived to be as good as the hybrid aid in terms of advice quality, thereby explaining why the complementarity effects of the two strategies on perceived advice quality did not emerge.
It is also worthwhile to point out that the number of products recommended by a decision aid varied and this could be a confounding factor. By its nature, the elimination strategy screens out those product alternatives that do not satisfy the attribute preferences set by the user. In contrast, the AC strategy does not intend to filter out any product alternatives, although the AC aid normally does not recommend products whose fit scores are very low. As a result, the number of recommendations returned by an AC aid is generally greater than that by an elimination aid. The number of recommen- dations by a hybrid aid falls between an elimination aid and an AC aid because, as specified by the user, the hybrid aid normally employs the elimination for some attributes and the AC for others.12 Therefore, the difference in the number of recommendations is integral to the strategy employed by decision aids, although this can also be influenced by the criteria levels set by the user during the user-aid dialogue.
Nevertheless, it follows that the number of recommendations may influence users’ perceptions about a decision aid, such as perceived cognitive effort. The examination of more recom- mendations will require more effort, assuming that users would examine them when more products are recommended by an aid. However, the assumption may not hold true
because the cognitive miser model suggests that to save effort, users will not examine more products even when they are recommended (Beach 1993). This was partially confirmed in this study because a majority of the participants chose a product from the first two pages of recommendations, as mentioned above. Therefore, the number of recommendations provided by a decision aid should not exert a significant impact on perceived cognitive effort.
In this study, we contended that instead of the number of recommendations, the number of iterations users went through when using a decision aid may largely determine the effort that they would spend. We reviewed the recorded videos from the experiment and found that, on the average, the number of iterations the participants went through was 4.6 times, 3.7 times, and 8.6 times for the AC, hybrid, and elimination aids, respectively. An iteration was counted as 1 when a participant used the aid to go through the user-aid dialogue page, answered the questions, obtained recommen- dations, and then made a decision. An iteration was also counted as 1 whenever the participants returned to the user- aid dialogue page, changed their answers, and obtained recommendations again before making a final decision. For a decision task, decision aids were iterated through repeatedly, and the elimination aid was used more times than the AC aid and the hybrid aid, leading to higher cognitive effort in using the elimination aid. This result confirmed the theoretical explanation presented in this study upon con- ception of the hypotheses.
We also further analyzed the potential impact of the number of recommendations returned by a decision aid on perceived advice quality, which was measured based on the participants’ product choices for the experimental tasks (i.e., whether or not participants accepted recommendations from the decision aid; if so, whether or not they accepted top recommendations that were shown on the first page of recommended products). We checked the possibility that a larger number of recom- mendations led the participants to more likely choose recom- mendations from later/ subsequent pages rather than those from the first page. If this were the case, the validity of the coding of advice quality would be threatened. In Table 10, we summarize the choices of the participants in the experi- ment. Due to the fact that the AC aid recommended the largest number of recommendations, whereas the elimination aid recommended the least, if the number of recommendations really influenced the participants’ product choices, it would follow that the participants using the elimination aid would most likely accept the top recommendations. On the other hand, those using the AC aid will least likely accept the top recommendations. The data in Table 10 show that this was not the case.
12 This was confirmed by the data in this study. On the average, the AC aid,
the elimination aid, and the hybrid aid recommended 48.9, 8.1, and 15.0 products to the participants, respectively.
MIS Quarterly Vol. 33 No. 2/June 2009 311
Wang & Benbasat/Interactive Decision Aids
Table 10. Distribution of Participants Choices for the Experimental Tasks
Following top recommendations (shown on the first
page)
Following non-top recommendations (shown
beyond the first page) Using search instead
of the decision aid Total*
Hybrid Aid 74 (72.5%) 13 (12.7%) 15 (14.7%) 102 (100%)
AC Aid 65 (62.5%) 31 (29.8%) 8 (7.7%) 104 (100%)
Elimination Aid 42 (44.7%) 21 (22.3%) 31 (33.0%) 94 (100%)
*Each participant finished two tasks. Fifty-two participants were assigned to each of the three decision aids. The choices of one participant using the hybrid aid and those of five participants using the elimination aid were missed (the experimental systems did not store all of the choices of the participants because there was a bug in the system which was later corrected). Therefore, the total number of choices for the hybrid aid and the elimination aid was not 104.
Moreover, we further analyzed the participants’ choices from the recommendations shown beyond the first page (i.e., they do not accept the top recommendations) to determine whether or not the larger number of recommendations “pushed” the participants to choose recommendations shown on the later pages. We found that for choices from recommendations shown beyond the first page, most of them were from pages 2 through 4.13 It was not evident that the participants were more likely to choose products from later pages when there were more pages of recommendations. Given these additional analyses, we believe that the differential number of recommendations returned by decision aids should not be a significant threat to the results of the study.
Contribution to Research
This study holds significant contributions to both research and practice. Its main contribution to research is threefold. First, in terms of theory, we successfully extended the effort– accuracy framework by taking into account the perceived strategy restrictiveness of online decision aids. This is impor- tant in a sense that previous research, in line with the effort– accuracy framework of cognition, has primarily focused on the benefits that can be delivered by decision aids, including the increase in decision quality and the reduction in cognitive effort. However, the negative impact of a decision aid caused
by its restrictiveness has been largely ignored. Recent IS studies have recognized the need to examine the negative impact of system features (or inhibitors) on the use of IT (e.g., Cenfetelli 2004). To the best of our knowledge, this is the first study to examine the role of perceived strategy restric- tiveness within the effort–accuracy framework of cognition. This study will help researchers better understand individuals’ online choice-making behaviors and their use and adoption of decision aids.
Second, we examine two underlying properties of decision aids that induce perceptual differences of the aids among users, namely, strategy normativeness and strategy comple- mentarity. Although these two mechanisms have been discussed in the behavioral decision-making literature (e.g., Elrod et al. 2004; Todd and Benbasat 2000), an empirical examination of their effects has not been attempted. We found support for the effects of normativeness, but not of complementarity. The implication of this is that although decision makers often employ hybrid strategies when making choices on their own, only the normative strategy (e.g., AC) is truly essential. Nevertheless, this does not mean that the hybrid strategies would not affect user behaviors or outcomes. Toward further clarification of this aspect, the impact of hybrid strategies deserves future research.
Third, we empirically confirm the role of explanation facilities in enhancing the perceptual evaluations of decision aids among users. Explanations promote the cognitive assessment of a decision aid by increasing understandability of the aid. This is achieved by delivering knowledge about a decision aid’s actions or functions, such as how it works and why its actions are appropriate (e.g., Dhaliwal and Benbasat 1996). In addition, explanations facilitate the behavioral assessment of a decision aid by helping users properly utilize its capabilities. Explanations, therefore, allow the desirable capabilities of a decision aid to be better conveyed to users,
13 In the hybrid aid treatment, the cumulative percentages of the participants’
choices selected from pages 1 through 4 of recommendations were 85.1%, 93.1%, 96.6%, and 97.7%, respectively; in the AC aid treatment, the four cumulative percentages were 67.7%, 82.3%, 89.6%, and 94.8%, respectively; and in the elimination aid treatment, the four percentages were 66.7%, 84.1%, 88.9%, and 93.7%, respectively. The choices from page 5 and lower pages accounted for between 2.3% and 6.3% for the three treatment conditions. The percentages were calculated based on all of the participants’ choices from the products recommended by the decision aids (those choices that were made by using the search query instead of the decision aids were excluded).
312 MIS Quarterly Vol. 33 No. 2/June 2009
Wang & Benbasat/Interactive Decision Aids
and as a consequence, users’ perceptions will more closely reflect the objective properties of the aid.
Implications for Practice
This study also has significant implications for practitioners. First, the results of the study suggest that the perceived restrictiveness of decision aids influences users’ intentions to adopt these aids. Accordingly, firms should provide less restrictive decision aids to improve online consumers’ deci- sion making. A review of the extant decision aids available on commercial websites shows that many of them are elimination based; however, this study’s results show that elimination aids do not produce desirable results. The per- ceived advice quality of the elimination aid was lower than that of the two other aids. It was the kind of decision aid that the participants perceived to be most restrictive. In contrast, the AC aid employs a normative strategy and is superior to the elimination aid in terms of perceived strategy restrictiveness, advice quality, and cognitive effort. As a result, users will be more willing to use it.14 As a design objective of decision aids is to promote users’ intentions to adopt the aids, firms are encouraged to provide AC aids rather than elimination aids.
Second, this study confirms that the perceived cognitive effort exerts higher effects on users’ intentions to adopt decision aids than perceived advice quality. The advice quality of a decision aid is largely determined by the choice strategy(ies) employed by the aid, whereas the cognitive effort of using the aid may be further reduced by improving the design of the aid interface. This study found that the participants iterated many times when using the decision aids, and therefore it might be desirable to reduce the effort required in adjusting users’ inputs to the aids. For example, the user may be allowed to adjust her inputs on the same page where recommendations are presented and after each time she adjusts her inputs, the recommendations from the aid should be automatically updated. In doing so, the user’s cognitive effort can be reduced. Of course, further empirical studies should be con- ducted to validate these designs.
Third, it is suggested that explanation facilities be embedded in a decision aid so that users can better understand the capabilities of the aid and apply them appropriately. An
examination of the current decision aid applications showed that many of them do not provide a full set of explanations. In the absence of such explanations, users may find it difficult to assess the aids and may be less likely follow the recom- mendations provided by the aids (Wang and Benbasat 2007).
Limitations and Future Research
Notwithstanding the contributions of this study, it has a number of limitations. First, as the experimental participants were university students, readers should exercise caution in generalizing the results of this study to other demographic groups. A more diverse sample would have offered greater opportunities for the generalization of the findings to various situations in which decision aids would be used. Further research is therefore suggested in this regard.
In addition, although the homogeneous sample of the parti- cipants with low product knowledge about digital cameras helped us deal with the potential bias and confounding that may be induced by product expertise, we were unable to examine the impact of product knowledge. The choice of decision rules and the use of explanation facilities may differ for users with a higher level of domain knowledge (Arnold et al. 2006). In this regard, the impact of product expertise on choice behavior in online decision making with decision aids merits future research.
The second limitation is that the study was conducted in a context in which the participants evaluated a decision aid in the early stage of their interaction with it. However, when users become more familiar with the aid, the factors that influence their post-adoption behavior may be different. For example, it is possible that the perceptions of the restric- tiveness of an aid may lessen when users become more accustomed to it, and as such, the effects of perceived restrictiveness on adoption intentions may also be lessened. Similarly, perceived advice quality may exert stronger effects, whereas the effects of perceived cognitive effort will be reduced. This notion echoes the findings in the literature that as the user gains more experience, the effects of perceived ease of use (similar to perceived cognitive effort) on inten- tions to use a technology decrease, whereas the effects of perceived usefulness (similar to perceived advice quality) increase (Taylor and Todd 1995). Future research is needed to further examine the relative importance of various factors on post-adoption perceptions and behaviors toward online decision aids.
Third, the present research focuses on two decision strategies that are suitable for online decision aids (in terms of norma-
14 In the experiment, we counted the number of participants who opted to use
the simple search query available on the homepage of the experimental website after trying the decision aid provided. We found that over 60% of these participants belonged to the elimination aid group. This indicates that many consumers will not follow the advice of the elimination aid at all, thus reducing the effectiveness of the decision support service.
MIS Quarterly Vol. 33 No. 2/June 2009 313
Wang & Benbasat/Interactive Decision Aids
tiveness, the AC and elimination are the two typical ends of high and low normativeness, respectively). The two strategies have been widely examined by empirical studies on prefer- ential choices. Nevertheless, there are other decision stra- tegies that may be employed by decision makers (Svenson 1979), and a comparison of decision aids that employ such strategies is thereby deserved.
In addition, perceived strategy restrictiveness can be affected by factors other than the decision strategy(ies) employed by an aid, such as the elicitation of users’ requirements (ques- tions asked by the aid) (Komiak and Benbasat 2006). Therefore, caution should be exercised in the interpretation of the magnitude of restrictiveness of the aids presented in this study. Moreover, the impact of other factors on perceived strategy restrictiveness warrants future research.
Fourth, we did not directly measure or elicit each participant’s preferred decision strategy(ies) in the experiment. Having such a direct measure might have allowed further analyses, and could have provided more insights on the formation of restrictiveness perceptions. For instance, we would have been able to judge whether restrictiveness perceptions are influ- enced by the match (or mismatch) between the user’s pre- ferred decision strategy(ies) and the decision processes that are supported by the aid. Further research exploring this issue is hereby suggested.
Finally, we investigated the role of explanation facilities in influencing user perceptions of strategy restrictiveness, advice quality, and cognitive effort. However, other factors may also influence the formation of users’ perceptual evaluations of a decision aid and accordingly influence the extent to which the perceptual evaluations differ from the aid’s actual properties. For example, product expertise and familiarity with a decision aid may influence a user’s ability to understand the aid and evaluate it properly (Swaminathan 2003). In addition, a priori expectations of the decision processes and func- tionalities supported by an aid may play a role because users’ evaluations are anchored, at least in part, on their a priori expectations (Szajna and Scamell 1993). These factors were not examined in this study, and therefore deserve future research.
Conclusion
Online firms are increasingly providing interactive decision aids to improve consumers’ ability to arrive at informed decisions. By examining consumers’ perceptions toward and use of these decision aids, we contribute to theoretical
advances in consumer decision making in e-commerce. Particularly, this research fills in the research gap in understanding the role of perceived restrictiveness in users’ intentions to adopt the aids by testing an extended effort– accuracy–restrictiveness framework. We also empirically compare three decision aids to test the role of strategy norma- tiveness and strategy complementarity in inducing perceptual differences in strategy restrictiveness, advice quality, and cognitive effort among users. In doing so, we advance the theory of behavioral decision making in the context of online choice-making supported by decision aids. Guidelines for promoting an effective provision of online decision support services for consumers are likewise provided.
Acknowledgments
The work described in this paper was partially supported by grants from the Natural Sciences and Engineering Council of Canada (NSERC) and City University of Hong Kong (Project No. 7200059). The authors would like to thank the senior editor, Bernard Tan, the associate editor, and three anonymous reviewers for their excellent comments and suggestions that have significantly improved the quality of this paper.
References
Ansari, A., Essegaier, S., and Kohli, R. 2000. “Internet Recom- mendation Systems,” Journal of Marketing Research (37:3), pp. 363-375.
Arnold, V., Clark, N., Collier, P. A., Leech, S. A., and Sutton, S. G. 2006. “The Differential Use and Effect of Knowledge-Based System Explanations in Novice and Expert Judgment Decisions,” MIS Quarterly (30:1), pp. 79-97.
Balzer, W., Doherty, M. E., and O'Connor, R. 1989. “Effects of Cognitive Feed-back on Performance,” Psychological Bulletin (106:3), pp. 410-433.
Barkhi, R. 2001-2002. “The Effects of Decision Guidance and Problem Modeling on Group Decision-Making,” Journal of Man- agement Information Systems (18:3), pp. 259-283.
Baron, J. 2004. “Normative Models of Judgment and Decision Making,” in Blackwell Handbook of Judgment and Decision Making, D. J. Koehler and N. Harvey (eds.), London: Black- well, pp. 19-36.
Baron, R. M., and Kenny, D. A. 1986. “The Moderator–Mediator Variable Distinction in Social Psychological Research: Con- ceptual, Strategic, and Statistical Considerations,” Journal of Personality and Social Psychology (51:6), pp. 1173-1182.
Beach, L. R. 1993. “Broadening the Definition of Decision Making: The Role of Prechoice Screening of Options,” Psycho- logical Science (4:4), pp. 215-220.
Beaulieu, M., and Jones, S. 1998. “Interactive Searching and Interface Issues in the Okapi Best Match Probabilistic Retrieval System,” Interacting with Computers (10:3), pp. 237-248.
314 MIS Quarterly Vol. 33 No. 2/June 2009
Wang & Benbasat/Interactive Decision Aids
Benbasat, I. 1989. “Laboratory Experiments in Information System Studies with a Focus on Individuals: A Critical Appraisal,” in The Information Systems Research Challenge: Experimental Research Methods, Volume 2, I. Benbasat (ed.), Harvard Business School Press, Boston, Massachusetts, 1989, pp. 33-47.
Bennet, J. L. 1983. “Analysis and Design of the User Interface for Decision Support Systems,” in Building Decisioin Support Systems, J. L. Bennett (ed.), Reading, MA: Addision-Wesley Publishing Company, pp. 41-64.
Bettman, J. R., Luce, M. F., and Payne, J. W. 1998. “Constructive Consumer Choice Processes,” Journal of Consumer Research (25), pp. 187-217.
Brehm, J. W. 1966. A Theory of Psychological Reactance, New York: Academic Press, 1966.
Brehm, S. S., and Brehm, J. W. 1981. Psychological Reactance: A Theory of Freedom and Control, New York: Academic Press, 1981.
Brunswik, E. 1952. The Conceptual Foundation of Psychology, Chicago: University of Chicago Press, 1952.
Burke, R. R. 2002. “Technology and the Customer Interface: What Consumers Want in the Physical and Virtual Store,” Journal of the Academy of Marketing Science (30:4), pp. 411-432.
Burns, E. 2006. “Worldwide Teen Generation Dubbed ‘SuperCon- nectors’,” The ClickZ Network, March 10 (http://www. clickz.com/showPage.html?page=3590801).
Carroll, J. M., and McKendree, J. 1987. “Interface Design Issues for Advice-Giving Expert Systems,” Communications of the ACM (30:1), pp. 14-32.
Cenfetelli, R. T. 2004. “Inhibitors and Enablers as Dual Factor Concepts in Technology Usage,” Journal of the Association for Information Systems (5:11-12), pp. 472-492.
Clee, M. A., and Wicklund, R. A. 1980. “Consumer Behavior and Psychological Reactance,” Journal of Consumer Research (6:4), pp. 389-405.
Dhaliwal, J. S. 1993. An Experimental Investigation of the Use of Explanations Provided by Knowledge-Based Systems, unpub- lished doctoral dissertation, University of British Columbia, Vancouver, Canada.
Dhaliwal, J. S., and Benbasat, I. 1996. “The Use and Effects of Knowledge-Based System Explanations: Theoretical Founda- tions and a Framework for Empirical Evaluation,” Information Systems Research (7:3), pp. 342-362.
Edwards, S. M., Li, H., and Lee, J.-H. 2002. “Forced Exposure and Psychological Reactance: Antecedents and Consequences of the Perceived Intrusiveness of Pop-Up Ads,” Journal of Advertising (31:3), pp. 83-95.
Elrod, T., Johnson, R. D., and White, J. 2004. “A New Integrated Model of Noncompensatory and Compensatory Decision Stra- tegies,” Organizational Behavior and Human Decision Processes (95:1), pp. 1-19.
Fasolo, B., McClelland, G. H., and Lange, K. A. 2005. “The Effect of Site Design and Interattribute Correlations on Interactive Web-Based Decisions,” in Online Consumer Psychology: Understanding and Influencing Consumer Behavior in the Virtual World, C. P. Haugtvedt, K. A. Machleit, and R. F. Yalch (eds.), Mahwah, NJ: Lawrence Erlbaum Associates, pp. 325-342.
Goldstein, W. M., Barlas, S., and Beatie, J. 2001. “Talk About Tradeoffs: Judgments of Relative Importance and Contigent Decision Behavior,” in Conflict and Tradeoffs in Decision Making, E. U. Weber, J. Baron, and G. Loomes (eds.), New York: Cambridge University Press, pp. 175-204.
Goodhue, D. L., and Thompson, R. L. 1995. “Task–Technology Fit and Individual Performance,” MIS Quarterly (19:2), pp. 213-236.
Green, P. E., and Srinivasan, V. 1990. “Conjoint Analysis in Marketing: New Developments with Implications for Research and Practice,” Journal of Marketing (54:4), pp. 3-19.
Gregor, S., and Benbasat, I. 1999. “Explanations from Intelligent Systems: Theoretical Foundations and Implications for Practice,” MIS Quarterly (23:4), pp. 497-530.
Grenci, R. T., and Todd, P. A. 2002. “Solutions-Driven Mar- keting,” Communications of the ACM (45:3), pp. 65-71.
Hair, J. F., Anderson, R. E., Tatham, R. L., and Black, W. C. 1998. Multivariate Data Analysis (5th ed.), Englewood Cliffs, NJ: Prentice Hall.
Häubl, G., and Trifts, V. 2000. “Consumer Decision Making in Online Shopping Environments: The Effects of Interactive Deci- sion Aids,” Marketing Science (19:1), pp. 4-21.
Hostler, R. E., Yoon, V. Y., and Guimaraes, T. 2005. “Assessing the Impact of Internet Agent on End Users’ Performance,” Decision Support Systems (41:1), pp. 313-323.
Jarvenpaa, S. L. 1989. “The Effect of Task Demands and Graphical Format on Information Processing Strategies,” Management Science (35:3), pp. 285-303.
Johnson, C. A. 2005. “The North American Consumer: Online Retail Update,” Forrester Research, Cambridge, MA.
Keeney, R., and Raiffa, H. 1976. Decisions with Multiple Objec- tives: Preferences and Value Trade-Offs, New York: John Wiley.
Kirk, R. E. 1995. Experimental Design: Procedures for the Behavioral Sciences (3rd ed.), Pacific Grove, CA: Brooks/Cole Publishing.
Komiak, S. Y. X., and Benbasat, I. 2006. “The Effects of Person- alization and Familiarity on Trust and Adoption of Recom- mendation Agents,” MIS Quarterly (30:4), pp. 941-960.
Leavitt, N. 2006. “Recommendation Technology: Will It Boost E-Commerce?,” IEEE Computer Society (39:5), pp. 13-16.
Maes, P. 1994. “Agents that Reduce Work and Information Overload,” Communications of the ACM (37:7), pp. 31-40.
Maes, P., Guttman, R. H., and Moukas, A. G. 1999. “Agents that Buy and Sell,” Communications of The ACM (42:3), pp. 81-91.
Mahoney, L. S., Roush, P. B., and Bandy, D. 2003. “An Investiga- tion of the Effects of Decisional Guidance and Cognitive Ability on Decision-Making Involving Uncertainty Data,” Information and Organization (13:2), pp. 85-110.
McSherry, D. 2005. “Explanation in Recommender Systems,” Artificial Intelligence Review (24:2), pp. 179-197.
Montgomery, H., and Svenson, O. 1976. “On Decision Rules and Information Processing Strategies for Choice Among Multi- attribute Alternatives,” Scandinavian Journal of Psychology (17), pp. 283-291.
Moore, G. C., and Benbasat, I. 1991. “Development of an Instru- ment to Measure the Perceptions of Adopting an Information
MIS Quarterly Vol. 33 No. 2/June 2009 315
Wang & Benbasat/Interactive Decision Aids
Technology Innovation,” Information Systems Research (2:3), pp. 192-222.
Nunnally, J. C., and Bernstein, I. H. 1994. Psychometric Theory, New York: McGraw-Hill.
Payne, J. W. 1976. “Task Complexity and Contingent Processing in Decision Making: An Information Search and Protocol Analysis,” Organizational Behavior and Human Performance (16), pp. 366-387.
Payne, J. W. 1982. “Contingent Decision Behavior,” Psychological Bulletin (92:2), pp. 382-402.
Payne, J. W., Bettman, J. R., and Johnson, E. J. 1992. “Behavioral Decision Research: A Constructive Processing Perspective,” Annual Review of Psychology (43), pp. 87-131.
Payne, J. W., Bettman, J. R., and Johnson, E. J. 1993. The Adaptive Decision-Maker, New York: Cambridge University Press.
Pereira, R. E. 2000. “Optimizing Human-Computer Interaction for the Electronic Commerce Environment,” Journal of Electronic Commerce Research (1:1), pp. 23-44.
Russo, J. E. 2002. “Aiding Purchase Decisions on the Internet,” in Proceedings of the Winter 2002 SSGRR (Scuola Superiore Guglielmo Reisss Romoli) International Conference on Advances in Infrastructure for Electronic Business, Education, Science, and Medicine on the Internet, L’Aquila, Italy, January 21-26.
Rust, R. T., and Kannan, P. K. 2003. “E-Service: A New Paradigm for Business in the Electronic Environment,” Communications of the ACM (46:6), pp. 37-42.
Silver, M. S. 1988. “User Perceptions of Decision Support System Restrictiveness: An Experiment,” Journal of Management Infor- mation Systems (5:1), pp. 51-65.
Silver, M. S. 1990. “Decision Support Systems: Directed and Non- directed Change,” Information Systems Research (1:1), pp. 47-70.
Silver, M. S. 1991a. “Decisional Guidance for Computer-Based Decision Support,” MIS Quarterly (15:1), pp. 105-122.
Silver, M. S. 1991b. Systems that Support Decision Makers: Description and Analysis, Chichester, UK: John Wiley and Sons.
Silvia, P. J. 2006. “Reactance and the Dynamics of Disagreement: Multiple Paths from Threatened Ffreedom to Resistance to Persuasion,” European Journal of Social Psychology (36:5), pp. 673-685.
Song, J., Jones, D., and Gudigantala, N. 2007. “The Effects of Incorporating Compensatory Choice Strategies in Web-Based Consumer Decision Support Systems,” Decision Support Systems (43:2), pp. 359-374.
Svenson, O. 1979. “Process Descriptions of Decision Making,” Organizational Behavior and Human Performance (23:1), pp. 86-112.
Swaminathan, V. 2003. “The Impact of Recommendation Agents on Consumer Evaluation and Choice: The Moderating Role of Category Risk, Product Complexity, and Consumer Knowledge,” Journal of Consumer Psychology (13:1/2), pp. 93-101.
Szajna, B., and Scamell, R. W. 1993. “The Effects of Information System User Expectations on Their Performance and Percep- tions,” MIS Quarterly (17:4), pp. 493-516.
Tan, C.-H. 2003. “Comparison-Shopping Websites: An Empirical Investigation on the Influence of Decision Aids and Information
Load on Consumer Decision-Making Behavior,” in Proceedings of the 24th International Conference on Information Systems, S. T. March, A. P. Massey, and J. I. DeGross, Seattle, Washington, December 14-17, pp. 1-14
Taylor, S., and Todd, P. 1995. “Assessing IT Usage: The Role of Prior Experience,” MIS Quarterly (19:4), pp. 561-570.
Todd, P., and Benbasat, I. 1991. “An Experimental Investigation of the Impact of Computer Based Decision Aids on Decision Making Strategies,” Information Systems Research (2:2), pp. 87-115.
Todd, P., and Benbasat, I. 1992. “The Use of Information in Decision Making: An Experimental Investigation of the Impact of Computer-Based Decision Aids,” Management Information Systems Quarterly (16:3), pp. 373-393.
Todd, P., and Benbasat, I. 1994a. “The Influence of Decision Aids on Choice Strategies Under Conditions of High Cognitive Load,” IEEE Transactions on Systems, Man and Cybernetics (24:4), pp. 537-547.
Todd, P., and Benbasat, I. 1994b. “The Influence of Decision Aids on Choice Strategies: An Experimental Analysis of the Role of Cognitive Effort,” Organizational Behavior and Human Decision Processes (60:1), pp. 36-74.
Todd, P., and Benbasat, I. 1999. “Evaluating the Impact of DSS, Cognitive Effort, and Incentives on Strategy Selection,” Info- rmation Systems Research (10:4), pp. 356-374.
Todd, P., and Benbasat, I. 2000. “Inducing Compensatory Infor- mation Processing through Decision Aids that Facilitate Effort Reduction: An Experimental Assessment,” Journal of Behav- ioral Decision Making (13:1), pp. 91-106.
Venkatesh, V., Morris, M. G., Davis, G. B., and Davis, F. D. 2003. “User Acceptance of Information Technology: Toward a Unified View,” MIS Quarterly (27:3), pp. 425-478.
Vessey, I., and Galletta, D. 1991. “Cognitive Fit: An Empirical Study of Information Acquisition,” Information Systems Research (2:1), pp. 63-84.
Wang, W., and Benbasat, I. 2005. “Trust in and Adoption of Online Recommendation Agents,” Journal of the Association for Information Systems (6:3), pp. 72-101.
Wang, W., and Benbasat, I. 2007. “Recommendation Agents for Electronic Commerce: Effects of Explanation Facilities on Trusting Beliefs,” Journal of Management Information Systems (23:4), pp. 217-246.
Wang, W., and Benbasat, I. 2008. “Attributions of Trust in Decision Support Technologies: A Study of Recommendation Agents for E-Commerce,” Journal of Management Information Systems (24:4), pp. 249-273.
Wilson, E. V., and Zigurs, I. 1999. “Decisional Guidance and End-User Display Choices,” Accounting, Management and Information Technology (9:1), pp. 49-75.
Xiao, B., and Benbasat, I. 2007. “E-Commerce Product Recom- mendation Agents: Use, Characteristics, and Impact,” MIS Quarterly (31:1), pp. 137-209.
Ye, L. R., and Johnson, P. E. 1995. “The Impacts of Explanation Facilities on User Acceptance of Expert Systems Advice,” MIS Quarterly (19:2), pp. 157-172.
316 MIS Quarterly Vol. 33 No. 2/June 2009
Wang & Benbasat/Interactive Decision Aids
About the Authors
Weiquan Wang is an assistant professor in the Department of Infor- mation Systems at the City University of Hong Kong. He received his Ph.D. in MIS from the University of British Columbia. His research interests include online consumer decision support, human– computer interaction, and e-commerce.
Izak Benbasat is a Fellow of the Royal Society of Canada and a CANADA Research Chair in Information Technology Management
at the Sauder School of Business, University of British Columbia, Canada. He received his Ph.D. in MIS from the University of Minnesota. He currently serves on the editorial board of Journal of Management Information Systems. He was editor-in-chief of Infor- mation Systems Research, editor of the Information Systems and Decision Support Department of Management Science, and a senior editor of MIS Quarterly. The general theme of his research is improving the communication between information technology, management, and IT users.
Appendix A
Experimental Decision Aids and Explanation Facilities
The experimental decision aids employed in the current research were designed to simulate those presented in other studies (Komiak and Benbasat 2006; Russo 2002; Wang and Benbasat 2007, 2008) and commercial applications. They utilized user-aid dialogues to elicit the product attribute preferences of users. For the AC aid, the participants were required to choose one of the options provided for each question and indicate the importance level of their choice on a nine-point scale (Figure A1-1). For the elimination aid, the participants needed to choose just one of the options for each question (Figure A1-2). For the hybrid aid, after choosing one of the options for each question in the user-aid dialogue, the participants were required to indicate whether their preference was “essential” or “nonessential.” When the nonessential button (i.e., AC strategy) was clicked, the participants were prompted to indicate an importance level (Figure A2-1), whereas when they clicked the “essential” button (i.e., the elimination strategy), they were not required to indicate the importance level (Figure A2-2).
Once the decision aid has elicited the requirements of the participants in the consultation dialogue, most of the participants would receive several pages of recommended products, with each page containing five recommendations (see Figure A3).
(1) AC Aid (2) Elimination Aid
Figure A1. User-Aid Dialogue
MIS Quarterly Vol. 33 No. 2/June 2009 317
Wang & Benbasat/Interactive Decision Aids
(1) Nonessential Chosen (AC) (2) Essential Chosen (Elimination)
Figure A2. User-Aid Dialogue with Strategy Chosen (Hybrid Aid)
(1) Recommendations on the first page (2) Recommendations on the second page
Figure A3. Recommendations from the Decision Aid
The elimination aid recommended the products that fit the requirements of the participants, whereas the AC aid and the hybrid aid provided products with a fit score over 70. The fit score of the alternative was deducted by the product of the importance weight and the gap between the user requirement and the level that the alternative can satisfy. The formula used to calculate the fit score is as follows:
Fit Score = 100 – (Importance_Weight × Attribute_Preference_Gap) i
r
=
1
where r refers to the number of questions in the user-aid dialogue. To simplify the calculation, three levels of Attribute_ Preference_Gap were used. When a product attribute (or feature) satisfies the user’s preference, the Attribute_Preference_Gap equals 0; otherwise it equals either 1 or 2 depending on the size of the gap between the attribute levels and user preferences. For example, if a user prefers a large-size LCD screen, then for cameras with an LCD screen of between 1.8 and 2.4 inches, the Attribute_Preference_Gap equals 1; for those with an LCD screen equal to or smaller than 1.8 inches, it equals 2; for those with an LCD screen size larger than 2.4 inches, it equals 0.
Prior to the main experiment, we conducted numerous pilot tests to determine the cut-off score. In most cases, users could receive many product recommendations with a fit score over 70. It was also confirmed with the participants during the pilot tests that products with fit scores below 70 very poorly matched their requirements. Therefore, the cut-off value of fit score was set to 70.
318 MIS Quarterly Vol. 33 No. 2/June 2009
Wang & Benbasat/Interactive Decision Aids
For the decision aids in the “with” explanation conditions, three buttons (i.e., a “How” button for how explanations, a “Why” button for why explanations, and a “Guidance” button for guidance) were provided for each question in the user-aid dialogue. When one of the buttons was clicked, the corresponding explanation was shown in the area below the “Explanation/Guidance” icon (see Figure A4 when the”How” button was clicked). Specific examples of the how explanations, the why explanations, and the guidance are all provided in Table A1.
For the hybrid aid, in addition to the three types of explanations for each question in the user-aid dialogues, guidance for choosing different strategies was provided. This was accessed by clicking the “Guidance on Non-Essential versus Essential” button. When it was clicked, the following guidance was shown. Note that for the hybrid aid, the guidance on choosing a different strategy was the same for all of the questions.
I will use different approaches to make recommendations based on your choice of “nonessential” or “essential” preference. If you want me to recommend only those cameras that exactly satisfy your desired choice to this question, then please select “essen- tial” preference. On the contrary, if you want me to recommend cameras that fit your overall preferences quite well but might not exactly satisfy your desired choice to this question, please select “nonessential” preference. Choosing “essential” preference or very high importance levels in the “nonessential” preference will significantly reduce the number of recommendations that I can provide.
For the decision aids in the “without” explanations conditions, the interfaces were similar to those in the “with” explanations condition, except that the “How,” “Why,” “Guidance,” and “Guidance on Nonessential versus Essential” buttons were not provided.
The three types of explanations were previously validated in Wang and Benbasat (2007), however, a similar pilot test was conducted in this study in an effort to assess the face validity and definitional accuracy of the explanations incorporated in the experimental decision aids. Eight graduate students majoring in MIS were asked to classify the explanations to be examined into one of the three types (i.e., how explanations, why explanations, and guidance) or none of them. In addition, to ascertain the level of certainty about their judgment, the participants were asked to indicate on a five-point scale the extent to which the explanation fitted the definition of the type of explanation chosen.
The validation results showed most of the explanations (96.6 percent of the how explanations, 81.8 percent of the why explanations, and 95.5 percent of the guidance) to be classified correctly. The certainty levels were high (average scores on a five-point scale: 4.6, 4.3, and 4.7 for the how explanations, why explanations, and guidance, respectively). The explanations thus appeared to be consistent with their definitions. The suggestions that were made during the pilot test regarding clarity of wording were incorporated into the explanations used for the main experiment. Note that differential effects of the three types of explanations are not addressed in this research because our primary research questions are not concerned with the relative effectiveness of different types of explanations. This issue has already been studied by Wang and Benbasat (2007).
Figure A4. Hybrid Arm; How Explanation Shown
MIS Quarterly Vol. 33 No. 2/June 2009 319
Wang & Benbasat/Interactive Decision Aids
Table A1. Examples of How Explanations, Why Explanations, and Guidance
Question in the User- Aid Dialogue
How far will you be from most of the subjects that you photograph? (1) Immediate vicinity. (2) A moderate distance or less. (3) Far away, in addition to immediate and moderate vicinity.
How Explanation Your distance from the subjects you want to focus on most often will determine the suitable zoom level of a digital camera. If you choose “nonessential” preference, cameras with your desired optical zoom level will be given higher priority in my recommendations; if you choose “essential preference,” I will only recommend cameras with your desired optical zoom level.
Specifically, the four options will determine the following zoom levels. (1) 2X optical zoom and below. (2) Between 2X and 4X optical zoom. (3) 4X optical zoom and above.
Why Explanation The purpose for asking this question is to know what kinds of photo you will take most often. It is quite useful to take photos at different distances. For example, for portraits of family and friends, your subjects may be close to your camera, but for many scenery or artistic photos, the subjects may be far from your camera.
Guidance Most digital cameras can take pictures beyond the immediate vicinity. However, cameras capable of taking pictures from very far away will be more expensive. At the same time, your choice will be more limited (only about 20% of cameras can focus on distant objects), so be careful not to over estimate your needs.
Appendix B
Measures
The following items used seven-point Likert scales with endpoints labeled “strongly disagree” and “strongly agree.”
• Perceived Strategy Restrictiveness (PSR) PSR1: I could select the way this virtual advisor processed my preferences in generating its recommendations. (dropped) PSR2: This virtual advisor allowed me to specify my preferred approach to generate recommendations. (reversed) PSR3: I had limited control over the way this virtual advisor made recommendations. PSR4: This virtual advisor constrained my choice of possible approaches it can use to generate recommendations. PSR5: In terms of my preferred way of selecting a digital camera, the approach this virtual advisor used to generate recommendations was
rigid. PSR6: In terms of my preferred way of selecting a digital camera, this virtual advisor’s reasoning processes for generating
recommendations were restricted.
• Perceived Cognitive Effort (PCE) PCE1: The task of selecting digital cameras using this virtual advisor was very frustrating. (dropped) PCE2: Using this virtual advisor, I easily found the information I wanted to help me decide what to buy. (reversed) (dropped) PCE3: The task of selecting digital cameras using the virtual advisor took too much time. PCE4: The task of selecting digital cameras using the virtual advisor was easy. (reversed) (dropped) PCE5: Selecting digital cameras using the virtual advisor required too much effort. PCE6: The task of selecting digital cameras using the virtual advisor was too complex.
• Intentions to Use the Decision Aid (INT) INT1: Assuming I had access to the system, I intend to use the virtual advisor. INT2: Assuming I had access to the system, I predict that I would use the virtual advisor. INT3: Assuming I had access to the system, I plan to use the virtual advisor.
320 MIS Quarterly Vol. 33 No. 2/June 2009