Plain Language Assignment

ACE.M

Chp8.pdf

Home >Reading homework help >Plain Language Assignment

Page 149

CHAPTER 8

Evaluating the Complete User Experience: Dimensions of Usability

Roger A. Grice

Much of the focus on usability and ease of use has centered around these questions, and certainly they are important ones—if people do not have access to accurate,

complete information that they can understand, they will not be happy. But is there more to usability than that? There is, and those concerned with providing

information—through printed and online documents, through training, and through interactive interfaces between people and computers—have broadened their

understanding of all that usability encompasses. To accompany this broadening of understanding, we must also broaden our views of how we test and evaluate usability.

In this chapter, I discuss several dimensions, or aspects, of usability and how they come into play when evaluating users’ experiences and satisfaction with using

products. I then take a look at how we might define tests and evaluations to obtain a more complete assessment of usability and develop plans for improving usability

than we might otherwise have done.

TWO STORIES

Here are two stories that involve my own reactions to computers and computer programs.

The first is about my first desktop computer. I received the computer at work and was very excited and eager to start using it. In those days, the

“Is it accurate?” ‘‘Is it complete?” Maybe even “Is it clear and understandable?”

Co py ri gh t @ 20 02 . Gr ee nw oo d Pu bl is hi ng G ro up .

Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he

pu bl is he r, e xc ep t fa ir u se s pe rm it te d un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO : eBook Collection (EBSCOhost) - printed on 7/1/2019 11:35 AM via UNIV OF HOUSTON - DOWNTOWN AN: 85818 ; Gurak, Laura J., Lay, Mary M..; Research in Technical Communication Account: s8478270.main.ehost

Page 150

software diskettes were shipped with the manuals, and both were packaged in a cardboard slip case surrounded by a stiff, glossy paper band that identified the

product. In my eagerness to get started, I put my finger through the plastic wrap that covered the package and ripped it off with one big sweeping motion.

Unfortunately, my finger ran against the edge of the stiff, glossy paper, and I got a very deep paper cut. The cut was painful, and the software package got smeared

with blood. Every time I used the software, the memory of the incident and the pain in my finger came back. The computer was a good one; the software was very

usable. But I was never comfortable using it—the memory of the pain got in the way.

The second is about the first software package I ever used to make slides. Again, I was excited and eager to use it. The software was preinstalled, so I clicked on its

icon to activate it. A sample slide appeared—done up in full, rich colors (coincidentally, with a background that was my favorite color) with a very attractive design. I

was impressed and wanted to jump right in and produce slides of my own that looked like that.

The moral of these stories? Both incidents made a strong impression on me and affected my perception, and subsequent use of the products to a very large degree.

Yet neither story reflects the types of issues we typically look for when evaluating usability. Usability as a wider scope than the mere sum of its component parts.

THE COMPLETE USER EXPERIENCE

Much of the work that has been done on assessing usability of products and documentation has focused on determining how well users can perform selected, key

tasks. We measure accuracy, success, and attitude toward the product and the tasks that were performed. Although this is certainly important—critical to success and

acceptance of products—it only tells part of the story.

A more realistic assessment of how usable a product is must take into account all the aspects of users’ involvement with the product, that is, their complete

experience. This assessment must take into account the ways that users first become aware of the product, their motivations and decision to purchase or use it, their

actual use of the product, and then their satisfaction and acceptance as people associated with the product and identified with its use.

Although this assessment is wide ranging and complex, it can more easily be defined and measured by looking at certain key aspects—or dimensions—of the

product and its use.

Co py ri gh t @ 20 02 . Gr ee nw oo d Pu bl is hi ng G ro up .

Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om

th e pu bl is he r, e xc ep t fa ir u se s pe rm it te d un de r U. S. o r ap pl ic ab le c op yr ig ht

la w.

Page 151

DIMENSIONS OF USABILITY

For purposes of this discussion, I identify six dimensions of usability that can be used to characterize people’s involvement with a product and assessment of their

satisfaction. These dimensions are

Although some of these dimensions may overlap with others, they are independent enough to give us a handle on assessing usability. I discuss the elements that

constitute each dimension and then move on to exploring techniques for assessing each.

Task Dimension

The task dimension measures people’s comfort and satisfaction with using the product to perform meaningful tasks. This dimension is concerned with how people do

tasks, how happy they are doing them, and their sense of closure for completing the task. Some measures of the task dimension are:

• Task dimension

• Motivation or goal dimension

• Product dimension

• Cognitive dimension

• Interactivity dimension

• Comfort dimension

• Comfort with task itself. At the core of assessing the task dimension is how comfortable people are with performing the task itself. They may feel comfortable

with the task because it is one with which they are familiar, because it is one that is presented and structured in a clear, logical manner, or because it is one for

which the product provides clear indications of successful progress for each step of the way. If people are not satisfied at this level, if they are uncomfortable

going through each step of the task, it is not likely that the product will be considered usable, no matter how many other good and positive features the product

possesses.

• Complexity of task. Tasks that are overly complex place high cognitive demands on users. If people are continually being forced to make complex decisions, to

work through involved sets of prerequisite and corequisite conditions, they will (rightly) perceive the task as complex. If the complexity is compounded by

inadequate or confusing directions, prompts, and feedback, any complexity inherent in the task will be compounded.

Co py ri gh t @ 20 02 . Gr ee nw oo d Pu bl is hi ng G ro up .

Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt

fa ir u se s pe rm it te d un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

Page 152

Motivation or Goal Dimension

The degree to which people are motivated—or unmotivated—to use a product to perform their work will also affect the satisfaction with the product. If product

designers and information designers are aware of the potential level of motivation, they can design the product and its interface to take advantage of this factor. Level of

motivation can affect:

• Frustration level inherent in task. Some tasks are inherently frustrating, making it unlikely that a product will score very highly in the task dimension. For example,

I do enjoy filling out and filing my tax forms. Some of the software packages that have been developed simplify the task greatly and do a very good job of

removing some of the frustration, but the work involved in locating, collecting, and sorting the tax records still makes the task very frustrating for me. The tax

preparation packages seemed to be designed with an awareness of the inherent frustration level, and help to work through it. However, if the designers were not

aware of the inherent frustration, or did not address it well, working with taxpreparation products could increase the frustration level already associated with the

task, compounding the problem.

• Rewards for completing the task. Some tasks have rewards associated with them. When we write a memo or other document, for example, we can print it out

and get a tangible measure of success: We get to hold the printed document in our hands. When we order products over the Internet, we get the products that

we ordered. Rewards such as these can increase our satisfaction with the task we performed. However, if the task has no associated reward, or an award we do

not value highly (such as, in most cases, cleaning up the hard drives on our computers), we may not receive much sense of satisfaction. Similarly, if we do receive

some payback but are not clearly made aware that we are receiving it, we may not get a great sense of satisfaction.

• Willingness to overcome obstacles. People who are highly motivated to use a product will work through obstacles placed in front of them: a poorly designed

interface, poor information, or lack of sufficient and meaningful feedback, for example. Designers working to this audience may have a relatively easier job

because the users will ignore deficiencies or, at least, not place too much weight on them. However, people who are not motivated to use the product will see

obstacles and defects as a reason to stop using the product or to complain about its design and functioning. They will not have a pleasant or rewarding

experience with the product.

• Rewards and penalties. People who are highly motivated to use the product to achieve their goals may be less dependent on rewards—compli

Co py ri gh t @ 20 02 . Gr ee nw oo d Pu bl is hi ng G ro up .

Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he

pu bl is he r, e xc ep t fa ir u se s pe rm it te d un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

Page 153

Product Dimension

Some products are lucky enough to have a base of customers who truly appreciate them, who will sing their praises to the world, and who enjoy the time they spend

using them. A high rating on the product dimension will certainly enhance the complete user experience. In some instances, users are so enthralled with a product that

they will not only overlook flaws and obstacles, but they also may remain unaware that they exist at all. Some factors contributing to the product dimension are:

mentary messages, for example—for their satisfaction; achieving their goals may be enough. They may not be overly concerned by penalties such as error

messages or the need to reenter small amount of data. They will most likely not be happy with the penalties, and their experience will be less pleasant and

rewarding than it might otherwise be, but they will be less daunted than lessmotivated users might be.

• Immediacy of reward. The lower the user’s motivation, the more important it is to provide rewards, confirmation of success, and status of work in a very timely

manner. Delays in providing this information will be tolerated less well by users with low motivation than by users with high motivation.

• Mental attitude. Highly motivated users will have a more positive mental attitude towards a product, its interface, and its performance than will users with less

motivation. Designers can take advantage of any advance knowledge of mental attitude by helping those with a negative attitude to overcome that attitude, if

possible.

• Opinion of product. People may form their opinion of a product based on their own experience and judgment or based on the experience and judgment of those

whose opinion they respect. In general, the higher their opinion of the product, the more positive is the complete user experience—provided that the product lives

up to expectations. If the product, or use of the product, does not live up to expectations, however, users may well be more frustrated and unhappy than they

would have been had there been no prior expectation.

• Perceived status of product. If a product is perceived as being a highstatus product, users may achieve an additional degree of satisfaction: the satisfaction of

being seen with, or associated with, a winner. (This may be why people often drive around aimlessly, just to be seen, when they get a new car.)

• Prior experience with product. People’s prior experiences with a product carry over, in some form, to their current use of the product—the “halo effect.” If their

prior experiences have been pleasant and productive, they

Co py ri gh t @ 20 02 . Gr ee nw oo d Pu bl is hi ng G ro up .

Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om

th e pu bl is he r, e xc ep t fa ir u se s pe rm it te d un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

Page 154

Cognitive Dimension

The amount of thinking and recall of facts affects user satisfaction with a product. Although it is usually necessary to do some thinking and to recall information stored in

people’s memories, the demands are sometimes higher than they need be. Asking users to enter a piece of information more than once is wasteful and annoying. People

assume—quite rightly—that if they have entered information (e.g., their name or address), the information should be stored somewhere and available for use. They have

the right to feel imposed upon if they are asked to enter the same information a second, or third, time. Some other factors that place an unnecessary cognitive load on

people are:

will anticipate a similar experience during their current use. However, if their experiences have not been pleasant, they will approach their current use of the

product with apprehension and low expectations. Although there is generally little opportunity to change this attitude during the current version of a product,

shows of good faith and concrete promises of improvement can eliminate some of the negatives. And certainly steps should be taken in subsequent releases to

dim the bad memories—by ensuring that the offending characteristics are fixed and that the improvements are highlighted.

• Experience with similar products. Just as prior experiences with a product have their effect on current use, so too can experiences with similar products leave an

impression on the current use of a product. If it is known that users may have had unfavorable prior experiences with a similar product, highlighting the differences

and positive characteristics can help.

• Demands on longterm and shortterm memory. The process of retrieving information from longterm memory is not the same as the process of retrieving

information from shortterm memory. But people are often asked to mix the two processes during their interaction with a computer system. This can be both

tiring and confusing. A more usercentered approach would be to group the requests for information into long term and short term. For example, mixing

questions about the current time, date, and transaction number with questions about date of birth, how long at the current address, and year graduated from

school can be taxing. But if the requests are booked by type, the cognitive demand may be less.

• Analytic versus spatial thinking. Similarly, asking people to provide information in such a way that they are required to mix analytic thinking (often referred to as

“left brain”) with spatial or relational thinking (often called ‘‘right brain”) can increase the cognitive load and cause confusion.

Co py ri gh t @ 20 02 . Gr ee nw oo d Pu bl is hi ng G ro up .

Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om

th e pu bl is he r, e xc ep t fa ir u se s pe rm it te d un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

Page 155

Interactivity Dimension

The amount of interaction that people have with a product, the amount of control that they have—or appear to have—over the interaction, and the clarity and comfort

of that interaction play a large role in determining how satisfied people are with the use of a product. Some items that shape the interactivity dimension are:

Again, grouping the information requests by type of information requested can be helpful in making the user’s experience less stressful and more pleasant and

productive.

• Amount of tedium. Just as requiring too much thinking can make product use unpleasant, so too can requiring people to go through tedious, almost mindless

actions for an extended period of time result in decreased satisfaction and productivity and, possibly, an increase in errors due to lack of attention and interest.

• Control of dialogue. Dialogues between people and computers can be structured in one of two basic modes: the person can appear to be in charge, or the

computer can appear to be. If people are in control, they issue commands or make requests of the computer and receive responses to those commands or

requests. If the computer is in control of the dialogue, users are asked questions or are asked to supply information, which can be accepted or not, depending on

the way the program is designed to accept responses. In many, but not all, cases, people are happier if they feel they are in control.

• Clarity of dialogue structure and requirements. Dialogues with computers, like dialogues with people, can be clear, logical, and meaningful, or they can be

otherwise. And people are generally happier with the former than with the latter. Although many factors contribute to the clarity of a dialogue and the

requirements that dialogues place on people, people can generally make an overall assessment of how satisfied they are with the dialogue. Although they may not

be able to express their assessment in quantifiable terms, they can usually be quite clear, and expressive, in their overall assessment.

• Amount and pace of interaction. Again, as with dialogues with people, dialogues with computers can vary widely in the amount and pace of interaction. If people

are required to deal with too much interaction, they may find it tiresome or redundant; if there is too little, they may lose interest or find the interaction to be not

satisfying. Interaction that is too slow may become boring, and people will lose interest; interaction that is too fast paced may be stressful, making people feel that

they must respond more rapidly than they would like to.

Co py ri gh t @ 20 02 . Gr ee nw oo d Pu bl is hi ng G ro up .

Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n

fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed u nd er U .S . or a pp li ca bl e

co py ri gh t la w.

Page 156

Comfort Dimension

The final dimension of usability to be discussed is comfort—how comfortable, relaxed, and “at home” users feel while working with a computer or computer

application. Two aspects of comfort are mental comfort and physical comfort.

COLLECTING DATA ABOUT THE COMPLETE USER EXPERIENCE—TRADITIONS WE CAN DRAW

FROM

Often when we think of testing or evaluating the usability of information and products, we think of one evaluation method—often the one we are most familiar with, or

the one that we have been told is “the one to use.” And even though triedand true methods can be useful and effective, we might do well to widen our repertoire and

consider a variety of traditions from which to draw. For example, we might consider:

• Tone of interaction. The tone of the interaction is a reflection of the perceived “personality” of the computer and real personality of the user. If the computer is

perceived as rude, condescending, inaccurate, or irrelevant, most users will not consider the dialogue to be a success.

• Mental comfort. Mental comfort can be measured as the extent to which people feel familiar with what they are doing and the degree of confidence or certainty

they have in their actions. If the actions that people must take to use a program are familiar to them—because they are like actions they would take in the

noncomputer world or because they are like actions they would take when using other computer programs—people will feel comfortable because they do not

need to make stressful decisions about what actions to take or how to interpret responses. If the choices they must make are clear and if the feedback and

responses they receive indicate clearly what has happened or what they must do, users will feel confident in interpreting them and acting upon them.

• Physical comfort. Physical comfort can be measured by observing how people must move and act physically to use a computer or a computer application.

Factors to be considered include how well, or poorly, the screen displays are designed to prevent eyestrain, the complexity of key strokes or mouse clicks

required to do the work, and the likelihood of repetitive motion injuries. And even though this aspect of use is often outside the realm of many interactivity design

sessions, it can be absolutely important to people who are faced with physical discomfort while trying to perform computerbased tasks.

Co py ri gh t @ 20 02 . Gr ee nw oo d Pu bl is hi ng G ro up .

Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he

pu bl is he r, e xc ep t fa ir u se s pe rm it te d un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

Page 157

Each of these traditions stresses a viewpoint, or set of related viewpoints, and lends itself most readily to a set of evaluation methods. Some of the traditions are, in fact,

at odds with each other. The skillful and conscientious usability evaluator will select from among the methods suggested by these traditions to obtain as full and accurate

an assessment of usability as possible.

Drawing on the Tradition of Cognitive Psychology

Much of the work of cognitive psychologists is clinical, based on experiments conducted in a research laboratory. Great attention is paid to experimental design, so that

experiments are “clean” and worth reporting to peer groups. Repeatability of experiments is of high priority, so that the experiments can be duplicated or extended to

verify or extend the results obtained. Results are generally analyzed using statistical methods, and achieving statistically significant results is a major goal.

Drawing on this tradition, usability evaluators could conduct highly structured usability evaluations in usability laboratories, carefully measuring and recording how

people perform specified scenarios of typical applications of product use.

Drawing on the Tradition of Anthropology/Ethnography

Work done in these traditions involves making neutral observations of people in their own environments; care must be given to causing as little disruption as possible to

the way that work is done. Anthropologists and ethnographers work to observe actions in natural, or ‘‘normal,” settings and place importance on observing and

recording subjects’ methodology and terminology.

Drawing on this tradition, evaluators could visit users at their place of work and observe them as unobtrusively as possible performing the tasks that they usually

perform in the manner and sequence in which they usually perform them.

• Cognitive psychology

• Anthropology/ethnography

• Software engineering

• Market research

• Language and literature

• Technical communication

Co py ri gh t @ 20 02 . Gr ee nw oo d Pu bl is hi ng G ro up .

Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er ,

ex ce pt f ai r us es p er mi tt ed u nd er U .S . or a pp li ca bl e co py ri gh t la w.

Page 158

Drawing on the Tradition of Software Engineering

Software engineers have worked to make the processes used to design, develop, and test software as efficient and effective as possible. Taking a cue from them,

usability evaluators can work to ensure that usability activities are integrated with other development activities so that the results obtained from usability evaluation can

be incorporated in a timely manner into the product being evaluated. There is an emphasis on engineering methodology and practice and on the timing of usability

evaluation activities and application of results.

Drawing on this tradition, evaluators would make sure that their evaluation activities are done early enough so that the results obtained can be applied to the product

while it is under development, rather than needing to hold them until the next version of the product is being worked on. This tradition also implies that the most efficient

and effective methods for each type of evaluation will be used appropriately.

Drawing on the Tradition of Market Research

Specialists in marketing products have concern for adoption and use products by a target audience. Their focus is on ensuring that the products conform to the

requirements that have been set for them, usually by discussing with potential users their needs and requirements. The major emphasis of this orientation is feedback

from users and improvement of a product by use of that feedback.

Drawing on this tradition, evaluators would make sure that they understand fully the needs of their target audience and would then make sure that whatever is

designed and developed truly meets those needs. Strong emphasis would be placed on having continuous communication with users and a high level of user

participation in design and evaluation of the product.

Drawing on the Tradition of Language and Literature

Those who focus on the study of language and literature have developed skills and techniques for analyzing language and meaning—ways of expressing thoughts, ways

of understanding texts. They focus on the structure and presentation of information and how structure and presentation can affect meaning and perception of quality,

usability, and validity. They have developed numerous techniques for achieving clarity of expression and ways of providing the proper notion of clarity for a wide range

of audience and environment.

Co py ri gh t @ 20 02 . Gr ee nw oo d Pu bl is hi ng G ro up .

Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he

pu bl is he r, e xc ep t fa ir u se s pe rm it te d un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

Page 159

Drawing on this tradition, usability evaluators could focus on clarity of expression and analytic techniques for structuring information presentation.

Drawing on the Tradition of Technical Communication

Those who practice or understand the theories and practice of technical communication focus heavily on understanding of, and concern for, audience. They have

developed techniques for producing clear, effective communication and for infusing clarity into information that is convoluted or obscure. Technical communication is

focused on audience and helping the audience perform their required tasks.

Drawing on this tradition, usability evaluators can work to ensure that all information is presented clearly and is structured and developed towards its stated purpose.

PULLING IT ALL TOGETHER—MAKING A WIDERANGING ASSESSMENT TO EVALUATE THE

COMPLETE USER EXPERIENCE

Using the dimensions of usability as a structure for performing a usability evaluation, we can draw on the range of usability techniques suggested by the various traditions

just discussed to put together a comprehensive evaluation plan, a plan to make a wideranging assessment of the complete user experience rather than a more narrowly

focused assessment of selected key tasks. This section brings the two previous sections together by following the analysis of the slidemaking software package

mentioned at the beginning of this article. The analysis covers the purchase, installation, and use of the package.

Although a specific product is not to be used as the example, the analysis includes uses of the methods of the various traditions just described to measure and

evaluate the factors that make up the dimensions of usability described at the beginning of this chapter.

A thorough analysis of the case needs to include the time before users come into contact with (or ownership of) the application package. For example, in the case of

my personal introduction to a slidemaking program, for some time before I purchased the package, I had admired the slides that others used at conferences and

wished that I could do the same.

A complete evaluation of my experience with the product would start with my first awareness of the product and the way I could use it, my decision to buy,

installation of the product, initial use, and subsequent use as

Co py ri gh t @ 20 02 . Gr ee nw oo d Pu bl is hi ng G ro up .

Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he

pu bl is he r, e xc ep t fa ir u se s pe rm it te d un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

Page 160

a more experienced user. We could do a similar analysis of other users whom we intended to study.

DETERMINING WHAT TO MEASURE AND HOW TO MEASURE

Measuring across the Task Dimension

For this dimension, we consider comfort with task itself, complexity of task, frustration level inherent in task, and reward for completing the task. We can measure this

dimension by asking questions such as:

To assess usability across the task dimension, we can use a mixture of quantitative and qualitative methods. For example, to assess how easy it is to install the product

and get started, we can collect data on time to install, time to bring up the first screen, and time to start filling in information. We can count the number of errors made,

the number of times that the manual or help system was consulted, or the number of times people redid work that was already completed because they were uncertain.

We could make quantitative assessments of complexity and frustration by measuring indirect indicators of complexity and frustration (references to manuals or help

systems, number of times users closed the application and restarted it rather than determining a cause of error).

We could also measure complexity and frustration qualitatively by observing users’ looks of anger, frustration, and bewilderment. We could record or write down

comments they make that indicate their feelings towards using the product. We could give them a posttask survey or questionnaire asking them to describe their

satisfaction with the way the application program worked for them.

And depending on what types of information we were looking for or needed to report, we could take quantitative measures using a Likert scale (“on a scale of 1 to

7, how satisfied…”) or qualitative measures (“describe what you thought when…”). Methods such as these are familiar to cognitive or behavioral scientists working in

laboratories and to anthropologists and ethnographers working in the field. Increasingly, these are becoming familiar to technical communicators and usability engineers.

• How easy is it to install the product and get started with it?

• How complex is it to develop individual slides and put the slides together into presentations?

• How frustrating is it to develop slides and presentations?

Co py ri gh t @ 20 02 . Gr ee nw oo d Pu bl is hi ng G ro up .

Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he

pu bl is he r, e xc ep t fa ir u se s pe rm it te d un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

Page 161

Measuring across the Motivation or Goal Dimension

For this dimension, we consider willingness to overcome obstacles, rewards and penalties, immediacy of reward, and mental attitude. We can measure this dimension

by asking questions such as:

Assessments across this dimension encompass users’ wants and feelings before they purchase or start to use a product, as well as their feelings and observations while

using the product. For example, if someone had observed others using wellmade, effective slides at conferences and meetings and wanted to be able to do the same,

they would be highly motivated to obtain the slidemaking program and learn to use it well. They would equate their ability to use the program with their ability to make

effective presentations and appear as a true professional in front of a group. Market researchers and advertising specialists have honed their skills at making products

seem appealing to potential audiences and at measuring the effectiveness of advertising and publicity campaigns. They often supply the motivation for people to want

products, and, to some extent, sales or licensing are one measure of motivation.

Measuring across the Product Dimension

For this dimension, we consider users’ opinion of the product, perceived status of the product, users’ prior experience with the product, and users’ experience with

similar products. We can measure this dimension by asking questions such as:

Much of the information used to measure this dimension is qualitative data obtained by methods familiar to anthropologists and ethnographers. Methods of contextual

inquiry are gaining in popularity as people become more familiar with techniques and applications (see, e.g., Beyer and Holtzblatt, 1997).

• Did the prospect of having a slide presentation for their own use make the effort worthwhile?

• How quickly could users see the finished product or drafts of it?

• Did users feel that they were reaching their goal as they developed the presentation?

• What did users think about the product and what it could do for them?

• What did users think about other people who were users of the product?

• Have they watched others use the product to develop slide presentations?

• How had they developed slide presentations in the past? C op yr ig ht @ 2 00 2. G re en wo od P ub li sh in g Gr ou p.

Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er ,

ex ce pt f ai r us es p er mi tt ed u nd er U .S . or a pp li ca bl e co py ri gh t la w.

Page 162

The importance of users’ perception of the product or application they are using cannot be overemphasized—it can color their whole experience. If the product is

perceived to be “a winner,” then those who use it are often given the same status. Measures for this dimension tend to be subjective—perceptions of status,

expectations for improved performance and recognition, and a sense of belonging are key. Information can be obtained through surveys, focus groups, and

observations.

Measuring across the Cognitive Dimension

For this dimension, we consider demands on longterm and shortterm memory, spatial versus analytic thinking required, and amount of tedium involved. We can

measure this dimension by asking questions such as:

Measures across this dimension can be quantitative or qualitative. By observing users in action, we could observe quantitative measures such as: how often they needed

to refer to reference information rather than being able to recall it from memory, how often they needed to refer back to the same information, or how often they

appeared lost, even though they had been exposed to the information they needed. We can obtain qualitative measures of cognitive load—perhaps more useful for

assessing usability—by asking them to explain their thought processes and how easy or difficult those processes seemed to be.

Measuring across the Interactivity Dimension

For this dimension, we consider user control of dialogue, clarity of dialogue structure and requirements, amount and pace of interaction, and tone of interaction. We can

measure this dimension by asking questions such as:

• Did users have to remember a lot of procedural and content information to develop slides?

• Could they separate the tasks of content development and formatting? Or did they need to do both simultaneously?

• Did slide development become boring?

• Did users feel like they were in charge of dialogues and activities?

• Did they usually know what they had to do next? If not, could they find out easily?

• Were they made to feel that they had done something dumb or wrong when slides did not look the way they had planned?

Co py ri gh t @ 20 02 . Gr ee nw oo d Pu bl is hi ng G ro up .

Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er ,

ex ce pt f ai r us es p er mi tt ed u nd er U .S . or a pp li ca bl e co py ri gh t la w.

Page 163

Few factors promote—or detract from—a product’s usability as does the feeling users have that they are in control of their interaction with an application or that the

application is in control. This sense is subjective, and a structure that may make some users feel in control may well make others feel that they are being controlled,

perhaps even manipulated. There is little about this dimension that can be measured quantitatively, short of analysis of dialogue structure, but that measure only

examines formal structure of a dialogue; it does not measure users’ sense of it.

The most useful measurements for this dimension are subjective and are obtained by talking with users—either directly or through focus groups and surveys.

Measuring across the Comfort Dimension

For this dimension, we consider mental comfort and physical comfort. We can measure this dimension by asking questions such as:

Physical comfort can often be measured by applying what we have learned from ergonomics—by measuring distances between eyes and computer screen, position of

head and hands, and so on. But, once again, the true measures of this dimension are subjective and individual. Do people feel comfortable? Did they feel anxious or

confused? This information is best obtained by talking with people, often coaxing them to reveal sources of anxiety or discomfort that they might otherwise gloss over.

COMBINING FINDINGS INTO A UNIFIED SET

Other dimensions of usability might well be defined. I find the set described in this chapter to be useful. The dimensions identified cover a sufficiently wide range of

usability concerns to make a total assessment possible. On the other hand, the set is small enough to be manageable when trying to make an assessment of a product’s

usability that encompasses more than just the performance of major tasks. Using this set of dimensions, we can explore users’ total experience with a product—from

the time they first become aware of a product or a need for the product, through learning and use of that product.

The range of needed findings requires a range of methods. It is generally not sufficient to pick a favorite technique or two and rely only on them, to the exclusion of all

other methods. The wise usability evaluator picks

• Were users anxious, concerned, or confused?

• Did they develop hand or eye strain while developing the slides?

Co py ri gh t @ 20 02 . Gr ee nw oo d Pu bl is hi ng G ro up .

Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he

pu bl is he r, e xc ep t fa ir u se s pe rm it te d un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

Page 164

and chooses from available tools and techniques to learn the most about a user’s complete experience as it is possible to learn.

CONCLUSIONS

Evaluating usability of products, interfaces, and documents involves more than applying a few simple rules; it is not an activity that can be done thoroughly, as if by

following a cookbook. There are a variety of evaluation methods available—some simple and inexpensive, others more elaborate and time consuming. Each has its

place in an overall evaluation process; each can be used to advantage to increase usability.

The skillful usability evaluator will chose wisely from the methods available, not limiting the evaluation process to a narrow subset of what is available. By using the

methods in combination and to best advantage, evaluators can improve not only selected aspects of usability, but the users’ complete experience, from start to finish.

REFERENCES

Barker, T. T. (1998). Writing software documentation: A taskoriented approach. Needham Heights, MA: Allyn & Bacon.

Beyer, H., & Holtzblatt, K. (1997). Contextual design: A customercentered approach to systems design. San Francisco, CA: Morgan Kaufmann Publishers.

Carroll, J.M. (Ed.). (1998). Minimalism beyond the Nurnberg funnel. Cambridge, MA: MIT Press.

Dumas, J.S., & Redish, J.C. (1994). A practical guide to usability testing. Norwood, NJ: Ablex Publishing Corp.

Hackos, J.T., & Redish, J.C. (1998). User and task analysis for interface design. New York: John Wiley & Sons, Inc.

Hargis, G., Hernandez, A.K., Hughes, P., Ramaker, J., Rouiller, S., & Wilde, E. (1998). Developing quality technical information. Upper Saddle River, NJ:

PrenticeHall.

Nielsen, J., & Mack, R.L. (Eds.). (1994). Usability inspection methods. New York: John Wiley & Sons, Inc.

Rubin, J. (1994). Handbook of usability testing. New York: John Wiley & Sons, Inc.

Co py ri gh t @ 20 02 . Gr ee nw oo d Pu bl is hi ng G ro up .

Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om

th e pu bl is he r, e xc ep t fa ir u se s pe rm it te d un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .