Week 6 updated

Chapter 15

Questionnaire Design

AT-A-GLANCE

I. Questionnaire Quality and Design: Basic Considerations

A. What should be asked?

B. Questionnaire relevancy

C. Questionnaire accuracy

II. Wording Questions

A. Open-ended response versus fixed-alternative questions

· Using open-ended response questions

· Using fixed-alternative questions

B. Types of fixed-alternative questions

C. Phrasing questions for self-administered, telephone, and personal interview surveys

III. Guidelines for Constructing Question

A. Avoid complexity: Use simple, conversational language

B. Avoid leading and loaded questions

C. Avoid ambiguity: Be as specific as possible

D. Avoid double-barreled items

E. Avoid making assumptions

F. Avoid burdensome questions that may tax the respondent’s memory

G. Make certain questions generate variance

IV. What Is the Best Question Sequence?

V. What Is the Best Layout?

A. Traditional questionnaires

B. Internet questionnaires

· Layout issues

· Software that makes questionnaires interactive

VI. How Much Pretesting and Revising Are Necessary?

VII. Designing Questionnaires for Global Markets

IX. Appendix 15A: Question Wording and Measurement Scales for Commonly Researched Topics

LEARNING OUTCOMES

1. Explain the significance of decisions about questionnaire design and wording

2. Define alternatives for wording open-ended and fixed-alternative questions

3. Summarize guidelines for questions that avoid mistakes in questionnaire design

4. Describe how the proper sequence of questions may improve a questionnaire

5. Discuss how to design a questionnaire layout

6. Describe criteria for pretesting and revising a questionnaire and for adapting it to global markets

CHAPTER VIGNETTE: J.D. Power Asks: It’s Interesting, But Do You Really Want It?

As car makers consider adding new features, they have to evaluate not only which ones appeal to consumers but also which ones will actually sell, considering their likely cost. In a J.D. Power survey, about 17,000 consumers were asked whether they were familiar with 22 different emerging technologies and their interest in each technology. Next, the study indicated the likely price of each technology, and consumers were asked their interest, given the price. Learning price information often changed consumers’ interest levels, but a few features remained in the top five (run-flat tires and stability control). Automakers can use findings such as these to determine which features are price-sensitive and which might be appealing even at a higher price.

SURVEY THIS!

Students are asked to critique the survey instrument from a standpoint of the learning objectives of this chapter. Comment on the order of questions or on the presence of any leading or double-barreled items. What are the strengths and weaknesses of the questionnaire?

RESEARCH SNAPSHOTS

· Corporate Reputations: Consumers Put Johnson & Johnson, Microsoft, and Google on Top

Since 1999, the Wall Street Journal has sponsored an annual research project, the Corporate Reputation Survey, to see what U.S. adults think about corporations. The study has two phases. In the first phase, researchers identified the most “visible” companies by using open-ended questions asking respondents to name two companies they felt had the best reputation and two that had the worst. The researchers selected the 60 named most often for the second phase of the study. The second phase was aimed at generating rankings of the corporations, so the researchers identified six dimensions of corporate reputation: products and services, financial performance, workplace environment, social responsibility, vision and leadership, and emotional appeal. Within each dimension, 20 attributes were identified, and respondents in an online survey were asked to rate one company an all 20 attributes. The top-ranked company for each of the first seven years was Johnson & Johnson. J&J also was the firm from which the largest share of people said they would “definitely purchase” products.

· What Citizens (Don’t) Know about Climate Change

Climate change as a result of global warming has frequently been featured in the news, and scientists at the Commonwealth Scientific and Industrial Research Organization in Australia conducted an online survey of 5,000 Australians to learn their views on climate change and the degree to which they engaged in behaviors that are environmentally-friendly. The researchers found that there was considerable confusion about what Australians believe is the underlying cause of climate change. There appeared to be clear differences between those who believe that climate change is happening, and is simply a naturally occurring cycle of the Earth’s climate, versus those that believe that climate change is human-induced. The results suggest that further research on human attitudes about climate change is necessary.

· Pretesting the CAHPS Hospital Survey

The federal government’s Centers for Medicare and Medicaid Services (CMS) is supposed to make information about hospital performance available to the public, so the CMS spent several years creating and modifying a questionnaire, the Consumer Assessment of Health Providers and Systems (CSHPS) Hospital Survey. The researchers put it through extensive pretesting. The first version (68 questions) was given to a sample of 18 individuals. Based on their reactions, the survey was modified and tested on 13 more people, resulting in 66 items. Next, the 66-item survey was pilot tested with almost 50,000 patients in hospitals in three states, and the questionnaire was reduced to 32 items based on the results. It was tested at several more hospitals and reviewed by the National Quality Forum, resulting in a 27-item survey that was ready for use nationwide.

OUTLINE

I. QUESTIONNAIRE QUALITY AND DESIGN: BASIC CONSIDERATIONS

· A research survey is only as good as the questions it asks.

· Questionnaire design is one of the most critical stages in the survey research process.

· Good questionnaire design requires far more that correct grammar.

· To fulfill a researcher’s purposes, the questions must meet the basic criteria of relevance and accuracy.

· To achieve these ends, the decisions in the following order must be made:

1. What should be asked?

2. How should questions be phrased?

3. In what sequence should the questions be arranged?

4. What questionnaire layout will best serve the research objectives?

5. How should the questionnaire be pretested? Does the questionnaire need to be revised?

· What Should Be Asked?

· The specific questions to be asked will be a function of the previous decisions.

· The later stages of the research process will have an important impact on the questionnaire wording.

· When designing the questionnaire, the researcher must also be thinking about the types of statistical analyses that will be conducted.

· Questionnaire Relevancy

· A questionnaire is relevant to the extent that all information collected addresses a research question that will help the decision maker address the current business problem.

· The researcher should be specific about data needs and have a rationale for each item requesting information.

· Irrelevant questions are more than a nuisance because they make the survey needlessly long.

· When planning the questionnaire design, researchers must think about possible omissions.

· Questionnaire Accuracy

· Accuracy means that the information is reliable and valid.

· While it is generally believed that one should use simple, understandable, unbiased, unambiguous, and nonirritating words, no step-by-step procedure can be generalized.

· Respondents tend to be most cooperative when the subject of the research is interesting.

· If questions are lengthy, difficult to answer, or ego threatening, there is a high probability of biased answers.

· Question wording and sequence substantially influence accuracy.

II. WORDING QUESTIONS

· There are many ways to phrase questions, and many standard question formats have been developed in previous research.

· Open-Ended Response versus Fixed-Alternative Questions

· We may categorize two basic types of questions asked on the amount of freedom respondents are given in answering.

· Open-ended response questions pose some problem or topic and ask respondents to answer in their own words.

· Open-ended response questions are free-answer questions.

· In contrast, fixed-alternative questions (sometimes called closed-ended questions) give respondents specific limited-alternative responses and ask them to choose the one closest to their own viewpoint.

· Using Open-Ended Response Questions

· Open-ended response questions are most beneficial when the researcher is conducting exploratory research.

· They may also be useful at the beginning of an interview as they allow the respondent to warm up to the questioning process.

· The cost of open-ended response questions is substantially higher than that of fixed-alternative questions because the job of coding, editing, and analyzing the data is quite extensive.

· Another potential disadvantage of open-ended response questions is the possibility of interviewer bias to influence the answer; even the best interviewer can take shortcuts in recording answers.

· Also, articulate individuals tend to give longer answers and such respondents often are better educated and from higher income groups, which may not be representative of the entire population but gives them a large share of the responses.

· Using Fixed-Alternative Questions

· Fixed-alternative questions require less interviewer skill, take less time, and are easier for the respondent to answer.

· Answers to closed questions are classified into standardized groupings prior to data collection, which provides comparability of answers and facilitates coding, tabulating, and interpreting the data.

· However, when a researcher is unaware of the potential responses to a question, fixed-alternative questions cannot be used.

· If the researcher assumes the responses and is wrong, he or she will have no way of knowing the extent to which the assumption was incorrect.

· Unanticipated alternatives emerge when respondents believe that closed answers do not adequately reflect their feelings, causing them to make comments to the interviewer or write additional answers on the questionnaire.

· Respondents may check off obvious alternatives if they do not see the choice they would prefer.

· May tempt respondents to check an answer that is more prestigious or socially acceptable than the true answer.

· Most questionnaires mix open-ended and closed questions, providing a change of pace that can eliminate respondent boredom and fatigue.

· Types of Fixed-Alternative Questions

· Simple-dichotomy (dichotomous) questions – require the respondent to choose one of two alternatives. The answer can be a simple “yes” or “no” or a choice between “this” and “that.”

· Several types of questions provide the respondent with multi-choice alternatives :

· The determinant-choice question requires the respondent to choose one–and only one—response from among several possible alternatives.

· The frequency-determination question is a determinant-choice question that asks for an answer about the general frequency of occurrence.

· Attitude rating scales, such as the Likert scale, semantic differential, Stapel scale, and so on, are also fixed-alternative questions and were discussed in Chapter 14.

· The checklist question allows respondents to provide multiple answers to a single question. In many cases, the choices are adjectives that describe a particular object.

· Alternatives should be totally exhaustive, meaning that all the response options are covered and that every respondent has an alternative to check.

· There should be no overlap among categories in the checklist—each alternative should be mutually exclusive, that is, only one dimension of an issue should be related to that alternative.

· Phrasing Questions for Self-Administered, Telephone, and Personal Interview Surveys

· The means of data collection—telephone interview, personal interview, self-administered questionnaire—will influence the question format and question phrasing.

· In general, questions for mail, Internet, and telephone surveys must be less complex than those used in personal interviews.

· Questionnaires for telephone and personal interviews should be written in a conversational style.

· Exhibit 15.1 illustrates how a question may be revised for a different medium.

III. GUIDELINES FOR CONSTRUCTING QUESTIONS

· In developing a questionnaire, there are no hard-and-fast-rules.

· Some guidelines have been developed to avoid the most common mistakes.

· Avoid Complexity: Use Simple, Conversational Language

· Words used in questionnaires should be readily understandable to all respondents.

· The technical jargon of top executives should be avoided (e.g., “brand image,” “marginal analysis,” etc.).

· Avoid Leading and Loaded Questions

· Leading and loaded questions are a major source of bias in question wording.

· Leading questions suggest or imply certain answers.

· Such questions may result in a “bandwagon effect” which threatens the study’s validity.

· Partial mention of alternatives is a variation of this phenomenon.

· Loaded questions suggest a socially desirable answer or are emotionally charged.

· Certain answers to questions are more socially desirable than others.

· Invoking the status quo is a form of loading that results in bias because the majority of people tend to be resistant to change.

· Asking respondents “how often” leads them to generalize about their behavior and one is more likely to portray one’s ideal behavior rather than one’s average behavior.

· An introductory counterbiasing statement or preamble to a question that reassures respondents that their “embarrassing” behavior is not abnormal may help yield truthful responses. Also, an assurance of anonymity may help elicit honest responses to embarrassing questions.

· A question statement may be leading because it is phrased to reflect either the negative or positive aspects of an issue.

· To control for this bias, split-ballot technique, which reverses the wording of attitudinal questions for 50 percent of the sample, can be used.

· Avoid Ambiguity: Be as Specific as Possible

· Items on questionnaires are often ambiguous because they are too general.

· Indefinite words such as frequently, occasionally, often, etc., have many different meanings.

· Avoid Double-Barreled Items

· A question covering several items at once is referred to as a double-barreled question and should always be avoided.

· The results may be exceedingly difficult to interpret.

· Avoid Making Assumptions

· The researcher should not place the respondent in a bind by including an implicit assumption in the question.

· Another frequent mistake is assuming that the respondent had previously thought about an issue; research that induces people to express attitudes on subjects that they do not ordinarily think about is meaningless.

· Avoid Burdensome Questions That May Tax the Respondent’s Memory

· In many situations respondents cannot recall the answer to a question.

· Questions that give no clue as to the brand of interest are referred to as unaided recall questions.

· Often researchers will devise aided recall questions that provide a clue to jog the respondent’s memory.

· While aided recall is not as strong a test of attention or memory as unaided recall, it is less taxing to the respondent’s memory.

· Telescoping occurs when respondents recall past events as more recent than they actually were.

· Squishing occurs when respondents think that recent events took place longer ago than they really did.

· In situations in which “I don’t know” or “I can’t recall” is a meaningful answer, simply including a “don’t know” response category may solve the question writer’s problem.

· Make Certain Questions Generate Variance

· We want our variables to vary!

· The response categories provided must cover the breadth of possibilities (totally exhaustive) but also yield variance across respondents.

· If all the respondents check the same box, we have not generated usable information.

IV. WHAT IS THE BEST QUESTION SEQUENCE?

· The order of questions may serve several functions for the researcher.

· For example, if respondents’ curiosity is not aroused at the outset, they can become disinterested and terminate the interview.

· It is generally not advisable to ask demographic or classificatory questions (i.e., education, income, etc.) at the beginning because it may embarrass or threaten respondents.

· Order bias can result from an alternative answer’s position in a set of answers or from the sequencing of questions.

· Order bias can distort survey results.

· Asking specific questions before asking about broader issues is a common cause of order bias.

· It is advisable to ask the general questions before the specific questions to obtain the freest of open-ended responses.

· This technique is known as the funnel technique, and it allows researchers to understand the respondent’s frame of reference before asking more specific questions about the level of the respondent’s information and intensity of opinion.

· With attitude scales, there also may be an anchoring effect.

· The first concept measured tends to become a comparison point from which subsequent evaluations are made.

· Randomization of these items on a questionnaire helps to minimize this order bias.

· A related problem is bias caused by the order of alternatives on closed questions.

· The order of these choices should be rotated if producing alternative forms of the questionnaire is possible.

· Unfortunately, researchers rarely print alternative questionnaires to eliminate problems arising from order bias.

· One advantage of Internet surveys is the ability to reduce order bias by having the computer randomly order questions and/or response alternatives.

· Filter questions minimize the asking of questions that may be inapplicable

· Another form of filter question, the pivot question, can be used to obtain information that the respondent may be reluctant to provide.

V. WHAT IS THE BEST LAYOUT?

· The layout and attractiveness of the questionnaire are of crucial importance in mail, Internet, and other self-administered questionnaires.

· For different reasons, a good layout in questionnaires designed for personal and telephone interviews is also important.

· Traditional Questionnaires

· In mail questionnaires, often the rate of return can be improved by adding the money that might have been used as an incentive to improve the attractiveness and quality of the questionnaire instead.

· Questionnaires should be designed to appear as short as possible.

· The multiple-grid question presents several similar questions and corresponding response alternatives arranged in a grid format.

· The title of a questionnaire should be phrased carefully to:

· capture the respondent’s interest

· underline the importance of the research

· emphasize the interesting nature of the study

· appeal to the respondent’s ego

· emphasize the confidential nature of the study

· Instructions are often capitalized or printed in bold to alert the interviewer that it may be necessary to proceed in a certain way.

· Skip question – either skip instructions or an arrow drawn pointing to the next question informs the respondent which question comes next.

· Layout is extremely important when questionnaires are long or require the respondent to fill in a large amount of information.

· Internet Questionnaires

· Questionnaires on a website should be easy to use, flow logically, and have a graphic look and overall feel that motivate the respondent to cooperate from start to finish.

· With graphical user interface (GUI) software, the researcher can exercise control over the background colors, fonts, and other visual features displayed on the screen so as to create an attractive and easy-to-use interface between the computer user and the Internet survey.

· GUI software allows respondents to click on the appropriate answer rather than having to type answers or codes.

· The possibility that the questionnaire the researcher/designer constructs on his or her computer may look different from the one that appears on the respondent’s computer should always be considered when designing Internet surveys.

· One solution is to limit the horizontal width of the questions to 70 characters or less.

· Layout Issues

· The first decision is whether the questionnaire will appear page by page, with individual questions on separate screens, or on a scrolling basis, with the entire questionnaire appearing on a single web page that the respondent scrolls from top to bottom.

· The paging layout (going from screen to screen) greatly facilitates skip patterns.

· The scrolling layout gives the respondent the ability to read any portion of the questionnaire at any time, but the absence of page boundaries can cause problems.

· Push buttons with labels should clearly describe the actions to be taken.

· Decisions must be made about the use of color, graphics, animation, sound, and other special features that the Internet makes possible.

· Because many Internet surveys offer no visual clues about the number of questions to be asked, it is important to provide a status bar or some other visual indicator of questionnaire length.

· An Internet questionnaire uses windows known as dialog boxes to display questions and record answers.

· Four common ways of displaying questions on a computer screen (see Exhibit 15.8):

· Radio buttons – clicking on an alternative response deactivates the first choice and replaces it with the new response (i.e., like push buttons on an automobile radio).

· Drop-down box – a space-saving device that allows the researcher to provide a list of responses that are hidden from view until they are needed.

· Checklist questions may be followed by check boxes, several, none, or all of which may be checked by the respondent.

· Open-ended boxes are boxes in which respondents type their answers to open-ended questions, and they may be designated as one-line text boxes or scrolling text boxes depending on the breadth of the expected answer.

· Pop-up boxes are message boxes that can be used to highlight important information.

· Software That Makes Questionnaires Interactive

· Internet software allows the branching off of questioning into two or more different lines, depending on a particular respondent’s answer, and the skipping or filtering of questions.

· Most of these programs have hidden skip logic so that respondents never see any evidence of skips.

· Variable piping software allows variables, such as answers from previous questions, to be inserted into unfolding questions.

· Other software can randomly rotate the order of questions, blocks of questions, and response alternatives.

· Software can be used to control the flow of a questionnaire (e.g., preventing respondents from backing up or allowing them to stop and return later).

· Error trapping – error message appears if the respondent fails to answer a question or answers it with an incorrect type of response.

· Forced answering software – program will not let respondents continue if they fail to answer a question.

· Some designers include interactive help desks in their web questionnaires so that respondents can solve problems they encounter.

· Respondents leaving the questionnaire website, prematurely terminating the survey may receive an e-mail message to encourage them to revisit the site and complete the questionnaire, often picking up where they left off.

VI. HOW MUCH PRETESTING AND REVISING ARE NECESSARY?

· Usually, the questionnaire is tried out on a group, selected on a convenience basis, that is similar in makeup to the one that ultimately will be sampled.

· It is not necessary to get a statistical sample for pretesting.

· Pretesting allows the researcher to determine if the respondents have any difficulty understanding the questionnaire and whether there are any ambiguous or biased questions.

· This process can save the potential disaster of administering an invalid questionnaire to several hundred individuals.

· A preliminary tabulation of the pretest results often illustrates that while a question is easily comprehended and answered by the respondent, it is inappropriate because it is does not solve the problem.

· Administering a questionnaire exactly as planned in the actual study often is not possible.

· Interviewers are often used for pretest work because they can record requests for additional explanation or comments that indicate respondents’ difficulty with question sequence or other factors.

· Pretests are typically conducted to answer questions about the questionnaire such as:

· Can the questionnaire format be followed by the interviewer?

· Does the questionnaire flow naturally and conversationally?

· Are the questions clear and easy to understand?

· Can respondents answer the questions easily?

· Which alternative forms of questions work best?

· Pretests also provide the means for testing the sampling procedure and estimates for the response rates for the mail surveys and completion rates for telephone surveys.

VII. DESIGNING QUESTIONNAIRES FOR GLOBAL MARKETS

· International researchers must take cultural factors into account when designing questionnaires.

· The most common problem involves translation into another language.

· Back translation is the process of translating the questionnaire from one language to another and then having it translated back again by a second, independent translator. The back translator is often a person whose native tongue is the language that will be used on the questionnaire.

· Literacy influences the designs of self-administered questionnaires and interviews, so knowledge of the literacy rates in foreign countries is vital.

IX. APPENDIX 15A: QUESTION WORDING AND MEASUREMENT SCALES FOR COMMONLY RESEARCHED TOPICS

· This appendix provides a bank of questions and scales to provide easy reference for satisfying traditional research objectives.

QUESTIONS FOR REVIEW AND CRITICAL THINKING/ANSWERS

1. Evaluate and comment on the following questions, taken from several different questionnaires. Do they follow the rules discussed in the chapter?

See the textbook for the actual questions.

a. The fixed alternative responses to this question are somewhat ambiguous. Infrequently, occasionally, frequently, and all the time may mean different things to different people. The inclusion of the time period in parentheses helps; however, the answers here are not exhaustive and mutually exclusive. Where does one who uses the computer center twice a semester record his or her response?

b. This question about the Federal Trade Commission’s corrective advertising policy is complex because there are really two issues. First, whether the individual understands the policy and, second, whether the individual likes the policy. It would be best to ask two questions.

c. The first question asks the respondents to imagine that they do 90 percent of their driving in town. If this is not the case, the answer has to be pure fantasy. The second question asks the respondent to compare the electric car with a “Big 3” car. Some of us may interpret this to be Ford, GM, and Chrysler. Others may interpret this to be Chevrolet, Pontiac, and Oldsmobile, or Toyota, Honda, and Nissan. The term “Big 3” should be defined.

d. Questions that ask the percentage of usage, time, and the like are fine if every respondent makes sure that his or her answer amounts to 100 percent. Unfortunately, many times respondents will only partially answer the question or incorrectly account for 100 percent. They may indicate that they walk 60 percent of the time without filling in any of the other responses. Or they may indicate that they walk 60 percent of the time, bicycle 30 percent of the time, and use public transportation 30 percent of the time. The answer exceeds 100 percent. This requires editing on the part of the researcher. If possible, questions like this should be avoided.

e. In this question it may be that not all respondents will believe that the cooperative advertising program is generous. The answers are limited to yes and no. By responding yes, one assumes that everything is just fine as it is. If a negative answer is given, it implies that the policy should be discontinued. A solution might be to ask degrees of satisfaction with the program, such as very satisfied, somewhat satisfied, etc.

f. This survey question, perhaps needless to say, is extremely complex.

g. This question seems to be somewhat loaded and emotionally charged.

h. This question is from a personal interview. Although the question asks for one of two alternatives, “up” or “down,” if the respondents answer “stays the same” or “don’t know,” these responses will be recorded. To some extent up and down are forced choice alternatives which will reduce the number of “stay the same” answers because it is not part of the options listed in the question.

i. This is a long and complex question. By the time the question is asked it is likely that the respondent will have forgotten what was read to him or her.

2. The following question was asked of a sample of television viewers:

We are going to ask you to classify the type of fan you consider yourself to be for different sports and sports programs.

· Diehard Fan: Watch games, follow-up on scores and sports news multiple times a day

· Avid Fan: Watch games, follow-up on scores and sports news once a day

· Casual Fan: Watch games, follow-up on scores and sports news occasionally

· Championship Fan: Watch games, follow-up on scores and sports news only during championships/playoffs

· Non-Fan: Never watch games or follow-up on scores

· Anti-Fan: Dislike, oppose, or object to a certain sport

Does this question do a good job of avoiding ambiguity?

This is an excellent way of defining terms that the respondent may not formally use. For example, Diehard Fan could mean a lifelong Chicago Cubs fan. However, the definition here makes it clear what this term means in the context of this study.

3. How might the wording of a question asking about income influence respondents’ answers?

This question allows the instructor to discuss the fact that wording of questions on sensitive issues, such as income, may provide the respondent with a tendency to answer more heavily in a particular category.

4. What is the difference between a leading question and a loaded question?

It is difficult to draw a fine line between the two. However, the wording of a leading question influences the answers by providing a hint about the purpose of the question or a clue that hints at something such that the respondent thinks a particular answer is an acceptable answer. A loaded question is emotionally charged.

5. Design one or more open-ended response questions to measure reactions to a magazine ad for a Xerox photocopier.

Students’ responses will vary, but they should apply the guidelines covered in this chapter.

6. Evaluate the layout of the filter question (shown in the textbook).

This question uses a thin black line to outline the “yes” answer and subsequent questions to be asked to respondents who answer “yes.” The “no” answer is clearly out of the box. This is an alternative to using skip instructions or arrows. It seems quite appropriate.

7. Develop a checklist of things to consider in questionnaire construction.

Students’ responses will vary on this activity. Here are some questions that might be included:

Are simple words used?

Is the language vague?

Are the questions short?

Are the questions easy to understand?

Are the questions specific or general?

Are the questions leading?

Are the questions loaded?

Is any objectionable language used?

Have any assumptions been made in the questions?

Are any hypothetical questions used?

Did you talk down to the respondent?

Are you asking two questions or one?

Are any questions too sensitive?

Is the question necessary?

Is the questionnaire layout appealing?

Is the questionnaire layout easy to follow?

Is there enough room to answer the questions?

Are the instructions easy to understand?

8. It has been said that surveys show that consumers hate advertising, but like specific ads. Comment.

In survey research, questions are often broadly asked. They ask about advertising in general. However, when customers are asked about advertising in particular, they do not present a preconceived notion about the generalized subject advertising. They communicate about something they understand. Questions about advertising or any research subject should be asked within a proper context with the researchers trying to be as specific as possible.

9. Design a complete questionnaire:

a. To evaluate a new fast-food fried chicken restaurant.

b. To measure consumer satisfaction with an airline.

c. For your local Big Brothers and Big Sisters organization to investigate awareness of and willingness to volunteer time to this organization

d. For a bank located in a college town to investigate the potential for attracting college students as checking account customers.

While students’ responses will vary, they should be expected to apply the guidelines outlined in this chapter. The appendix to this chapter provides several sample questions.

10. The Apple Assistance Center is a hotline to solve problems for users of Macintosh computers and other Apple products. Design a short (postcard-size) consumer satisfaction/service quality questionnaire for the Apple Assistance Center.

While students’ responses will vary, they should be expected to apply the guidelines outlined in this chapter.

11. [Internet Question] Visit the following Web site: http://www.history.org . What type of questions might be asked in a survey to evaluate the effectiveness of this Web site in terms of being informative and in terms of being an effective sales medium?

While students’ responses will vary, they should be expected to apply the guidelines outlined in this chapter.

12. A client tells a researcher that she wants a questionnaire that evaluates the importance of 30 product characteristics and rates her brand and 10 competing brands on these characteristics. The researcher believes that this questionnaire will induce respondent fatigue because it will be far too long. Should the researcher do exactly what the client says or risk losing the business by suggesting a different approach?

This is an eternal question. Does consumer-oriented mean doing exactly what the client requests or telling the client there is a better or more appropriate way? In many situations, clients will dogmatically say, “This is exactly what I want. Do it or I will get someone else to do it for me.” If the researcher believes the research would not accomplish its task, he or she should tell this to the client and then determine what to do.

13. [Ethics Question] A political action organization has on its website a series of short questionnaires that ask about your political position on an important topic. It also includes a “Support Reply Form,” a solicitation for donations. Is this approach ethical?

This is not ethical. The purpose of research is research, not promotion or membership solicitation. The researcher has an obligation to the respondent to keep everything above board because the respondent donates his or her time.

14. [Internet Question] Visit Mister Poll at http://www.misterpoll.com , where you will find thousands of user-contributed polls on every imaginable topic from the controversial to the downright zany. What you find will depend on when you visit the site. However, you might find something such as a movie poll, where you pick your favorite film of the season. Evaluate the questions in the poll.

Students’ responses will vary on this question, but they should use the guidelines presented in this chapter to evaluate the questions.

15. Try to find two friends that know the same foreign language. Write 10 Likert questions that measure how exciting a retail store environment is to shop in. Have one of your friends interpret the question into the foreign language. Have the other take the translation and state each question in English. How similar is the translated English to the original English? Comment.

Students’ responses will vary on this question.

RESEARCH ACTIVITY

1. Design eight questions that assess how effective an undergraduate college business course has been.

While students’ responses will vary, they should be expected to apply the guidelines outlined in this chapter.

CASE 15.1 Agency for Healthcare Research and Quality

Objective: To enable students to evaluate a questionnaire.

Summary: A survey developed by the Agency for Healthcare Research and Quality (AHRQ) of the U.S. Department of Health and Human Services to measure hospital employees’ attitudes about patient safety in their facilities is given in the case exhibit. The purpose of the survey is to help hospitals ensure safety by creating an environment in which employees share information, improve safety when problems are identified, and if necessary, change the way employees deliver care.

Questions

1. Evaluate the questionnaire. Can you suggest any improvements?

This is an excellent questionnaire. The instructions are clear, and the questions adhere to several of the guidelines covered in this chapter.

2. Will this survey meet its objectives? Explain.

The survey was designed to measure hospital employees’ attitudes about patient safety in their facilities, and the questionnaire appears to do that. However, the ultimate objective is to improve patient safety, and a questionnaire, no matter how good it is, will not do that. The users of the survey need to take the information gleaned from using the survey and putting that information into action to enhance patient safety.

CASE 15.2 Canterbury Travels

Objective: The purpose of this case is to critically evaluate a questionnaire.

Summary: Market and customer information is desired by a new travel agency. They design a questionnaire to obtain this information. See the case exhibit for the questionnaire.

Questions

1. Critically evaluate the questionnaire.

Evaluation of the specific questions:

1. There is no indication of the time period. It is highly likely that almost everyone has traveled out of state at one point in their lifetime.

2. Again there is no indication of a usual time period or the last time period. Many businesspeople that frequently travel on the job will travel for pleasure in the summer.

3. The categories are not mutually exclusive. Six or more times per year fits each of the monthly categories. What if someone travels eight times a year?

4. Assume that all travel is either airline or on some means of public transportation. The other categories should be expanded.

5. It is okay. However, boxes should be provided for responses. This is true of the other questions as well. The layout could be improved substantially.

6. Presents a rating scale without instructions. The instructions could be substantially clearer. Further, it seems that this question is really asking about the importance of attributes rather than whether or not they are good or bad. A number of respondents might feel that it is good to have a casual atmosphere but rate this as largely unimportant in the selection of a travel agency.

7. Could use a better layout and better instructions.

8. Does not provide enough layout of space for an answer. There is a typo here.

9. Okay.

10. Asks two questions.

11. Asks a yes or no question and then asks respondents to rate the frequency of using travel agencies. The appropriate question would be, “How often do you use each of these travel agencies?”

12. This is redundant if a respondent answers “never” (which is absent) from Question 11.

13. This is out of place. The name of Canterbury has already been indicated in Question 11.

14. Okay, although there may be some redundancy between credit card and company or personal charge. Also, boxes or lines should be provided, or if respondents are to circle the answer, then instructions should say so.

15. This will not yield any accurate indication of advertising effectiveness.

16. This should have a variable statement such as: This advertisement for (?) travel agency.

17. Suppose that you are interpreting the results of this question. How would you cross-tabulate it with the other questions?

18. Okay, except boxes or lines should be provided.

2. Will Canterbury Travels gain the information it needs from this survey?

Probably not if the purpose of this survey is to “determine the effectiveness of her advertising.” Most of the questions will not provide Roxanne with this information.

3. Design a questionnaire to satisfy Roxanne Freeman’s information needs.

While students’ responses will vary, they should be expected to apply the guidelines outlined in this chapter.

CASE 15.3 McDonald’s Spanish Language Questionnaire

Objective: To illustrate the need for back translation and to test the student’s ability to back translate from Spanish to English.

Summary: A series of questions from a customer satisfaction survey are presented in Spanish.

Questions

1. What is the typical process for developing questionnaires for markets where consumers speak a language other than English?

Usually a questionnaire goes through several revisions. The exact number of revisions depends on the researcher’s and client’s judgment. Revising the questionnaire usually ends when both agree that the desired information is being collected in an unbiased manner.

International business takes cultural factors into account when designing questionnaires. The most common problem involves translation into another language. A questionnaire developed in one country may be difficult to translate because equivalent language concepts do not exist or because of differences in idiom and vernacular. International business researchers often have questionnaires back translated. Back translation is the process of translating the questionnaire from one language to another and then translating it back again by a second, independent translator. The back translator is often a person whose native tongue is the language that will be used for the questionnaire. This can reveal inconsistency between the English version and the translation, for correction if necessary. For example, when a soft-drink company translated its slogan “Baby, it’s cold outside” into Cantonese for research in Hong Kong, the result read “Small Mosquito, on the inside, it is very cold.” In Hong Kong, small mosquito is a colloquial expression for a small child.

2. Find someone who speaks Spanish and have them back translate the questions that appear in Case Exhibit 15.3-1. Are these Spanish language questions adequate?

The translation is excellent. The English language questions appear below.

Start Here

1. Overall, how satisfied were you with your visit to McDonald’s today?

Not at all satisfied 1 2 3 4 5 Extremely satisfied

2. Were you visiting us Inside (I) or at the Drive-Thru (DT)?

Inside (I) Drive-Thru (DT)

3. Were you visiting us for Breakfast (B), Lunch (L), or Dinner (D)?

Breakfast (B) Lunch (L) Dinner (D)

4. Were you visiting us on a Weekday (W) or Weekend (E)?

Weekday (W) Weekend (E)

Food

5. Were you satisfied with the food you received today?

Yes (Y) No (N)

If NO, what was the problem(s)? Please fill in the appropriate circles.

 Cold sandwich/Entree

 Food didn’t look good

 Food didn’t taste good

 Not enough fried in box

 Cold fries/Hash browns

 Fries not salted right

 Watery/Poor tasting drink

CASE 15.4 Schonbrunn Palace in Vienna

Objective: To illustrate the need for back translation and to test the student’s ability to back translate from German to English.

Summary: A series of questions from a customer satisfaction survey are presented in German.

Questions

1. What is the typical process for developing questionnaires for markets where consumers speak a different language?

See the answer to Question 1 of the McDonald’s case presented previously.

2. Find someone who speaks German and have him or her back translate the questions that appear Case Exhibit 15.4-1. Are these German language questions adequate?

The translation is excellent. The English language questions appear below.

Schonbrunn Palace

Poll of Visitors

State of residence _________________________________

Age __________ years

Sex Male Female

Date Year___ - Month _________ - Day _____

Time __________________

*Have you been to Schonbrunn Palace for the first time today?

____ Yes ____ no, for the ________ time

*Which Tour did you choose?

___ Grand Tour (40 rooms)

___ Imperial Tour (22 rooms)

*What kind of guide did you choose?

___ guided tour (offered by Schonbrunn Palace)

___ my own guide (organized touring group)

___ Audio-Guide in __________________________(language)

___ no guided tour at all

*If you joined a guided tour:

How do you find your tour guide?

___ excellent ____ rather friendly ____ rather unfriendly ____ very unfriendly

*If you used the offered audio-guide:

How did you find this guide?

____ excellent ____ good ____ bad ____ very bad

because…________________________________________________________

*What is your general impression of Schonbrunn Palace?

____ excellent ____ good _____ bad _____ very bad

because… _______________________________________________________

* What is your impression of the personnel in the Castle?

____ excellent ____ good ____ bad _____ very bad

because… _______________________________________________________

How do you find your way in the park and the Palace of Schonbrunn (signposts, do you easily find the ticket-office, the toilets, and the exit, etc.)?

____ excellent ____ good ____ bad _____ very bad

because… _______________________________________________________

* Do you feel well informed about the palace and its history after the visit?

____ excellent ___ good ____ bad ____ very bad

Have you been disturbed during your visit?

by (other) groups:

_____heavily _____ slightly _____hardly _____ not at all

by individual visitors:

_____ heavily _____ slightly _____hardly _____ not at all

What do you think of the way the rooms are presented (furniture, light, decoration)?

_____ excellent _____ good _____ bad _____ very bad

Because … ______________________________________________________

Now, after the tour, do you have a lively idea of life at court in former days?

_____ yes _____ some ______ hardly ______ no, not at all

Because… ______________________________________________________

What else would you like to know about Schonbrunn Palace?

How do you find the entrance-fees?

_____ far too _____ somewhat _____ adequate/fair _____ low-priced

expensive expensive

Because… _______________________________________________________

What is your impression of the offer at the Schonbrunn shop?

_____ excellent _____ good ______ bad ______ very bad

Because… ______________________________________________________

* What do you think could be further improved or should be changed in Schonbrunn Palace?

Thank you very much for your visit and your suggestions!