Week 5

Chapter 10

Survey Research: Communicating with Respondents

Zikmund, W., Babin, B. J., Carr, J., & Griffin, M. (2013). Business research methods (9th ed.). Mason, OH: Cengage Learning.

AT-A-GLANCE

I. Interviews as Interactive Communication

A. Noninteractive media

II. Personal Interviews

A. Advantages of personal interviews

· Opportunity for feedback

· Probing complex answers

· Length of interview

· Completeness of questionnaire

· Props and visual aids

· High participation

B. Disadvantages of personal interviews

· Interviewer influence

· Lack of anonymity of respondent

· Cost

C. Door-to-door interviews and shopping mall intercepts

· Door-to-door interviews

· Callbacks

· Mall intercept interviews

D. Global considerations

III. Telephone Interviews

A. Mobile phone interviews

B. Phone interview characteristics

· Speed

· Cost

· Absence of face-to-face contact

· Cooperation

· Incentives to respond

· Representative samples

· Callbacks

· Limited duration

· Lack of visual medium

C. Central location interviewing

D. Computer-assisted telephone interviewing

E. Computerized voice-activated telephone interview

F. Global considerations

IV. Self-Administered Questionnaires

A. Mail questionnaires

· Geographic flexibility

· Cost

· Respondent convenience

· Anonymity of respondent

· Absence of interviewer

· Standardized questions

· Time is money

· Length of mail questionnaire

B. Response rates

C. Increasing response rates for mail surveys

· Cover letter

· Money helps

· Interesting questions

· Follow-ups

· Advance notification

· Survey sponsorship

· Other techniques

· Keying mail questionnaires with codes

D. Global considerations

V. Self-Administered Questionnaires Using Other Forms of Distribution

A. Fax surveys

B. E-mail surveys

C. Internet surveys

· Speed and cost-effectiveness

· Visual appeal and interactivity

· Respondent participation and cooperation

· Representative samples

· Accurate real-time data capture

· Callbacks

· Personalized and flexible questioning

· Respondent anonymity

· Response rates

· Security concerns

D. Kiosk interactive surveys

E. Survey research that mixes modes

F. Text-message surveys

VI. Selecting the Appropriate Survey Research Design

VII. Pretesting

VIII. Ethical Issues in Survey Research

LEARNING OUTCOMES

1. Summarize ways researchers gather information through interviews

2. Compare the advantages and disadvantages of conducting door-to-door, mall intercept, and telephone interviews

3. Evaluate the advantages and disadvantages of distributing questionnaires through the mail, the Internet, and by other means

4. Discuss the importance of pretesting questionnaires

5. Describe ethical issues that arise in survey research

CHAPTER VIGNETTE: Mobile Surveys Catching On, and Catching Respondents “On the Go”!

Mobile surveying technologies now integrate SMS text messaging (“texting”) with electronic surveys. Respondents answer single or multiple choice questions, or even provide open-ended responses to questions. These “instant feedback” types of survey responses can have many different business applications. Researchers interested in experiential surveying use mobile surveys to capture people’s feelings at that particular instant, and thus can create a longitudinal understanding of people’s attitudes and emotional states over time.

SURVEY THIS!

How would you classify the survey you participated in as part of this class? Which approach did it use? What media type was involved? What do you think the response rate for this survey is? Students are instructed to e-mail the survey link to 10 friends not taking this class. How many actually responded? What factors of this survey contribute to either a relatively high or relatively low response rate?

RESEARCH SNAPSHOTS

· Being Good Neighbors Means Learning About Them First

In 2004-2005, students from a community college engaged in a service learning project and captured information from an often missed demographic in our society—the elderly. They established a door-to-door survey of older adults, and their hard work led to the inclusion of elderly adult needs as part of the Good Neighbor Initiative, which included programs for literacy, hunger, homelessness, and health. Without going from house to house, it may not have been possible for the community to capture the specific needs of this important population in their city.

· Automated Phone Surveys of Teens

Computer-assisted telephone interviewing (CATI) and computerized self-interviewing, in which the subjects listen to prerecorded questions and then respond by entering answers with the telephone’s keypad, have been used to ask teens potentially sensitive questions. For example, teens were more likely to say they had smoked or lack a firm commitment not to smoke in the future using this method. However, when a parent was present, their responses were less likely to indicate smoking desire or susceptibility. This suggests that teens might be underreporting their smoking behavior.

· Mixed Mode Data Collection: The Case of Web and Telephone Surveys

While the advantages in time and cost efficiencies of Internet-based surveys are apparent, a major criticism is the low response rate. This gives rise to the question, “Would the information provided by sample members that did not respond to the web survey differ from that provided by web survey respondents?” Research investigated this using a “sequential mixed-mode data collection” procedure that compared web-based surveys and telephone surveys of nonrespondents. While response rates were much higher for telephone the telephone survey, comparison of the responses revealed very few significant differences. It appears that web-based surveys are not only fast and cost-effective, they provide the same information as the far more costly telephone surveys.

OUTLINE

I. INTERVIEWS AS INTERACTIVE COMMUNICATION

· Human interactive media are a personal form of communication.

· Electronic interactive media allow businesses to reach a large audience, personalize individual messages, and interact using digital technology.

· Largely controlled by the users themselves.

· No other human need be present.

· The Internet is radically altering many organizations’ research strategies, being the prominent form of new electronic interactive media.

· Electronic interactive media also include DC-ROM and DVD materials, touch-tone telephone systems, touch-screen interactive kiosks in stores, and other forms of digital technology.

· Noninteractive Media

· Self-administered questionnaires printed on paper are noninteractive.

· Less flexible than surveys using interactive communication media.

II. PERSONAL INTERVIEWS

· A personal interview is a form of direct communication in which an interviewer asks respondents questions face-to-face.

· This versatile and flexible method is a two-way conversation between interviewer and respondent.

· Advantages of Personal Interviews

· Opportunity for Feedback

· Offer the lowest chance that respondents will misinterpret questions because the interviewer can clarify the instruction or questions.

· Probing Complex Answers

· Although interviewers are expected to ask questions exactly as they appear on the questionnaire, probing allows them some flexibility.

· Personal interviews vary in the degree to which questions are structured and in the amount of probing required.

· Personal interview is especially useful for obtaining unstructured information.

· Length of Interview

· General rules of thumb:

· Mails surveys should not exceed 6 pages.

· Telephones interviews should not exceed 10 minutes.

· Personal interviews can be much longer (i.e., 1 ½ hours).

· Should be clear about how long participation should take when requesting participation.

· Online surveys should include a completion meter that shows a respondent’s progress.

· Completeness of Questionnaire

· The social interaction between a well-trained interviewer and a respondent increases the likelihood that the respondent will answer all of the items on the questionnaire.

· Item nonresponse – failure to provide an answer to a question—is least likely to occur when an experienced interviewer asks questions directly.

· Props and Visual Aids

· Face-to-face interviews allow the showing of new product samples, sketches of proposed advertising, or other visual aids.

· High Participation

· Presence of an interviewer generally increases the percentage of people willing to complete the interview.

· All respondents have to do is talk.

· Disadvantages of Personal Interviews

· Interviewer Influence

· Some evidence suggests that demographic characteristics of the interviewer influence respondents’ answers.

· Differential interviewer techniques may be a source of bias (i.e., tone of voice, appearance).

· Lack of Anonymity of Respondent

· Because a respondent is not anonymous, researchers often spend considerable time and effort to phrase sensitive questions to avoid social desirability bias.

· Cost

· Personal interviews are expensive.

· Cost is influenced by:

· geographic proximity of respondents

· length and complexity of the questionnaire

· number of people who are nonrespondents because they could not be contacted

· Door-to-Door Interviews and Shopping Mall Intercepts

· Door-to-Door Interviews

· Because door-to-door interviews increase the participation rate, they may provide a more representative sample of the population than mail questionnaires.

· Can reach people who do not have telephones, who have unlisted numbers, or who are otherwise difficult to contact.

· May underrepresent some groups and overrepresent others based on the geographic area covered.

· May exclude individuals who live in multiple-dwelling units with security systems or executives who are too busy to participate.

· Some people simply will not open the door when a stranger knocks.

· Becoming a thing of the past.

· Callbacks

· Callbacks, or attempts to recontact individuals selected for the sample, are the major means of reducing nonresponse error.

· Important in door-to-door interviews because not-at-home individuals (e.g., working parents) may systematically vary from those who are at home (e.g., nonworking parents, retirees).

· Mall Intercept Interviews

· Personal interviews conducted in shopping malls are referred to as mall intercept interviews, or shopping center sampling.

· Typically intercept shoppers at a central point within the mall or at an entrance.

· Costs are lower.

· No travel is required.

· Can be conducted quickly.

· A major problem is that individuals usually are in a hurry to shop, so refusal is high (i.e., around 50%).

· More conducted than door-to-door interviews.

· Researcher must recognize that he or she is not looking for a representative sample of the total population.

· Can show large, heavy, or immobile visual materials (e.g., television commercial).

· Can give an individual a product to take home to use and contact later by phone.

· Appropriate when a consumer durable product must be demonstrated.

· Global Considerations

· Willingness to participate in a personal interview varies dramatically around the world.

· In many Middle Eastern countries, women would never consent to be interviewed by a man.

· In some countries, discussing grooming behavior and personal-care products with a stranger would be highly offensive.

· Norms about appropriate business conduct also influence businesspeople’s willingness to provide information to interviewers (e.g., Japanese managers will not conduct interviews during business hours due to responsibility to oversee their employees while on the job).

III. TELEPHONE INTERVIEWS

· The mainstay of commercial survey research for decades.

· Quality of data may be comparable to that of data collected in personal interviews.

· Respondents are more willing to provide detailed and reliable information on a variety of personal topics than they are with personal interviews.

· Cellphone adoption and no-call laws limit the ability to generate a representative sample.

· Mobile Phone Interviews

· In the U.S., no telemarketing can be directed toward mobile phone numbers by law. Respondents have to “opt-in” before their phone number would be made available for such calls.

· The recipient of a mobile phone call is even more likely to be distracted than the recipient of a home or office call.

· The area codes for mobile phones are not necessarily tied to geography, so a researcher may be unable to determine whether or not a respondent fits into the desired geographic sampling population simply by taking note of the area code.

· The phones have varying abilities for automated responses and differing keypads.

· Phone Interview Characteristics

· Speed – hundreds of interviews can be collected literally overnight.

· Cost – estimated to be less than 25 percent of the cost of door-to-door personal interviews; travel time and costs are eliminated.

· Absence of Face-to-Face Contact – respondents may answer embarrassing or confidential questions more willingly, but interviewer and respondent cannot see each other, resulting in a greater tendency for interviewers to record no answers and incomplete answers than in personal interviews.

· Cooperation

· Telephone response rates have been falling due to caller ID and answering machines used to screen calls as well as more phone lines dedicated to fax machines and computers.

· It is illegal in the U.S. for researchers to contact anyone who would have to pay for the call (i.e., cellphones).

· Incentives to Respond – respondents should receive some incentive to respond.

· Representative Samples – practical difficulties complicate obtaining representative samples based on listings in the telephone book.

· Well over 95% of U.S households have land-line telephones.

· People without phones are likely to be poor, aged, rural, or living in the South.

· People have unlisted phone numbers for two reasons:

· they have moved recently

· by choice

· Random digit dialing eliminates the counting of names in a list and subjectively determining whether a directory listing is a business, institution, or legitimate household.

· Callbacks – an unanswered call, a busy signal, or a respondent who is not at home requires a callback, which is much easier to make than callbacks in personal interviews.

· Limited Duration – respondents who run out of patience can hang up.

· Lack of Visual Medium – medium is not appropriate if need to show respondent something (i.e., package, advertisement) or if using certain attitude scales and measuring instruments (i.e., semantic differential that requires the respondent to see a graphic scale).

· Central Location Interviewing

· Central location interviewing allows firms to hire a staff of professional interviewers and to supervise and control the quality of interviewing more effectively.

· Can benefit from cost economies.

· Computer-Assisted Telephone Interviewing

· Computer-assisted telephone interviewing (CATI) allows response to telephone interviews to be entered directly into the computer by the interviewer.

· Telephone management systems select phone numbers, dial the numbers automatically, and perform other labor-saving functions, such as generating an automatic callback schedule.

· Researchers can request daily status reports on the number of completed interviews relative to quotas.

· Interviews can also be conducted by a pre-recorded voice with the respondent answering by punching buttons on the phone.

· Computerized Voice-Activated Telephone Interview

· Allows researchers to conduct telephone interviews without human interviewers.

· Works best with very short, simple questionnaires.

· Entire data collection process can be automated because a recorded voice is used to both ask the questions and record answers.

· Global Considerations

· Different cultures have different norms about proper telephone behavior.

· Latin American businesspeople will not open up to strangers on the telephone.

· Japanese respondents consider it ill-mannered if telephone interviews last more than 20 minutes.

IV. SELF-ADMINISTERED QUESTIONNAIRES

· No matter how self-administered questionnaires are distributed (i.e., mail, in high-traffic locations in stores), they are different from interviews because the respondent takes responsibility for reading and answering the questions.

· Presents a challenge to researchers because they rely on the clarity of the written word rather than on the skills of the interviewer.

· Mail Questionnaires

· A mail survey is a self-administered questionnaire sent to respondents through the mail.

· Several advantages and disadvantages are listed below.

· Geographic Flexibility

· Can reach a geographically dispersed sample simultaneously because interviewers are not required.

· Isolated respondents (e.g., farmers) or those who are difficult to reach (e.g., busy executives) can easily be contacted by mail.

· Cost

· Relatively inexpensive compared with personal interviews, but they are not cheap.

· Most include follow-up mailings, which require additional postage and printing costs.

· Low response rates contribute to higher cost.

· Respondent Convenience

· Can be filled out when the respondents have time, so they are more likely to take time to think about their replies.

· Allow respondents to collect facts that they may not be able to recall without checking.

· Anonymity of Respondent

· Respondents are more likely to provide sensitive or embarrassing information when they can remain anonymous.

· Can reduce social desirability bias.

· Absence of Interviewer

· Once the respondent receives the questionnaire, the questioning process is beyond the researcher’s control.

· Respondent does not have the opportunity to question the interviewer.

· Respondents can read the entire questionnaire before they answer individual questions, which could affect responses.

· Standardized Questions

· Questionnaires typically are highly standardized, and the questions are quite structured.

· Questions and instructions must be clear-cut and straightforward.

· Time Is Money

· A minimum of 2 or 3 weeks is necessary for receiving the majority of the responses.

· Follow-up mailings require an additional 2 or 3 weeks.

· The time between the first mailing and the cut-off date normally is 6 to 8 weeks.

· Length of Mail Questionnaire

· General rule of thumb is not to exceed six pages in length.

· When a questionnaire requires a great deal of effort, and incentive should be given.

· Response Rates

· The basic calculation for obtaining a response rate is to count the number of questionnaires returned or completed, then divide the total by the number of eligible people who were contacted or requested to participate in the survey.

· Typically, the number in the denominator is adjusted for faulty addresses and similar problems that reduce the number of eligible participants.

· The major limitations of mail questionnaires relate to response problems.

· Respondents who complete it may not be typical of all people in the sample.

· Researcher has no assurance that the intended subject is the person who fills out the questionnaire (electronic surveying suffers same problem).

· Cooperation and response rates rise as home value increases.

· Rarely will have response rates of 50 percent or more, but follow-up mailings and other techniques may increase response rate to an acceptable level.

· Increasing Response Rates for Mail Surveys

· Individuals who are interested in the general subject of the survey are more likely to respond than those who are not.

· Ways to increase response rates include using a stamped return envelope, using attractive questionnaires, and wording questions so that they are easy to understand.

· Cover Letter

· A cover letter that accompanies a questionnaire or is printed on the first page is an important means of inducing the reader to complete and return the questionnaire.

· First paragraph explains why the study is important.

· The basic appeal alludes to the social usefulness of responding.

· Two other appeals are asking for help (“Will you do us a favor?”) and the egotistical appeal (“Your opinions are important!”).

· Most cover letters:

· promise confidentiality

· encourage recipient to use the postage-paid reply envelope

· describe any incentive for participation

· explain that answering the questionnaire will not be difficult or time consuming

· describe how the person was scientifically selected for participation.

· Personalized letter shows the respondent that he or she is important.

· Using letterhead rather than a printed form increases response rates.

· Money Helps

· Monetary incentives appear to be the most effective and least biasing incentive.

· Attracts attention and creates a sense of obligation.

· Works for all income categories.

· Sending the monetary incentive to a charity of the respondent’s choice increases response rates dramatically.

· Interesting Questions

· Certain interesting questions can be added to the questionnaire to stimulate respondents’ interest and to induce cooperation.

· Questions may be of little concern to the researcher, but respondents who are indifferent may be given a reason for responding.

· Follow-up

· Response rates are relatively high for the first two weeks, then the rates gradually taper off.

· Most studies use a follow-up letter or postcard reminder that may include a duplicate questionnaire or may merely be a reminder to return the original questionnaire.

· Multiple contacts almost always increase response rates.

· Advance Notification

· Advance notification (i.e., by letter or telephone) that a questionnaire will be arriving has been successful in increasing response rates.

· Notices that go out closer to the questionnaire mailing time produce better results than those sent out too far in advance.

· Optimum lead time is 3 days before the mail survey is to arrive.

· Survey Sponsorship

· Sponsorship by well-known and prestigious organizations (i.e., universities or government agencies) may also significantly influence response rates.

· Surveys mailed to a consumer panel receive exceptionally high response rates because members have already agreed to cooperate.

· Other Techniques

· Numerous other devices, such as type of postage (i.e., commemorative vs. regular stamp), envelope size, color of paper, etc, have been varied to increase response rates.

· Limited success.

· The researcher should consider his or her particular situation.

· Keying Mail Questionnaires with Codes

· One device for eliminating those who have already responded from the follow-up mailing list is to mark the questionnaire so that they may be keyed to identify members of the sampling frame who are nonrespondents.

· Blind keying of questionnaires can entail varying the job number or room number on the research department on a return envelope.

· Visible keying is a visible code number on the questionnaire, and a statement should indicate to respondents that the purpose is to avoid sending them a duplicate questionnaire.

· Global Considerations

· Postal services and cultural circumstances differ around the world.

· Issues to consider include the reliability of mail delivery, literacy rates, and trust that researchers can and will provide confidentiality.

· Hand delivery or door-to-door interviewing may be necessary.

· Some consumers may be discouraged from talking to an interviewer (e.g., women and children), so mailed questionnaires would be superior to interviews.

V. SELF-ADMINISTERED QUESTIONNAIRES USING OTHER FORMS OF DISTRIBUTION

· Many forms are similar to mail questionnaires.

· Warranty or owner registration cards often are used to collect demographic information and data about where and why products were purchased.

· People who fill these out may differ form those who do not.

· The drop-off method, in which an interviewer drops off a questionnaire (often an extremely long one) and then picks it up later, sacrifices some cost savings because it requires traveling to each respondent’s location.

· Fax Surveys

· Potential respondents receive and/or return questionnaires via fax machine.

· Disadvantage is that only respondents with fax machines who are willing to exert the extra effort will return questionnaires.

· People with extreme opinions will be more likely to respond.

· Researchers may use faxing as one of several options for replying to a survey.

· Faxing questionnaires reduces the sender’s printing and postage costs and can be delivered and returned faster than traditional mail surveys.

· Can deal with timely issues.

· E-Mail Surveys

· Some individuals cannot be reached this way.

· Some projects lend themselves to e-mail surveys (i.e., internal surveys of employees or satisfaction surveys of retail buyers who deal with the organization via e-mail).

· Benefits include:

· speed of distribution

· lower distribution and processing costs

· faster turnaround time

· more flexibility

· less handling of paper questionnaires

· Some researchers have argued that many respondents feel they can be more candid.

· But employees know that their e-mails are not secure.

· Maintaining respondents’ anonymity is difficult because reply will have the sender’s address.

· Not all e-mail systems have the same capacity or settings, which limits the types of questions and layout of the e-mail questionnaire.

· Guidelines for printed mail surveys apply to e-mail surveys.

· Should include a valid return e-mail address in the “from” box and reveal who is conducting the survey.

· E-mail should be addressed to a single person (the blind carbon copy, or BBC, field can be used if the same message must be sent to an entire sample).

· E-mail letters can be used as cover letter asking respondents to participate in an Internet survey, and they usually include a password and a link to a unique Web site.

· Internet Surveys

· An Internet survey is a self-administered questionnaire posted on a website.

· Speed and Cost-Effectiveness

· Internet surveys allow researchers to reach a large audience (possibly a global one), personalize individual messages, and secure confidential answers quickly and cost-effectively.

· Eliminates the costs of paper, postage, data entry, and other administrative costs.

· Once developed, the incremental cost of reaching additional respondents is minimal.

· Even with large samples, surveys can be conducted in a week or less.

· Visual Appeal and Interactivity

· Surveys can be interactive.

· Can use more sophisticated lines of questioning based on the respondent’s prior answers.

· Can use color, sound, and animation.

· Can present visual materials.

· Respondent Participation and Cooperation

· Participation may occur because computer users intentionally navigate to a particular website where questions are displayed.

· In some cases, visitors to a site cannot venture beyond the survey page without providing information.

· Participants can initially be contacted via e-mail and given logon instructions and a password, thus preventing access by individuals who are not part of the selected sample.

· Unique passwords also allow the researcher to track the responses of each respondent to identify anyone who makes an effort to participate more than once.

· The welcome screen should contain the name of the research company and information about how to contact the organization if the respondent has a problem or concern.

· Representative Samples

· Some individuals cannot access the Internet.

· Not all people have the same level of technology.

· Some people have minimal computer skills.

· Internet surveys should be simple so that all respondents can interact at the same level of technological sophistication.

· Accurate Real-Time Data Capture

· Each respondent’s answers are entered directly into the researcher’s computer as soon as the questionnaire is submitted.

· Software may be programmed to reject improper data entry.

· Real-time data capture allows for real-time data analysis.

· Callbacks

· Panel members who have not completed the survey can be easily recontacted.

· Software can automatically send e-mail reminders.

· Can also identify respondents who completed only a portion of the questionnaire and send those people customized messages.

· Personalized and Flexible Questioning

· Internet surveys are programmed in much the same way as computer-assisted telephone interviews.

· Major advantage is the ability to sequence questions based on previous responses.

· Another advantage is that it can prompt respondents when they skip over a question.

· Ability to customize questions and the low cost per recipient help researchers to keep surveys short.

· Designers of Internet questionnaires can be creative and flexible in the presentation of questions using a variety of dialog boxes, or windows that prompt the respondent to enter information.

· Respondent Anonymity

· The anonymity of the Internet encourages respondents to provide honest answers to sensitive questions.

· Response Rates

· Methods to increase response rates are similar to those for other kinds of survey research.

· A personalized invitation may be important.

· An e-mail message’s subject line is critical.

· Incentives to respond must be in the form of a promise of a future reward.

· Security Concerns

· Hackers or competitors may access Web sites to discover new product concepts, new advertising campaigns, and other top-secret ideas.

· Respondents may worry whether personal information will remain private.

· Kiosk Interactive Surveys

· A computer with a touch screen may be installed in a kiosk at a trade show, at a professional conference, in an airport, or in any other high-traffic location to administer an interactive survey.

· Self-selection often is a problem.

· Computer-literate individuals are most likely to participate.

· Temporary locations often require a fieldworker to be at the location to assist users.

· Survey Research That Mixes Modes

· Research objectives may dictate the use of some combination of telephone, mail, e-mail, Internet, and personal interview (e.g., conduct a short telephone screening interview to determine whether respondents are eligible for recontact in a more extensive personal interview).

· Mixed-mode surveys combine the advantages of the various methods.

· Conducting research in two or more waves, however, creates the possibility that some respondents will no longer cooperate or will be unavailable in the second wave.

· Text-Message Surveys

· These may use the SMS (short-message service) or MMS (Multi-Media Service).

· Perhaps the newest survey approach.

· Has all the advantages and disadvantages of mobile-phone surveys.

· MMS messages can include graphic displays or even short videos.

VI. SELECTING THE APPROPRIATE SURVEY RESEARCH DESIGN

· There is no best form of survey; each has advantages and disadvantages.

· Exhibit 10.4 summarizes the major advantages and disadvantages the various types.

VII. PRETESTING

· Pretesting involves a trial run with a group of respondents to iron out fundamental problems in the instructions or design of a questionnaire.

· This stage is sometimes eliminated because of costs or time pressures.

· Three basic ways to pretest exist:

· The first two involve screening the questionnaire with other research professionals.

· The third (i.e., the one often called pretesting) is a trial run with a group of respondents.

· An alternative type of screening might involve a client or the research manager who ordered the research.

VII. ETHICAL ISSUES IN SURVEY RESAERCH

· Many ethical issues apply to survey research (see Chapter 5), such as:

· respondents’ right to privacy

· use of deception

· respondents’ right to be informed about the purpose of the research

· need for confidentiality

· need for honesty in collecting data

· need for objectivity in reporting data

QUESTIONS FOR REVIEW AND CRITICAL THINKING/ANSWERS

1. What type of communications medium would you use to conduct the following surveys? Why?

a. Survey of the buying motives of industrial engineers

A personal interview conducted by a professional interviewer is probably the most likely type of communication medium. Industrial engineers are busy individuals. The presence of an interviewer will generally increase participation among professional groups. The answers to questions may be complex and considerable probing may be necessary. Depending upon the nature of the questions, props may be necessary.

b. Survey of the satisfaction levels of hourly support staff

An e-mail/Internet or drop-off survey might be appropriate. Employees will want their responses to be kept confidential.

c. Survey of television commercial advertising awareness

Most likely this will be a telephone survey during the program or within 24 hours after a certain program has been broadcast. Using central location interviewing, program viewers may be quickly contacted at a relatively low cost.

d. Survey of top corporate executives

This is a difficult question to answer. The survey’s objectives will dictate this answer. However, telephone interviewing often is not the best way to contact corporate executives. Secretaries very easily screen these calls. Busy executives may fail to return calls unless they are highly interested in the topic or agency sponsor. Personal interviews may be the best method to gain access. Participation generally will be higher with the personal interview. However, mail or Internet surveys may allow the executive to fill out the questionnaire at his or her own pace.

2. A publisher offers college professors one of four best-selling mass-market books as an incentive for filling out a 10-page mail questionnaire about a new textbook. What advantages and disadvantages does this incentive have?

The major advantage of any incentive is to increase mail survey response rates. The book incentive, generally a fifteen- to twenty-five-dollar value, may stimulate a high rate of return. However, because the incentive is so high, it is more than just a token of appreciation. Some respondents may quickly provide superficial answers to the questionnaire just to receive this substantial incentive. Needless to say, this may cause considerable problems in the data analysis.

3. “Individuals are less willing to cooperate with surveys today than they were 50 years ago.” Comment on this statement.

The research activities of businesses have grown considerably in recent years. With this increased frequency of requests for research, many individuals could become less willing to cooperate.

A major trend that is influencing door-to-door field work is that public safety has declined—at least perceptions of public safety have declined in many parts of the country, especially inner cities. It is now harder to interview people at their doors because many people are fearful and they won’t talk to you. It also may be increasingly risky for interviewers.

Interviewing in shopping centers has increased over the years. Shopping center interviewing has the highest refusal rate because may shoppers are too busy to cooperate.

Over the years, more companies are becoming marketing-oriented and this usually means more surveys. Some companies new at research want to ask consumers everything in a single survey.

Finally, it is more difficult to reach consumers via phone due to caller ID and answering machines used to screen calls, more phone lines dedicated to fax machines and computers, and increased cellphone usage.

4. What do you think should be the maximum length of a self-administered email questionnaire?

In general, the same guidelines for printed mail surveys apply to e-mail surveys. That is, no more than six pages. This is especially true because some respondents will print out the survey, complete it in writing, and return it via regular mail. Furthermore, the extensive differences in the capabilities of respondents’ computers and e-mail software limit the types of questions and the layout of the e-mail questionnaire.

5. Do most surveys utilize a single communication mode (for example, the telephone) as most textbooks suggest?

The answer is generally yes. However, many times we do not have pure telephone surveys or pure personal interviews. Oftentimes a survey will use telephone interviews to contact respondents before personal interviews are conducted.

Sometimes it is necessary to make an advance call, usually by telephone, before a personal interview may be conducted. For example, if a “test” commercial is being inserted into a program that will actually air, calls may be made to sample numbers and sample members. The telephone caller will say, “We would like you to watch (a given) program. When we call we are going to ask you some questions later.” After the airing of the program, respondents are asked whether they remembered any of the commercials.

6. A survey researcher reports, “205 usable questionnaires out of 942 questionnaires delivered in our mail survey converts to a 21.7 percent response rate.” What are the subtle implications of this statement?

The first thing a student should mention is that the terms “usable questionnaires” and “delivered questionnaires” need to be clarified. A usable questionnaire may mean that the researcher eliminated, for whatever reason, some questionnaires because they were incomplete, illegible, or falsified, or for another reason. The term delivered questionnaires indicates that the response rate would be lower than if the total number of questionnaires sent out were used to calculate response rate. Thus it is possible that some questionnaires were marked “return to sender” but the postal service statement is not exactly clear about what this means.

7. Evaluate the following survey designs.

a. A researcher suggests mailing a small safe (a metal file box with a built-in lock) without the combination to respondents with a note explaining that the respondent will be called in a few days for a telephone interview. During the course of the telephone interview, the combination is given and the lock box may be opened.

The purpose of this technique is to allow visual props and visual attitude scales in conjunction with the telephone interview. It may arouse the respondents’ curiosity prior to the interview. This technique increases the expense of the telephone interview and it reduces the speed of telephone interviewing. The cost of mailing is high because the lock box is heavy. The respondent may fumble with the combination.

b. A shopping mall that wishes to evaluate its image places packets including a questionnaire, cover letter, and a stamped return envelope in the mall where customers can pick it up, if they wish.

This self-administered questionnaire is an inexpensive way of surveying shoppers. However, self-selection bias is likely to be high. Only those people who come into the mall will pick up the questionnaire. Are there any distinguishing characteristics of shoppers that visit the mall which might affect the survey results? It should be determined whether those who fill out the self-administered questionnaire differ from those who do not.

c. An e-mail message is sent to individuals who have computers, asking them to complete a questionnaire on a website. Respondents answer the questions and then have the opportunity to play a slot-machine game on the website. Each respondent is guaranteed a monetary incentive but has the option of increasing it by playing the slot-machine game.

This technique has the advantages of other computer interactive techniques; that is, skip patterns can be imbedded into questionnaires on the website and automatically followed when respondents make certain answers to prior questions. The program can also incorporate the respondent’s own vocabulary into subsequent questions. However, it has the major disadvantages of all self-administered questionnaires that are “sent” to a sample of people who have not agreed to cooperate in advance. Thus, there is a potential for a low response rate. Further, it can only be used by people who have personal computers. The sample is likely to be biased unless the subject is highly relevant to users of the Internet. The slot-machine game is an atypical incentive. In many situations, the respondent’s motivation for returning a questionnaire may be increased by offering monetary incentives or premiums. Monetary incentives appear to be the most effective and least biasing incentive. Although money may be useful to all respondents, its primary advantage may be as a way to attract attention and create a sense of obligation. So using the gambling incentive may not be highly motivating.

d. A mall intercept interviewing service is located in a regional shopping center. The facility contains a small room for television and movie presentations. Shoppers are used as sampling units. However, mall intercept interviewers recruit additional subjects for television commercial experiments by offering shoppers several complementary tickets for special sneak previews. Individuals contacted at the mall are allowed to bring up to five guests. In some cases the complementary tickets were offered through newspaper ads in a local newspaper.

This type of facility is extremely advantageous for television commercial tests, taste tests of new “hot” foods, and other research activities that require props and materials that are difficult to transport from house to house. Products may be demonstrated, then placed with target consumers for home use. Rather than having people come to a “downtown” advertising agency or research agency office for an interview, the shopping center facility is more in line with most consumers’ lifestyle patterns.

The major disadvantage of the research store concept is related to the representative nature of the sample. People who shop at a particular shopping center are clearly self-selected. Thus, it is important that the shopping center where a research store is located have a high geographic and demographic dispersion of the population. Quota sampling is often utilized to minimize this problem. “Interviewing” in several stores (within the mall) representing various community economical levels may help control the sample. It should also be noted that not all research is suited to shopping malls. Intercepting someone in a shopping mall for a long, complex interview may not be appropriate. Many respondents may tire of the long interview and respond with “let’s get this over with” answers.

e. Time magazine opts to conduct a mail survey rather than a telephone survey for a study to determine the demographic characteristics and purchasing behavior of its subscribers.

Only subscribers’ responses are included. Nevertheless, if this is the objective of the research, there is not a problem. Further, high response rates are often achieved because subscribers have a positive feeling about their magazines.

8. What type of research studies lend themselves to the use of e-mail for survey research? What are the advantages and disadvantages using e-mail?

In certain circumstances, such as internal surveys of employees or surveys of retail buyers who regularly deal with the organization via e-mail, there are obvious benefits of cheaper distribution and processing fees, faster turnaround time, more flexibility, and less paper chasing when using e-mail surveys. In certain high-technology industries, one can expect that almost all employees will have e-mail access. The key is to find situations where a list has little sampling frame error. A variation of the e-mail survey is the Internet survey where researchers send an e-mail that contains a password and a hyperlink to a web address where the respondent will find a questionnaire. This type of survey works best when respondents are panel members who have already opted-in by agreeing to cooperate with the survey.

9. [Ethics Questions] Comment on the ethics of the following situations:

a. A researcher plans to use invisible ink to code questionnaires to identify respondents in a distributor survey.

Students’ opinions will vary, but they should realize that the purpose of coding questionnaires is primarily to avoid sending duplicate follow-up questionnaires to participants who have already responded.

b. A political action committee conducts a survey about its cause. At the end of the questionnaire, it includes a request for a donation.

The research community is absolutely against this type of action. The purpose of a survey is research and to ask for a donation is an abuse of research. Many organizations have used the “ploy” of a survey to solicit donations. Many have not thought of the damage this does to the research industry.

c. A telephone interviewer calls at 1:00 P.M. on Sunday and asks the person who answers the phone to take part in an interview.

Most research companies do not call respondents when they will be disturbed or upset. Many citizens consider Sunday a day of rest and a day to practice religious beliefs. A basic ethical consideration is to consider the rights and obligations of the other parties in the survey situation.

d. An industrial manufacturing firm wishes to survey its own distributors. It invents the name “Mountain States Research Corporation” and sends out a mail questionnaire under this name.

Like most ethical issues, this tactic is subject to debate. Those favoring this practice suggest that it is not possible to gain truthful information using the company’s own letterhead. To avoid sponsorship bias, the fictitious letterhead should be used. Others feel that it is a deception to create a fictitious company.

e. A questionnaire is printed on the back of a warranty card included inside the package of a food processor. The questionnaire includes a number of questions about shopping behavior, demographics, and customer lifestyles. At the bottom of the warranty card is a short note in small print that says “Thank you for completing this questionnaire. Your answers will be used for further studies and to help us serve you better in the future. You will also benefit by receiving important mailings and special offers from a number of organizations whose products and services relate directly to the activities, interest, and hobbies in which you enjoy participating on a regular basis. Please indicate if there is some reason you would prefer not to receive this information.”

To begin with, this is a legal activity that several companies use to prepare databases for direct marketing activities. There is no actual deception; however, the small print does not make it readily apparent that the questions are not for research purposes but for database building and direct marketing purposes. Another issue to discuss in class is the negative option. That is, only if the person reads the statement and writes a note will they not be put on the data base. The issue revolves around a researcher’s ethics versus direct marketer’s ethics. Ask the students who they side with.

10. [Ethics Question] How might the business research industry take action to ensure that the public believes that telephone surveys and door-to-door interviews are legitimate activities and that firms that misrepresent and deceive the public using research as a sales ploy are not true researchers?

A company that engages in this practice risks alienating the general public who is unable to differentiate between research and telemarketing. Telephone researchers should emphasize to the respondents that no element of selling is involved. Researchers should also inform respondents about why and how their numbers are selected. In other words, researchers should let it clearly be known that research—not sales—is being conducted

11. Why is the mobile phone likely to be an ineffective way or reaching potential respondents in America?

Regulations by the Federal Communications Commission make it illegal for researchers to use automated dialing equipment to call mobile phones. Even if researchers dial the calls by hand, they may not contact anyone who would have to pay for the call – that is, most cellphone users.

12. The American Testing Institute (also known as the U.S. Testing Authority) mails respondents what it calls a “television” survey. A questionnaire is sent to respondents, who are asked to complete it and mail it back along with a check for $14.80. In return for answering eight questions on viewing habits, the institute promises to send the respondent one of 20 prizes ranging in value from $200 to $2,000 – among which are video recorders, diamond watches, color televisions, and two nights of hotel accommodations at a land development resort community. The institute lists the odds of winning as 1 in 150,000 on all prizes except the hotel stay, for which the odds are 149,981 out of 150,000. During a three-month period, the institute sends out 200,000 questionnaires. What are the ethical issues in this situation?

The American Testing Institute appears to be attempting to bilk “respondents” with a scam. The company abuses and misuses the word “survey.” No legitimate survey organization would charge the general public for supplying it with information. It’s just a scam. Further, the prizes have extremely low odds except for the hotel accommodations, which are near certain. The catch, however, is that the hotel accommodations are in a resort community that sells land and vacation residences; recipients most likely will have to listen to a high-pressure sales pitch for time-share vacations.

13. [Internet Question] Go to the Pew Internet and American Life page at http://www.pewinternet.org/ . Several reports based on survey research will be listed. Select one of the reports. What were the research objectives? What were the first three questions on the survey?

Answers to this will depend on the date and survey the student selects.

14. [Internet Question] Go to the NPD Group Web site ( http://www.npd.com ) and click on the Store link. What types of custom and syndicated survey research services does the company offer?

This link is very easy to find, and students should have no trouble obtaining this information. Students’ answers will vary depending when this site is accessed.

15. [Internet Question] Go to the CASRO (Council of American Survey Research Organizations) home page ( http://www.casro.org ). Locate the organization’s mission under the About tab. What are the key aspects of this research organization’s mission?

According to this Web site, the purpose of this organization is to communicate with, to educate, to protect, and to represent professionals in this industry.

RESEARCH ACTIVITIES

1. [Internet Question] Visit this web site: http://www.zoomerang.com . What unique service does this company offer? Then visit this site: http://www.qualtrics.com . How does this service differ from Zoomerang? Create a short survey and e-mail it to 10 of your friends without any advanced notice. At the end of the survey, ask them if they would have responded had they not noticed the survey came from you. What is the response rate? What would it have been if the respondent did not know you?

It appears as though both Web sites offer the same type of product. That is, both provide online survey services. Most students will likely answer that response rates would have been lower if respondents did not know they were sending the e-mail survey.

CASE 10.1 National Do Not Call Registry

Objective: To encourage students to think about the advantages and disadvantages of Internet surveys and telephone interviews as well as to gain an understanding of the Do Not Call Registry law.

Summary: Congress passed a law setting up a National Do Not Call Registry. By law, salespeople may not call numbers listed on this registry, but charities and researchers can still call these numbers. However, a recent Internet survey showed that less than 25 percent of respondents knew that researchers “are allowed to call,” and over half (63 percent) weren’t sure about researchers’ rights under the law.

Questions:

1. Was an online survey the best medium for a poll on this subject? What were some pros and cons of conducting this poll online?

Internet surveys offer the advantages of speed and cost effectiveness, visual appeal and interactivity, respondent participation and cooperation, accurate real-time data capture, callbacks (if part of a panel), personalized and flexible questioning, respondent anonymity, and similar response rates to other kinds of survey research. It appears in this instance that speed and cost effectiveness are the most important advantages relevant in this survey. However, the sample may not be representative because not all consumers have access to the Internet, and security concerns could be issue if hackers can tamper with the survey.

2. How might the results have differed if this poll had been conducted by telephone?

One difference is that perhaps fewer consumers would have participated. Response rates for telephone interviews have fallen due to technological devices that allow consumers to screen calls. Plus, 75 percent of the respondents said they had signed up for the registry, so they would probably not participate in the study if contacted by telephone.

3. As a researcher, how would you address people’s doubts about whether pollsters may contact households listed on the Do Not Call Registry?

It appears than an educational effort is necessary on the part of the industry.

CASE 10.2 Royal Bee Electric Fishing Reel

Objectives: The purpose of this case is to teach students to understand the relationship between research objectives and the information required for the research project. Students have the opportunity to plan a research design and evaluate the type of errors that may be associated with the research design.

Summary: Royal Barton developed an electronic reel for handicapped individuals who would like to fish. The system works on a six-volt battery; when a retrieval button is pushed, the electrical system engages two gears. Only one hand is needed to operate the system. A professional who was temporarily handicapped now uses the reel on a full-time basis.

Questions

1. What business research problem does Royal Barton face? What are his information needs? Outline some survey research objectives for a research project on the Royal Bee system.

The broad marketing problem is whether or not there is a need for this product: To learn if this product appeals to the handicapped individual.

The company must learn if there is a market for the Royal Bee system and how large the market is. This calls for a concept evaluation study and a market segmentation study.

As with most research problems for new product concepts, the researcher must eliminate the scope of the research problem. Exploratory research is often conducted to help in this manner. However, in this case on survey research the student should assume Royal Bee will not be able to conduct exploratory research. He or she may make some assumptions about the nature of the market. For example, the case hints that the primary market consists of handicapped individuals. Although it is mentioned that a professional fisherman uses the Royal Bee system, the case indicates his inducement to use the Royal Bee system was the result of a temporary handicap. Although this professional market should not be ignored in the long run, the researchers and Mr. Barton should come to an agreement about the scope of the research. A logical assumption is to focus on the handicapped market. Needless to say, if the student assumes that the primary focus should be on professional fisherman the research objectives would be somewhat different from those that appear below.

The purpose of the research will be to gather information about the electrical reel system from individuals who have lost the mobility of one arm. This presents a difficult sampling problem, but for the moment we will just mention that the target group has to do with individuals who have lost the mobility of one arm.

• to determine current and past fishing behavior of members of the target group.

• to determine awareness of the Royal Bee reel among members of the target group.

• to measure the concept’s appeal by measuring the target group likes and dislikes.

• to determine perceptions of ability to use the product (ease of use) described in the concept statement.

• to identify potential problems with the concept as perceived by members of the target group.

• to identify any design modifications that might be suggested.

• to measure demographic and psychographic characteristics of the respondents.

• to differentiate those members of the target group that are most likely to purchase the product.

2. What type of survey—personal interview, telephone interview, or mail survey—should be selected?

Students should evaluate the characteristics of each of the potential techniques.

Speed of data collection, a major advantage of telephone survey, does not seem to be an important consideration. Geographical flexibility should not be a key factor because handicapped individuals should not vary in different parts of the country. However, urban/rural differences may be a factor influencing experience and attitudes toward fishing.

It may be assumed that discussing this product is not a very sensitive issue and anonymity of the respondent should not be much of a factor. However, this issue is debatable and a good source for classroom discussion.

The notion of the electronic fishing reel is difficult to conceptualize. There does seem to be a high possibility for respondent misunderstanding and a need for a versatility of questioning. These considerations can tilt the decision toward personal interviews. Cost may be a consideration for Mr. Barton. He may wish to assume that a local study may be conducted.

A major concern is that visual materials probably will have to be shown to explain the concept. Ideally a videotape demonstrating how the Royal Bee system is operated might be the best possible means. However, this causes problems with respondents need to visit the researcher at a “video location” rather than visiting the respondents at their residences. While video can be used in Internet surveys, not all respondents will have computers with capabilities for this to be feasible.

Personal interviews might be obtained by visiting hospitals or other areas where handicapped individuals might be grouped together.

A researcher using telephone interviewers obviously would have a great deal of difficulty explaining this concept over the telephone.

3. What sources of survey error are most likely in a study of this type?

It is very unlikely that simple random sampling could be utilized because a list of potential individuals who are handicapped or have limited mobility in a single arm is not likely to exist.

How can one obtain a list of individuals who have only one arm or the use of one arm? Will physicians provide the confidential information? Probably not. Students, who have not read the sampling chapters, will not be able to solve this problem at this point, but it is a worthwhile issue to raise. There will most likely be some sample selection error whatever the design.

In the survey, the degree of an individual’s handicap may be the first thing a personal interviewer has to establish. Of course, this may be the most sensitive aspect of the study and some possible response bias may occur. Thus, a source of error has to do with qualifying the respondent’s degree of handicap. At some point the interviewers will have to ask or establish if the individual is “handicapped.” Would the statement that someone was handicapped be offensive? Should the phrase be “limited disability in one arm” or something else? The exact wording of the definition of the type of person who would have a need for the Royal Bee system will be a problem.

A tendency for a social desirability bias and/or a bias due to the respondents’ wishing to please the interviewers is also possible. A number of handicapped individuals may be so pleased that someone is thinking of their particular needs that there may be a tendency to say they felt more positive toward the product concept than they actually do. It may also be that the handicapped individual will always answer that they should strive to do their physical best.

Because the concept is a difficult one to understand, it is possible that there will be some unconscious misunderstanding.

Of course, any other sources of survey error are possible, but the above seems to be the most likely.

4. What means should be used to obtain a high response rate?

The answer to this question will, of course, depend on which particular technique is chosen. However, this question provides a springboard to discuss how to increase response rates to personal interviews, telephone surveys and mail surveys.

If a personal interview is chosen, the fact that the interviewer is actually going door-to-door is the main means to bring about a higher response rate. Callbacks will be important in both personal interviews and telephone interviews.

Probably more important, however, is to use some appeal to indicate that this research will benefit the consumer in the long run. Saying, “Your opinions count. We’re trying to build new products for handicapped individuals,” may increase response rate. However, on the other side of the coin, this is likely to induce some potential social desirability bias.