New assignment

Part Three

Research Methods for Collecting Primary Data

Chapter 9

Survey Research: An Overview

Zikmund, W., Babin, B. J., Carr, J., & Griffin, M. (2013). Business research methods (9th ed.). Mason, OH: Cengage Learning.

AT-A-GLANCE

I. Introduction

II. Using Surveys

A. Advantages of surveys

III. Errors in Survey Research

A. Random sampling error

B. Systematic error

C. Respondent error

· Nonresponse error

· Response bias

· Deliberate falsification

· Unconscious misrepresentation

· Types of response bias

1. Acquiescence bias

2. Extremity bias

3. Interviewer bias

4. Social desirability bias

D. Administrative error

· Data-processing error

· Sample selection error

· Interviewer error

· Interviewer cheating

E. Rule-of-thumb estimates for systematic error

F. What can be done to reduce survey error?

IV. Classifying Survey Research Methods

A. Structured/unstructured and disguised/undisguised questionnaires

B. Temporal classification

· Cross-sectional studies

· Longitudinal studies

· Consumer panels

V. Total Quality Management and Customer Satisfaction Surveys

A. What is quality?

B. Internal and external customers

C. Implementing total quality management

LEARNING OUTCOMES

1. Define surveys and explain their advantages

2. Describe the type of information that may be gathered in a survey

3. Identify sources of error in survey research

4. Distinguish among the various categories of surveys

5. Discuss the importance of survey research to total quality management programs

CHAPTER VIGNETTE: Media Phones—The Next Wave of Communication Technology?

What’s next in the world of electronic communication? According to In-Stat, a research firm, it will be the media phone. The media phone represents a new category of broadband multimedia devices that has the potential to become the 4th screen in the home, complementing the PC, TV, and mobile handset. It will combine the power of a PC with the always-on functionality of the home telephone. In-Stat has conducted extensive survey research to learn consumer perceptions, attitudes, and desires regarding media phones and estimates the consumer market for this device could reach nearly 50 million units and $8 billion in worldwide revenue by 2013. In-Stat’s research also determined the characteristics of the product customer desired across consumer age groups and geographic locations.

SURVEY THIS!

Students are asked to review the online survey and address the following:

1. Identify at least three different sources of potential error or bias and offer suggestions on how this error can be reduced.

2. Classify this survey on the structured, disguised, and temporal dimensions.

3. How could this survey help your academic institution implement a total quality management program?

RESEARCH SNAPSHOTS

· Intuit Gets Answers to Satisfy Customers

Intuit, the maker of Quicken, QuickBooks, and Turbo Tax software for accounting and tax preparation, has enjoyed years of growth and profits, in part due to its efforts to learn what customers want. A “net promoter survey” is used to determine which customers are likely to promote the products to others. To learn more about why customers are satisfied, these individuals are invited to go online and provide more detailed opinions, and Intuit has made changes based on these comments. The company also conducts direct observation of customers using their products, and they have learned that small business owners were not always familiar with the accounting jargon, so products were changed to include every day terminology.

· Overestimating Patient Satisfaction

The challenge facing researchers interested in satisfaction is whether the responses come from a cross-section of customers. Some researchers studied data from patient satisfaction surveys and found that the actual data closely matched simulated data in which responses were biased so that responses were more likely when satisfaction was higher. In other words, more-satisfied patients were more likely to complete and return the survey, which would overestimate satisfaction.

· The “Mere -measurement” Effect

The mere-measurement effect means that simply answering a question about intentions will increase the likelihood of the underlying behavior – if the behavior is seen as socially desirable. If it is not, answering the question tends to decrease the likelihood of the behavior. One study found this to be true for eating fatty food or flossing. However, this effect did not occur if the surveys indicated that they were sponsored by groups that would be likely to want to persuade the subjects (i.e., the American Fruit Growers Association and the Association of Dental Products Manufacturers). Subjects decreased their frequency of flossing if they took the supposedly manipulative survey that asked about flossing. Thus, if individuals receive information that puts them on their guard against persuasion, the mere-measurement effect is lessened and sometimes even generates the opposite behavior.

TIPS OF THE TRADE

· Surveys are the most widely used method of collecting primary quantitative data for business research, but a common error is to begin questionnaire design and the survey process too soon. Before undertaking primary research activities:

· Be sure to fully exhaust secondary research sources.

· Have a clear understanding of the research issues and objectives.

· Error exists in all survey research.

· Random sampling error is present due to chance variation in the sample elements and can only be addressed through large sample size.

· Systematic error is due to a flaw in the research design or execution, and it is the job of the researcher to minimize this error.

OUTLINE

I. INTRODUCTION

· Surveys require asking people (respondents) to provide answers to written or spoken questions.

· Questionnaires or interviews collect data through the mail, on the telephone, online, or face-to-face.

· A survey is defined as a method of collecting primary data based on communication with a representative sample of individuals.

· Surveys provide a snapshot at a given point in time.

· The more formal term, sample survey, emphasizes that the purpose of contacting respondents is to obtain a representative sample of the target population

II. USING SURVEYS

· The type of information gathered varies considerably depending on its objectives.

· Most survey research is descriptive research, which attempts to identify and explain a particular business activity.

· Most surveys typically have multiple objectives; few gather only a single type of factual information.

· Although consumer surveys are a common form of business research, not all survey research is conducted with the ultimate consumer.

· Although surveys are often conducted to quantify certain factual information, certain aspects of surveys may also be qualitative. For example, in new-product development the qualitative objective of a survey is often to test and refine new-product concepts.

· Advantages of Surveys

· Provide a quick, inexpensive, efficient, and accurate means of assessing information about a population.

· Surveys can also be poorly conducted and certain errors can occur to render such surveys useless.

III. ERRORS IN SURVEY RESEARCH

· Exhibit 9.1 outlines the various forms of survey error which can affect the accuracy of a survey.

· Survey error can be broken down into two major sources: random sampling error and systematic error.

· Random Sampling Error

· Most surveys try to portray a representative cross-section of a particular target population, but even with technically proper probability sampling random sampling errors will occur because of chance variation.

· Without increasing sample size, these statistical problems are unavoidable.

· However, random sampling errors can be estimated (see Chapters 16 and 17).

· Systematic Error

· Systematic error results from some imperfect aspect of the research design or from a mistake in the execution of the research.

· Also called nonsampling errors.

· A sample bias exists when the results of a sample show a persistent tendency to deviate in one direction from the true value of the population parameter.

· Two general categories are: respondent error and administrative error.

· Respondent Error

· If the respondents do not cooperate or do not give truthful answers then two major types of respondent error may cause sample bias.

· Nonresponse Error

· To use the results of a survey the researcher believes that those who did respond to the questionnaire are representative of those who did not.

· If only those who responded are included in the survey then nonresponse error will occur.

· People who are not contacted or who refuse to cooperate are called nonrespondents.

· The number of no contacts has been increasing because of the proliferation of answering machines and growing usage of Caller ID to screen telephone calls.

· Refusals occur when people are unwilling to participate in the research.

· Comparing the demographics of the sample with the demographics of the target population is one means of inspecting for possible biases in response patterns.

· Self-selection may also occur in self-administered questionnaires; in this situation, only those who feel strongly about the subject matter will respond, causing an over representation of extreme positions

· Response Bias

· Response bias occurs when respondents tend to answer with a certain slant.

· This bias may be caused by an intentional or inadvertent falsification or by a misrepresentation of the respondents’ answers.

· Deliberate Falsification

· Occasionally people deliberately give false answers.

· They may become bored with the interview and provide answers just to get it done.

· They may try to appear well informed.

· They give answers to please the interviewer.

· Average-person hypothesis – individuals prefer to be viewed as average, so they alter their responses to conform more closely to their perception of the average person.

· Unconscious Misrepresentation

· Response bias can arise from the question format, the question content, or some other stimulus.

· In many cases consumers cannot adequately express their feelings in words.

· Language differences also may be a source of misunderstanding.

· Time lapse influences people’s ability to precisely remember and communicate specific factors.

· Some consumers may unconsciously avoid facing the realities of a future buying situation.

· Types of Response Bias

1. Acquiescence Bias – a tendency to agree (or disagree) with all or most questions. Particularly prominent in new-product research.

2. Extremity Bias – some individuals tend to use extremes when responding to questions; others consistently avoid extreme positions and tend to respond more neutrally.

3. Interviewer Bias – the interviewer’s presence influences respondents to give untrue or modified answers.

4. Social Desirability Bias – respondent wishes to create a favorable impression or save face in the presence of an interviewer.

· Administrative Error

· The results of improper administration or execution of the research task are administrative errors.

· Such errors are inadvertently caused by carelessness, confusion, neglect, omission or some other blunder.

· There are four types of administrative error:

· data-processing error

· sample selection error

· interviewer error

· interviewer cheating.

· Data-Processing Error

· Processing data by computer is subject to error because data must be edited, coded, and entered into the computer by people.

· Can be minimized by establishing careful procedures for verifying each step in the data-processing stage.

· Sample Selection Error

· Sample selection error is systematic error that results in an unrepresentative sample because of an error in either the sample design or the execution of the sampling procedures.

· For example, selecting a sample from the phone book will have some systematic error because unlisted numbers are not included; stopping respondents during daytime hours in shopping centers excludes working people who shop by mail, Internet, or telephone.

· Interviewer Error

· Introduced when interviewers record answers but check the wrong response or are unable to write fast enough to record answers verbatim.

· Selective perception may cause interviewers to misrecord data that do not support their own attitudes and opinions.

· Interviewer Cheating

· Occurs when an interviewer falsifies entire questionnaires or fills in answers to questions that have been intentionally skipped.

· The term curb-stoning is sometimes used to refer to interviewers filling in responses for respondents that do not really exist.

· Rule-of-Thumb Estimates for Systematic Error

· Sampling error may be estimated using certain statistical tools, but ways to estimate systematic error are less precise.

· Many researchers have found it useful to use some standard of comparison in order to understand how much error can be expected. For example, one cable TV company knocks down the number of people saying that they intend to purchase the service by a “ballpark ten percent” because previous experience has indicated a ten percent upward bias on the intention questions.

· What Can Be Done To Reduce Survey Error?

· Chapters to follow discuss various techniques for reducing bias in survey research.

· Chapter 15 on questionnaire design discusses the reduction of response bias.

· Chapters 16 and 17 discuss the reduction of sample selection and random sampling error.

IV. CLASSIFYING SURVEY RESEARCH METHODS

· Surveys can be classified in three ways:

1. method of communication, such as personal interview, telephone interviews, mail surveys, and Internet surveys (all discussed in Chapter 10)

2. the degrees of structure and disguise in the questionnaire

3. the time frame in which the data are gathered (temporal classification)

· Structured/Unstructured and Disguised/Undisguised Questionnaires

· A structured question limits the number of allowable responses.

· Unstructured questions do not restrict the respondent’s answers (e.g., an open-ended question).

· The researcher can also use undisguised questions or disguised questions.

· Using disguised questions is particularly advisable if the subject matter is of a threatening or sensitive nature.

· Other questions do not need to be disguised as it is assumed that the respondent is willing to reveal the information.

· Questions can be categorized according to their degree of structure and disguise.

· Unstructured-disguised

· Structured-undisguised

· Unstructured-undisguised

· Structured-disguised

· However, it is not always easy to categorize surveys as the categories are not clear-cut, and most surveys are a hybrid of structured and unstructured questions.

· Temporal Classification

· Cross-Sectional Studies

· Cross-sectional studies collect data at a single point in time.

· Various segments of the population are sampled so that relationships among variables may be investigated by cross-tabulation.

· Most business research surveys fall into this category.

· Longitudinal Studies

· In a longitudinal study respondents are questioned at multiple points in time.

· Purpose is to examine continuity of response and to observe changes that occur over time.

· Longitudinal studies which involve two or more samples at different times are called cohort studies because similar groups of people who share a certain experience during the same time interval (cohorts) are expected to be included in each sample.

· In applied business research, a longitudinal study that uses successive samples is called a tracking study because successive waves are designed to compare trends and identify changes in variables.

· Conducting surveys in waves with two or more sample groups avoids the problem of response bias resulting from a prior interview, but the researcher can never be sure that the changes in the variable being measured are not actually due to having different people in the sample.

· Consumer Panel

· A longitudinal study that gathers data from the same sample of individuals or households over time.

· Diary data that are recorded regularly over an extended period enable the researcher to track repeat-purchase behavior and changes in purchasing habits that occur in response to changes in price, special promotions, or other aspects of marketing strategy.

· Expensive to maintain.

· Often managed by contractors who offer their services to many organizations (i.e., National Family Opinion (NFO), Inc., Markettools, Inc.).

· Internet panels have grown in popularity.

· The first questionnaire typically includes questions about product ownership and usage, pets, family members, and demographic data.

· Economical means for marketers to reach respondents who own their products.

V. TOTAL QUALITY MANAGEMENT AND CUSTOMER SATISFACTION SURVEYS

· Total quality management is a business strategy that emphasizes market-driven quality as a top priority.

· Involves implementing and adjusting the firm’s business activities to assure customers’ satisfaction with the quality of goods and services.

· In an organization driven by the quality concept, business research plays an important role in the management of total product quality.

· What Is Quality?

· Organizations used to define quality by engineering standards, but most companies no longer see quality that way.

· Now, managers believe the level of quality is the degree to which a good or service corresponds to buyers’ expectations.

· Internal and External Customers

· A focus on customers must include more than external customers.

· Every employee must know exactly who his or her customers are and what output internal and external customers expect.

· Also important to know how customers perceive their needs are being met.

· Implementing Total Quality Management

· Requires considerable survey research.

· A firm must routinely ask customers to rate it against its competitors.

· Firm must periodically measure employee knowledge, attitudes, and expectations.

· Must monitor company performance against benchmark standards.

· Exhibit 9.3 illustrates the total quality management process.

· Overall tracking of quality improvement requires longitudinal research.

· Stages include:

· Commitment and exploration stage.

· Management makes a commitment to total quality assurance.

· Researchers explore external and internal customers’ needs and beliefs.

· Benchmarking stage

· Research must establish quantitative measures that can serve as benchmarks or points of comparison against which to evaluate future efforts.

· Surveys must establish initial measures of overall satisfaction, of the frequency of customer problems, and of quality ratings for specific attributes.

· Researchers must identify the company’s or brand’s position relative to competitors’ quality positions.

· Initial quality improvement stage

· Establishes a quality improvement process within the organization.

· Establish performance standards and expectations for improvement.

· Continue to track satisfaction and quality ratings in successive waves.

· Continuous quality improvement

· Consists of many consecutive waves with the same purpose – to improve over the previous period.

· Allow employees to initiate problem solving without a lot of red tape.

· Reward performance.

· Measure performance against customers’ standards – not against standards determined by quality engineers within the company.

· Continuous quality improvement is an ongoing process.

· Process applies to service quality as well.

QUESTIONS FOR REVIEW AND CRITICAL THINKING/ANSWERS

1. Name several nonbusiness applications of survey research.

Students should be able to provide numerous examples. Many not-for-profit organizations such as zoos, museums, charities, and the like conduct patron or donor research. Their objective is often to determine whether or not individuals’ satisfaction can be increased. Survey research is often used by federal, state and local governments to find the public’s reaction to political programs.

2. What is self-selection bias? How might we avoid this?

Self-selection bias is a problem that frequently plagues self-administered questionnaires, such as a satisfaction card left at the table at a restaurant. It distorts surveys because they overrepresent extreme positions while underrepresenting responses from those who are indifferent. This is a problem that frequently plagues self-administered questionnaires, and one way to avoid it is to use tactics to encourage participation. Comparing demographics of the sample with the demographics of the target population is one means of inspecting for possible biases in response patterns. If a particular group is underrepresented or if any potential biases appear in a response pattern, additional efforts should be made to obtain data form the underrepresented segments of the population.

3. Do surveys tend to gather qualitative or quantitative data? What types of information are commonly measured with surveys?

A survey is defined as a method of collecting primary data based on communication with a representative sample of individuals. Because most survey research is descriptive research, the term survey is most often associated with quantitative findings, but some aspects of surveys may also be qualitative. The type of information gathered in a survey varies considerably depending on its objectives. Typically, surveys attempt to describe what is happening or to learn the response for a particular activity. Identifying characteristics of target markets, measuring consumer attitudes, and describing consumer purchasing patterns are common survey objectives. Questions about product use and desirable features help with product development and advertising messages. Demographic information and information on media exposure might also be collected in the survey to help plan a market segmentation strategy.

4. Give an example of each type of error listed in Exhibit 9.1.

Students should be able to generate numerous examples of errors other than those provided in the textbook.

Some possible answers are:

Random sampling error—A random sample includes all millionaires. They occurred in the sample by chance alone.

Administrative error—Sample selection error—Sample frame for a tennis player survey is taken from the membership list of Forest Hills Tennis Club.

Administrative error—Interviewer error—Interviewer checks a “4” rather than a “3” on a questionnaire.

Administrative error—Interviewer cheating—An interviewer guesses respondent income rather than directly asking this embarrassing question. Interviewer doesn’t go to a house in the sample because it looks too “low class.”

Administrative error—Data processing error—A keyboard operator skips a questionnaire because of negligence. Coders may not understand a particular jargon and miscode the response.

Nonresponse error—A mail survey among executives has a low response rate.

Self selection error results if the loyal fans of The Disney Channel are the one’s filling out in the Disney Channel’s questionnaire magazine.

Response bias (general)—Unconscious misrepresentations—People unconsciously say they do not drink many soft drinks.

Response Bias (General)—A bank officer is asked what percentage of the time she spends talking on the phone, in meetings, working on a computer, and on other on-the-job activities. Most people really don’t know how much time they spend on these different activities. Few people have really ever thought much about the actual percentage of time they spend but they are prone to answer questions about these percentages. They may unconsciously falsify or deliberately falsify the answers depending on the particular circumstances of the questionnaire.

In a course evaluation survey asking students who was the best professor they had taken a course from during their college “career,” students may have a tendency to mention professors whose courses they are currently enrolled in at the time of the survey.

Response Bias—Interviewer Bias—Female interviewers asking questions about sexual harassment or attitudes about being a woman in personal selling may get slightly different answers to the same question than would men asking the same questions.

Response bias (general)—Deliberate falsification—It has been said that “The average age of women over 40 is under 40.” This may be because of a deliberate understatement of age for women (or men).

Response bias—Acquiescence—It is easy to say yes, that you’ll buy a new product, when asked by an interviewer.

Response bias—Extreme bias—Someone “hates” all politicians on a questionnaire but doesn’t think about relative attitudes or relative positions. For some individuals, the understanding of the extreme positions (categories) is often clearer than the middle positions (category) and this may influence individuals to respond in the extreme positions.

Response bias—Interviewer bias—Interviewer is an employee of sponsoring firms and he frowns when negative answers are given. Interviewer reads the question: “Please tell me the names of all soft drinks you can think of in the next ten seconds.” The individual says: “Coke.” The interviewer interrupts and says: “Regular or Diet?” Then allows 20 seconds for additional answers.

Response bias—Social desirability bias—A respondent feels he or she must claim to always drive at speeds under 70 m.p.h. When a respondent doesn’t know an answer he may feel compelled to give some kind of response in order not to appear to be uninformed or unintelligent.

5. In a survey, chief executive officers (CEOs) indicated that they would prefer to relocate their businesses in Atlanta (first choice), San Diego, Tampa, Los Angeles, or Boston. The CEOs who said they were going to build the required office space in the next year were asked where they were going to build. The CEOs indicated they were to build in New York, Los Angeles, San Francisco, or Chicago. Explain the difference between these two responses.

The difference between the chief executive officers’ answers of where they want to go and where they will actually end up reflects the fact that the two questions address different issues. Atlanta and other cities with high quality of life reflect preferences but companies tend to expand into areas where they already have operations. And for most of the companies those are the major business centers, such as New York.

6. What potential sources of error might be associated with the following situations?

a. In a survey of frequent fliers age 50 and older, researchers concluded that price does not play a significant role in airline travel because only 25 percent of the respondents check off price as the most important consideration in determining where and how they travel, while 35 percent rated price as unimportant.

There is a potential for response bias due to social desirability, which may be deliberate falsification or unconscious misrepresentation. Furthermore, the frequent fliers, who are likely to be business people who have someone else arrange for travel, may not know the importance of price to the actual decision maker. Thus, potential administration error due to sample selection is another likely source of error.

b. A survey of voters finds that most respondents do not like negative political ads—that is, advertising by one political candidate that criticizes or exposes secrets about the opponent’s “dirty laundry.”

This question has the potential of response bias because the question asks about negative advertising in general. It may be that in an advertisement for a specific candidate the viewer is persuaded by the ad. This is reflected in a general phenomenon in which people tend to say “I hate advertising (general), but I love those Bud Light ads (specific).” Also, people do like “dirty laundry” but may not admit it. Remember people do buy and read those negative tabloid papers.

c. Researchers who must conduct a 45-minute personal interview decide that they will offer $25 to each respondent because they believe that people who will sell their opinions are more typical than someone who will talk to a stranger for 45 minutes.

In general, researchers avoid paying people for responses. Some individuals would welcome the money and say things to please the interviewers (interviewer bias). Also this has the potential of self-selection bias.

d. A company’s sales representatives are asked what percentage of the time they spend making presentations to prospects, traveling, talking on the telephone, participating in meetings, working on the computer, and engaging in other on-the-job activities.

A respondent in this typical situation will give an answer to questions about time spent in various activities. However, it is very likely that the respondent has some memory problems and does not know the exact percentages of time spent on each activity. Asked for an answer, in general, the individual will tend to give a generalized answer, reflecting the ideal or expected behavior for situations. This is an example of unconscious misrepresentation.

e. A survey comes with a water hardness packet to test the hardness of the water in a respondent’s home. The packet includes a color chart and a plastic strip to dip into hot water. The respondent is given instruction in six steps on how to compare the color of the plastic strip with the color chart that indicates water hardness.

A major source of error would be nonresponse error. This task requires considerable effort on the part of the respondent, and many will simply refuse to participate.

7. A researcher investigating public health issues goes into a junior high school classroom and asks the students if they have ever smoked a cigarette. The students are asked to respond orally in the presence of other students. What types of error might enter into this process? What might be a better approach?

A social desirability bias will most likely occur, but perhaps not in the direction one may first assume. Children of this age know they should not smoke, so they may say they don’t in the presence of an adult interviewer. However, depending on the peer pressure a student may be facing, he or she may reply “yes” when, in fact, they have never smoked to appear “cool” to the other students. Either way, it is still a social desirability bias at work. A better approach would be to use an anonymous survey.

8. A survey conducted by the National Endowment for the Arts asked, “Have you read a book within the last year?” What response bias might arise from this question?

A direct question like this would often be answered “yes” to keep the respondent from being embarrassed. Because most feel that reading books is socially desirable, they would probably answer yes. Also, people may define “book” in various ways. For example, some poorly educated people who read magazines may perceive this “heavy reading” to be like “book reading” so they would say yes.

9. Name some common objectives of cross-sectional surveys.

Typically, surveys attempt to describe what is happening or to learn the response for a particular activity. Identifying characteristics of target markets, measuring consumer attitudes, and describing consumer purchasing patterns are common survey objectives. Questions about product use and desirable features help with product development and advertising messages. Demographic information and information on media exposure might also be collected in the survey to help plan a market segmentation strategy.

10. Give an example of a political situation where longitudinal research might be useful. Name some common objectives for a longitudinal study in a business situation.

The presidential campaign is a long, involved process. Candidates almost always track their progress during the campaign for a year or more prior to the election.

Business applications of longitudinal studies include cohort studies, tracking studies, and consumer panels. Cohort studies use successive samples; its researchers survey several different samples at different times. These are called cohort studies because similar groups of people share a certain experience during the same time interval (cohorts) are expected to be included in each sample. A longitudinal study that uses successive samples is called a tracking study because successive waves are designed to compare trends and identify changes in variables such as consumer satisfaction, brand image, or advertising awareness. A longitudinal study that gathers data from the same sample of individuals or households over time is called a consumer panel. Diary data that are recorded over an extended period enable the researcher to track repeat-purchase behavior and changes in purchasing habits that occur in response to changes in price, special promotions, or other aspects of marketing strategy.

11. What are the advantages and disadvantages of using consumer panels?

Consumer panels involve a longitudinal study which includes gathering data from the same sample over time. The panelists record their purchasing habits in a diary for a set period of time. Panels are generally expensive and, thus, are usually managed by contractors that specialize in maintaining consumer panels. Such panels enable the investigator to keep track of repeat purchase behavior habits affected by changes in price, special promotions, or other aspects of marketing strategies. Panels generally give high response rates to surveys because the panel members have already agreed to cooperate with an organization’s research.

12. Search either your local newspaper, the Wall Street Journal, or USA Today to find some stories derived from survey research results. Often, these stores deal with public opinions about product complaints, product consumption, job-related issues, marriage and family, public policy issues, or politics. Was the study’s methodology appropriate to draw conclusions?

This is an individual student project and the answer will vary. The professor should encourage students to look at USAToday, which often reports survey findings. Many times these are presented in a box on the front page of each section of the paper. While the source is given, rarely is the methodology presented in these highlight boxes, however.

13. Suppose you are the research director for your state’s tourism bureau. Assess the state’s information needs, and identify the information you will collect in a survey of tourists who visit your state.

The state will want to know how many tourists visit the state and where they live. The state will want to know if the state is the final destination or a place where the tourists are just passing through. The state may wish to access visitors’ knowledge about the state’s facilities and their image of the state. This type of information can be easily collected in surveys.

14. [Ethics Question] A researcher sends out 2,000 questionnaires via e-mail. Fifty are returned because the addresses are inaccurate. Of the 1,950 delivered questionnaires, 100 were completed and e-mailed back. However, 40 of these respondents wrote that they did not want to participate in the survey. The researcher indicates the response rate was 5.0 percent. Is this the right thing to do?

The 5% (100/2000 = .05 or 5%) response rate figure is a bit deceptive. Though the chapter did not cover how to report response rates, students should discuss whether not the 40 responses that indicated that they did not want to participate should be included as “responses.” Furthermore, should the 50 undelivered questionnaires be included in the number of questionnaires sent out?

RESEARCH ACTIVITES

1. [Internet Question] Go to Survey Monkey (http://www.surveymonkey.com). Then, visit http://www.mysurvey.com. What is the difference between the two web sites in terms of the services they provide to users?

Survey Monkey’s purpose is “to enable anyone to create professional online surveys quickly and easily.” Mysurvey.com is similar to a consumer panel where participants are notified via e-mail that a survey is ready for them, and they go to this site, log in, and complete the survey.

2. [Internet Question] The National Longitudinal Surveys (NLS) conducted by the Bureau of Labor Statistics provide data on the labor force experience (current labor force and employment status, work history, and characteristics of current or last job) of five groups of the U.S. population. Go to http://www.bls.gov/opub/hom/homtoc.htm to learn about the objectives and methodology for this study. How accurate do you believe the information reported here really is? What sources of error might be present in the data?

Instructors will have to explore this site themselves to get a feel for what information is provided as there is quite a bit of information. Students will probably indicate that the data is accurate as it is collected by the government.

3. Ask a small sample of students at your local university to report their GPA. Then, try to find the average GPA of students at your school. If you have to, ask several professors to give their opinion. Does it seem that the student data are subject to error? Explain.

Students’ responses will vary, but they will probably discuss response bias as a potential error with this survey. A response bias occurs when respondents tend to answer questions with a certain slant. Respondents may not truthfully give their GPAs because they may be embarrassed that it is too low or too high. Students may also exhibit social desirability bias because they wish to create a favorable impression or save face in the presence of an interviewer.

4. [Internet Question] Located at the University of Connecticut, the Roper Center is the largest library of public opinion data in the world. An online polling magazine and the methodology and findings of many surveys may be found at http://www.ropercenter.uconn.edu. Report on an article or study of your choice.

Students’ responses will vary, but they should not have difficulty finding reports. Instructors may want to review this website and assign students to review specific reports and have them report the findings to the rest of the class to get more variety.

CASE 9.1 SAT and ACT Writing Tests

Objective: To appreciate how survey research can be used to develop products and services that meet customers’ needs.

Summary: The SAT and ACT college entrance exams once were completely multiple-choice, but both tests recently began including an essay portion (which is optional for the ACT). One survey suggests that some admission officers harbor doubts about the essay tests. Research conducted by ACT, Inc. reported that among schools it surveyed, only about one-fifth require the writing portion of the exam, and another one-fifth merely recommend the essay. Kaplan, Inc., which markets test preparation services, asked colleges whether they would be using the SAT writing test to screen candidates, and 47 percent said they would not use it, and another 22 percent said they would use it but give it less weight. Kaplan also surveys students and says on its Web site that “More than 25% of students run out of time on the essay!”

Questions:

1. What survey objectives would ACT have in asking colleges how they use its essay test? What objectives would Kaplan have for its survey research?

ACT, Inc. would be interested in learning if colleges will actually use the results of the essay in making acceptance decisions. If schools are not using it, then ACT, Inc. might decide against including it as a part of the exam. Kaplan would also like to know if schools are using the essay test in making acceptance decisions, because if they are, Kaplan’s customers (i.e., students) will need to include this as part of is test preparation services.

2. If you a marketer for the College Board (the SAT’s company) or ACT, Inc., what further information would you want to gather after receiving the results described here?

They would probably want to know if colleges will be increasing its use of the essay test in the future.

3. What sources of error or response bias might be present in the surveys described here?

The reported use of the essay test might be higher than it actually is due to acquiescence bias and social desirability bias.

CASE 9.2 Walker Information Group

Objective: To appreciate how consumer research firms can help clients with various products and services to do market research.

Summary: Walker Information Group is one of the largest research firms in the world. It is based in Indianapolis, Indiana. Founded in 1939 as a field interviewing service by Frank Walker’s mother, it is now a full service research supplier.

The Walker Information Group specializes in business, health care, and consumer research, as well as database marketing. The company is organized into size strategic business units.

Walker Research conducts traditional market research services ranging from questionnaire design and data collection to advanced analysis and consultation. Walker has expertise helping companies measure how their actions are perceived by the audiences most important to them, and how this affects their image, reputation, corporate citizenship, recruiting, sales and more.

Data Source is a business unit primarily concerned with data collection and processing data. They specialize in telephone data collection.

Customer Satisfaction Measurement (CSM), as the name implies, specializes in measuring customer satisfaction and helping clients improve their relationships with customers.

CSM Worldwide Network spans more than fifty countries. It is the first international network of professional research and consulting businesses dedicated to customer satisfaction measurement and management. The CSM Worldwide Network assures that multi-country customer satisfaction research is consistent by taking into account local conditions and cultural norms. Network members are trained to use consistent methods that allow standardization and comparability of information from country to country.

Walker Direct designs and develops databases and implements direct marketing programs that help generate leads for businesses and raise funds for nonprofit organizations.

Walker Clinical is a health-care product use research company. Walker helps pharmaceutical, medical device and consumer-product manufacturers test how well new products work and how customers like them.

Questions

1. What type of custom survey research projects might a research supplier like Walker Information, Inc. (www.walkerinfo.com) conduct for its clients?

They do surveys for consumer soft goods and grocery store products. For example, they might do a study to see what people are doing with their baby’s diapers (and from this might come an idea or a concept for a product). From this idea, a prototype product may be developed in a lab and then introduced to a test market, where Walker does more consumer research. They can do name testing to learn what names consumers associate with a certain product. They can do customer satisfaction research.

2. What stages are involved in the conducting of a survey? For which stages might a client company hire a research supplier like Walker Research? Data Source?

The stages are problem definition (possibly using exploratory research), statement of the research objectives, selection of the type of survey (e.g., mail, telephone, mall intercept, door-to-door, etc.) and design of the questionnaire, selection of the sample design, data gathering, data processing and analysis, interpretation of the findings, and preparation of the conclusions and report.

The company may be hired only for data collection or for help with the entire research process. The company is organized into three strategic business units for business research.

3. What is the purpose of customer satisfaction measurement?

Effective executives who subscribe to a total quality management philosophy, however, believe that the product’s quality must go beyond acceptability for a given price range. Rather than having consumers relieved that nothing went wrong, consumers should experience some delightful surprises or reap some unexpected benefits. In other words, customer satisfaction is more than just meeting minimum standards. Knowing and tracking the level of customer satisfaction is essential to a total quality management program. Managers should try to establish what quality level their target markets expect, and then to produce and market goods and services that continually surpass expectations.

Implementing a total quality management program requires considerable survey research. A firm must routinely ask customers to rate it against its competitors. It must periodically measure employee knowledge, attitudes, and expectations. It must monitor company performance against benchmark standards. It must determine whether customers found any delightful surprises or major disappointments. In other words, a total quality management strategy expresses the conviction that to improve quality an organization must regularly conduct surveys to evaluate quality improvement.

4. What measures, other than findings from surveys, might a company use to evaluate the effectiveness of a total quality management program?

There are many measures from observation and secondary data that can be used to measure quality. For example, percentage of on-time delivery is a frequent measure of service quality. Counting the number of complaint letters or dissatisfied customers who call company toll-free telephone lines is another source of measuring the effectiveness of total quality management programs.