s
dua
Week-3-Set-3.pdf
Survey design, Programming & Management
Survey research & questionnaire Design
2
Survey Research and Questionnaires
• Questionnaire is the broader term; • Survey research represents a specific application of questionnaires
• In a survey one samples a population and asks each respondent in the sample a fixed set of questions
• The goal is projectable results—answers that are descriptive of the larger population in the same
proportion as for the sample
• For example: • 57% would visit Best Buy to purchase Wi-Fi
networking gear
• 85% of people have heard of your brand • On average … customers spend 2.5 hours
researching a major purchase online before
deciding which brand to purchase
3
Why Do a [Descriptive] Survey?
• To get market data • Survey % x known base x survey $$ = market size
• For complex inter-group comparisons on multiple descriptors • E.g., opinions and perceptions of large vs. small buyers, brand A vs. brand B owners, industry 1 vs. 2 vs. 3
• For purposes of developing a segmentation scheme or profiling segments
• To track a numerical quantity over time • E.g., satisfaction, buying intentions, awareness
4
Survey Strengths
• Provides large sample precision
• Supplies objectivity of large probability samples
• Wealth of high-powered analytic tools available
• Reveal complex, multi-level distinctions among groups
• Inter-temporal comparison • Intertemporal choice is the process by which people make decisions about what and how much to do at
various points in time, when choices at one time influence the possibilities available at other points in time.
5
Survey Weaknesses
• Gives broad but shallow data • What but not why
• Can’t reveal what you don’t know to ask about
• Like interviews, represent self-reports (which may include biases)
• Participation in surveys is intrinsically unrewarding
• Large amount of nonresponses … which can lead to biased data
6
Survey Research & Questionnaires • Effective survey research requires: • Mastery of questionnaire design • Mastery of sampling • Effective supporting procedures, which include: • Envisioning what sort of descriptive information would best address some research question of interest
• Fielding the questionnaire and managing data collection • Dealing with non-response • Data analysis
7
Types of Questionnaires
• Heterogeneous (typical) • Mix of classification data (e.g., demographics), behaviors, ratings of diverse topics, beliefs, etc.
• Opinions • Focus on agree/disagree opinion items and importance ratings
• Lifestyle • Focus on activities, interests, and opinions (generally for purposes of segmentation)
• Satisfaction • Focus on global and specific evaluations and on prioritization of same
• Readership • Classification questions combined with use of various media outlets
• Concept tests
8
Questionnaire Design Process 1. Generate potential topics, issues, items
2. Construct a first draft, with actual questions, in a sequence,
along with tentative answer categories
3. Vet against research questions and feasibility, then edit/redraft
as necessary
4. Circulate 2nd draft for comments
5. Finalize and program the survey
6. Pretest
7. Tweak based on pretest and administer
9
Design with the end in mind
• Mock up the crucial tables that will be produced via analysis of answers - think about that you want to include
in your final report
• Now, looking at these mock-ups, how do you feel? • Is this the information you need to make decisions? • Are the answer categories exhaustive?
• This is a great way to identify superfluous questions and also missing questions
10
Content Generation for Questionnaires
• Revisit your research objectives • Generate specific topics and issues to reflect information required
• Begin to generate specific questions • Emphasis should be on closed-ended questions • Make preliminary decision about answer categories: yes/no, multiple choice, ratings, rankings, etc.
• Re-use categories from prior research whenever possible, to promote comparability
• Review answer categories, with an eye to data analysis, to assist in judging whether the questionnaire will address the research question
11
Questionnaire Development: Do’s
• Be focused on a single issue or topic • What type of food do you prefer? • What type of food do you prefer when you go out to eat dinner?
• The question should be interpreted the same way by all respondents
• What is you favorite TV show? • What is your favorite cable television show?
• The question should use the respondents’ core vocabulary • Do you think the ad strives for parsimony? • Do you think the ad is simple?
12
Questionnaire Development: Do’s
• The question should be brief and grammatically simple
• What are the considerations that would come to your mind while you are confronted with the decision to have some type of repair done on the automatic
icemaker in your refrigerator, assuming that you noticed it was not making cubes as
well as it did when you first bought it?
• If your icemaker was not working right, how would you correct the problem?
13
Questionnaire Development: Don’ts
•The question should not assume criteria that are not obvious or subject to interpretation
• In a typical month how often do you go to a movie theater to see a movie?
vs.
14
q Never q Occasionally q Sometimes
q 0 q 1 -3 times q 4 times or more
Questionnaire Development: Don’ts
• The question should not be beyond the respondents ability or experience: use filter
questions to screen the respondents
• Typically, how much do you spend on cell phone service each month?
• Are you the person in charge of paying the cell phone bill? • If yes to above question: Typically, how much do you spend on cell phone service each month?
15
Questionnaire Development: Don’ts
• The question should not use one specific example to represent the general case
• How regularly do you use electronic devices like computers for browsing the Internet?
• How regularly you use smartphones, tablet PCs, laptop computers, and desktop PCs for browsing the Internet?
• Which of the following do you typically use to browse the Internet? • Smartphone • Tablet PC • Laptop • Desktop PCs
16
Questionnaire Development: Don’ts
• The question should not ask for details that respondents are not likely to remember;
they will end up guessing • How many total minutes did you spend online last month?
• On average, how many minutes do you spend online per day?
17
Questionnaire Development: Don’ts
• The question should not be “double- barreled”
• Does NikeTown offers better prices and variety than other Nike stores?
• Does Nike Town offers better prices than other Nike stores?
• Does Nike Town offers better variety than other Nike stores?
18
Questionnaire Development: Don’ts
• The question should not lead the respondent to a particular answer, nor
should be loaded
• Don’t you agree that you need to use dental floss everyday?
• Do you use dental floss every day, as recommended by the American Dental Association?
• How often do you use dental floss? • Daily • Weekly • Monthly • Don’t usually use dental floss
19
Questionnaire Flow options
• Funnel: uses a general to specific flow of questions
• Work: easier questions are placed at the beginning and more difficult ones that require more
concentration are placed after
• Sections: organizes questions into sets based on a common objective: topic, response format
• Chorological Order: asks questions in the order in which things typically happen
20
Questionnaire Organization
• Introduction • Screening • Warm-ups • Transitions (throughout, as needed) • Skip Questions • Complicated Questions • Classification and Demographic Questions
21
Survey question types & Scales
22
Survey Response Formats
23
Question
Type
Answer
Options
(scales)
(aka Nominal)
(aka Scale)
Open-Ended Responses
• Asks respondent to write in their answer:
24
Q: What brand of shampoo did you buy last time you purchased a shampoo? A: _____________________
Categorical (aka nominal) Responses
• Dual-Choice Question: has only two response options
25
Q: Do you own a car? A: o Yes o No
Q: Are you … ? A: o Male o Female
Categorical Responses
• Multiple-Choice Questions: provide more than two options
26
Q: What type of car do you own? A:
q Compact q Sedan q Van q SUV q Truck q Other
Metric (aka scale) Responses
• Requires the respondent to think in terms of amounts or levels of the construct being measured
• Two types: • Natural Metric Response Format: corresponds to a number • Synthetic Response Format: utilizes descriptors or “numbers” to indicate levels of a construct
27
Natural Metric Responses
• Respondents provide a number that is appropriate or natural to the property being measured
28
Q: How many times have you purchased gasoline in the past month?
A: 1 time 2 times 3 times 4 times 5 times or more
Synthetic Metric Response Format
• Utilizes artificial descriptors or numbers to describe a value
29
Q: The Guacamole Doritos are:
A: Mild -3 -2 -1 0 1 2 3 HotQ: How important is the price when you
purchase a shampoo? A:
1. ……………. Not at all important 2. ……………. Not very important 3. ……………. Somewhat important 4. ……………. Very important 5. ……………. Extremely important
Consideration in Choosing a Response Format
• Nature of what you are measuring
• Previous research studies
• The level of measurement required for the data analysis: • Categorical data could only be summarized by frequencies and percentages
• Metric data can be summarized by means and standard deviations
30
Scales used in Marketing Research
• Likert Scale: measures agreement-disagreement with a list of statements.
• These statements could measure intentions, attitudes, beliefs, values, emotions, etc.
31
Q: How much do agree with the following statement?
The Guacamole Doritos taste good.
A: Strongly Disagree
Disagree Neither Agree nor Disagree
Agree Strongly Agree
1 2 3 4 5
Scales used in Marketing Research
• Semantic Differential Scale: A series of bipolar adjectives, it is useful in concept tests
32
Q: Indicate your impression of Red Lobster Restaurant by checking the line corresponding to your opinion, for each pair of descriptors.
A: High prices ___ ___ ___ Low prices High quality ___ ___ ___ Low quality Fast service ___ ___ ___ Slow service Limited menu ___ ___ ___ Wide menu
Scales used in Marketing Research
• Stapel Scale: Uses positive and negative numbers to indicate the intensity of the response
33
Q: Please select the number that best represents your opinion for each of the following statements about your fast food experience.
A: Checkout was fast -5 -4 -3 -2 -1 0 1 2 3 4 5
Food tasted good -5 -4 -3 -2 -1 0 1 2 3 4 5
Scales used in Marketing Research
• N-point Scale: meaning a 5-point, 7-point, or 10-point scale format, is a popular choice for researchers measuring constructs on nonsymmetric attributes
34
Q: How would you rate each of the following elements of your most recent vacation?
A: Poor 1 2 3 4
Excellent 5
Overall experience
Accommodations
Food and beverage
Activities offered
Scales used in Marketing Research
• Graphic Scale: uses linear or pictorial representations of intensity
35
The Research objective recipe
36
Setting Good Research Objectives
1. Research tool
2. Research audience
3. Main idea of the
research (the
construct)
4. Types of questions to
be asked
TBD Prof. Zapata
Combine each of the 4 ingredients by
filling in the blanks below.
Conduct a(n) ________ among
________ to learn about ________
as measured by ________.
(1)
(2) (3)
(4)
To be continued…
37
