SO 300 Research Methods Paper - 2 Pages - APA 7e Format
tricia2020
Chambliss6e_PPT_Ch07.pptxSurvey Research
Chapter 7
1
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
Learning Objectives
Explain the strengths and weaknesses of omnibus surveys.
Explain the problem of sampling on the dependent variable.
Discuss the advantages and disadvantages of including “don’t know” and neutral responses among response choices and of using open-ended questions.
2
2
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
Learning Objectives
List the different methods for improving survey questions.
Outline a cover letter for a survey that contains each of the required elements.
List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
Discuss the key ethical issues in survey research.
3
3
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
Why Is Survey Research So Popular?
Survey research
Popularity due to three advantages:
Versatility
Efficiency
Generalizability
Omnibus survey
4
Learning Objective: 7-1: Explain the strengths and weaknesses of omnibus surveys.
Survey research: Research in which information is collected from a sample of individuals through their responses to a set of standardized questions. Omnibus survey: A survey that covers a range of topics of interest to different social scientists.
4
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should We Write Survey Questions?
Be Clear; Avoid Confusing Phrasing
Simple, direct approach
Avoid negative phrases or words
Especially double negatives
Avoid double-barreled questions
Identify what information each question is to obtain
5
Learning Objective: 7-1: Explain the strengths and weaknesses of omnibus surveys.
Learning Objective: 7-4: List the different methods for improving survey questions.
Double negatives: A question or statement that contains two negatives, which can muddy the meaning of the question.
Double-barreled question: A single survey question that actually asks two questions but allows only one answer.
5
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should We Write Survey Questions?
Minimize Bias
Words should not trigger biases (unless this is the intent)
Must test reactions to phrasing of a question.
6
Learning Objective: 7-1: Explain the strengths and weaknesses of omnibus surveys.
Learning Objective: 7-4: List the different methods for improving survey questions.
6
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should We Write Survey Questions?
Allow for Disagreement
Some respondents “agree” with a statement just to avoid disagreeing.
Present both sides of attitude scales in question itself.
Phrase all response choices in socially approved ways.
7
Learning Objective: 7-1: Explain the strengths and weaknesses of omnibus surveys.
Learning Objective: 7-4: List the different methods for improving survey questions.
7
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should We Write Survey Questions?
Don’t Ask Questions They Can’t Answer
Respondents should be competent to answer questions.
Kinds of questions to mitigate this:
Filter questions
Skip pattern
Contingent question
8
Learning Objective: 7-1: Explain the strengths and weaknesses of omnibus surveys.
Learning Objective: 7-4: List the different methods for improving survey questions.
Filter questions: A survey question used to identify a subset of respondents who then are asked other questions.
Skip pattern: The unique combination of questions created in a survey by filter questions and contingent questions.
Contingent question: A question that is asked of only a subset of survey respondents.
8
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should We Write Survey Questions?
Allow for Uncertainty
Different types of respondents:
Floaters
Fence-sitters
Can be managed by . . .
Including “no opinion” category
Including neutral category
Adding open-ended questions
9
Learning Objective: 7-3: Discuss the advantages and disadvantages of including “don’t know” and neutral responses among response choices and of using open-ended questions.
Learning Objective: 7-4: List the different methods for improving survey questions.
Floaters: Survey respondents who provide an opinion on a topic in response to a closed-ended question that does not include a “Don’t know” option but who will choose “Don’t know” if it is available.
Fence-sitters: Survey respondents who see themselves as being neutral on an issue and choose a middle (neutral) response that is offered.
9
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should We Write Survey Questions?
Make Response Categories Exhaustive and Mutually Exclusive
Must be one and only one possible response for everyone who is asked the question.
Two exceptions:
Filter questions
“Check all that apply” questions
10
Learning Objective: 7-3: Discuss the advantages and disadvantages of including “don’t know” and neutral responses among response choices and of using open-ended questions.
Learning Objective: 7-4: List the different methods for improving survey questions.
10
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should Questionnaires Be Designed?
Questionnaire
Interview schedule
11
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
Questionnaire: A survey instrument containing the questions in a self-administered survey.
Interview schedule: A survey instrument containing the questions asked by the interviewer in an in-person or phone survey.
11
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should Questionnaires Be Designed?
Build on Existing Instruments
Use an existing instrument if it measures the concept of concern.
Consider appropriateness for your survey population.
12
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
12
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should Questionnaires Be Designed?
Refine and Test Questions
Methods for testing questions:
Cognitive interview
Conduct a pilot study
Interpretive questions
13
Learning Objective: 7-4: List the different methods for improving survey questions.
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
Cognitive interview: A technique for evaluating questions in which researchers ask people test questions, and then probe with follow-up questions to learn how they understood the question and what their answers mean.
Interpretive questions: Questions included in a questionnaire or interview schedule to help explain answers to other important questions.
13
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should Questionnaires Be Designed?
Maintain Consistent Focus
Have measures of all independent and dependent variables you plan to use.
Long lists of redundant or unimportant questions bore respondents.
Omnibus surveys are an exception.
14
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
14
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should Questionnaires Be Designed?
Order the Questions
Sort questions into thematic categories.
First question should . . .
Interest the respondent
Be easy to answer
Apply to everyone
NOT be sensitive
Context effects
15
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
Context effects: In survey research, refers to the influence that earlier questions may have on how subsequent questions are answered.
15
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should Questionnaires Be Designed?
Make the Questionnaire Attractive
Plenty of white space between questions.
List response choices vertically.
Distinguish response choices clearly.
Indicate skip patterns with graphics.
Use distinctive formatting for instructions.
Booklets are better than multi-pagers.
16
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
16
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
What Are the Alternatives for Administering Surveys?
Surveys can be administered in five ways
Each approach differs in one or more way:
Manner of administration
Questionnaire structure
Setting
Cost
17
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
Learning Objective: 7-5: Outline a cover letter for a survey that contains each of the required elements.
17
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
What Are the Alternatives for Administering Surveys?
Mailed, Self-Administered Surveys
Low response rate hurts representativeness.
Response rates can be improved with a cover letter.
18
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
Learning Objective: 7-5: Outline a cover letter for a survey that contains each of the required elements.
Mailed (self-administered) survey: A survey involving a mailed questionnaire to be completed by the respondent.
Cover letter: The letter sent with a mailed questionnaire that explains the survey’s purpose and auspices and encourages the respondent to participate.
18
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
What Are the Alternatives for Administering Surveys?
19
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
Learning Objective: 7-5: Outline a cover letter for a survey that contains each of the required elements.
19
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
What Are the Alternatives for Administering Surveys?
Group-Administered Surveys
Purpose of survey should be explained before distribution.
Use a cover letter.
Distribute envelopes to ensure confidentiality.
If respondents feel coerced, may not answer honestly.
20
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
Group-administered survey: A survey that is completed by individual respondents who are assembled in a group.
20
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
What Are the Alternatives for Administering Surveys?
Telephone Surveys
Reaching Sampling Units
Most use RDD to contact households.
With cell phones, accurate coverage of random samples is almost impossible.
Use multiple callbacks to maximize response.
Interactive voice response
21
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
Phone survey: A survey in which interviewers question respondents over the phone and record their answers.
Interactive voice response: A survey in which respondents receive automated calls and answer questions by pressing numbers on their touch-tone phones or speaking numbers that are interpreted by computerized voice recognition software.
21
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
What Are the Alternatives for Administering Surveys?
In-Person Interviews
Advantages:
High response rates
Questionnaires can be longer, more complex
Interview can control order of questions
Can monitor setting of the interview
Interpretations of questions can be clarified
22
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
In-person interview: A survey in which an interviewer questions respondents face-to-face and records their answers.
22
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
What Are the Alternatives for Administering Surveys?
In-Person Interviews
Disadvantages:
Must carefully train and supervise interviewers.
Computer-assisted personal interview
Harder for respondents to give honest answers about sensitive topics.
23
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
Computer-assisted personal interview (CAPI): A personal interview in which the laptop computer is used to display interview questions and to process responses that the interviewer types in, as well as to check that these responses fall within allowed ranges.
23
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
What Are the Alternatives for Administering Surveys?
In-Person Interviews
Maximizing Response to Interviews
Contact rates tend to be lower in central cities.
Less education associated with higher rates of “Don’t know” responses.
Simply asking certain questions can depress response rates.
24
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
24
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
What Are the Alternatives for Administering Surveys?
Electronic Surveys
E-mail survey
Easy to develop and use.
Cumbersome for longer surveys.
25
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
Electronic survey: A survey that is sent and answered by computer, either through e-mail or on the web.
E-mail survey: A survey that is sent and answered through e-mail.
25
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
What Are the Alternatives for Administering Surveys?
Electronic Surveys
Web or online survey
Advantages:
Questions can be tailored for respondents
Can elicit more honest reports about socially undesirable behavior.
Disadvantages:
Coverage bias
Low rates of completion
26
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
Web/online survey: A survey that is accessed and responded to on the world wide web.
26
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
A Comparison of Survey Designs
Mailed surveys
Weakest from sampling standpoint
Cheap and useful for sensitive questions
Phone surveys
Sampling and response problems
Must be of limited length and complexity
27
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
27
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
A Comparison of Survey Designs
In-person surveys
Can be long and complex
Require training
Electronic surveys
Advantages depend on population
Can be effective for those with access and ability.
28
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
28
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
Ethical Issues in Survey Research
Survey research poses fewer ethical dilemmas than experimental designs.
Confidentiality is primary focus of ethical concern.
Preserve anonymity
Possible only if no follow-ups needed
29
Learning Objective: 7-7: Discuss the key ethical issues in survey research.
Anonymity: Provided by research in which no identifying information is recorded that could be used to link respondents to their responses.
29
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
Conclusion
Surveys are efficient and productive
Most popular research method in sociology
Due to widespread use, some survey results are worthless
Careful planning and construction are necessary
30
