Exam
Monet013
Chapter5LT1.pptxResearch Methods in Psychology
Survey Research
1
Survey Research
Survey research
Describe thoughts, opinions, feelings
Allows predictions based on correlations
Questionnaires
Predetermined set of questions
Sample represents a population
Examine survey procedures and analyses for sources of bias
2
Sampling in Survey Research
Use sample to represent the larger population
“Representative”: similar to
Requires careful selection of a sample
Goal: Generalize survey findings from representative sample to the population
3
Basic Terms of Sampling (p. 141)
Population
Set of all cases of interest (e.g., all students on a college campus)
Sampling Frame
List of the members of a population (e.g., registrar’s list of enrolled students)
Sample
Subset of population drawn from sampling frame
Element
Each member of the population
4
Biased Samples
A biased sample
Characteristics of the sample differ systematically from those of the population
Sample over-represents or under-represents segment(s) of a population
Population is 50% urban, 30% suburban, and 20% rural
In a sample of 200 people, which one of the following would be representative? (p. 142)
50 rural, 70 suburban, and 80 urban
20 rural, 85 suburban, and 105 urban
40 rural, 60 suburban, and 100 urban
5
Biased Samples
Two sources
Selection bias
Researcher’s procedures for selecting sample cause bias
Response-rate bias
Individuals selected for the sample do not complete the survey
6
Biased or Unbiased Sample Selection?
A graduate student interested in sports psychology investigates fan reactions to a proposed change in the name of a local sports team to make it more culturally sensitive. As fans exit a game, he selects every 10th person to ask about the proposed name change. From his findings, he concludes: “Local sports fans do not want to see a name change.”
Biased or Unbiased Sample Selection?
A psychology professor offers extra credit for students in her class who attend a campus movie on the topic of adolescent depression and suicide. At the end of the movie a graduate student assistant asks if they liked this way of earning extra credit. Nine out of 10 students raise their hands, and the graduate student reports to the professor: “Almost everyone in your class likes this way of earning extra credit.”
Approaches to Sampling
“Sampling”
Procedures used to obtain a sample
Two basic approaches
Probability sampling
Nonprobability sampling
9
Probability sampling
All members (elements) of population have an equal chance of being selected for the survey
Simple random sample (p. 146)
Random selection, random-digit dialing
Stratified random sample
Divide population into strata and sample proportionally (e.g., freshman, sophomore, etc.)
Improves representativeness of sample
Approaches to Sampling, continued
10
Nonprobability sampling
No guarantee each member of population has an equal chance of being in the sample
“Convenience sampling”
Individuals are available and willing to respond to the survey
Example: magazine surveys, call-in radio/TV surveys (see page 145), person on the street
Sample likely not representative of population
Approaches to Sampling, continued
11
Probability vs. Nonprobability Sampling
If we want to know the views of UG students on the college’s recycling efforts
Go to cafeteria and sample 50 students (Probability or nonprobability? Why?)
Randomly select students from the registrar’s list (Probability or nonprobability? Why?)
Survey Methods
Four methods for obtaining survey data
Mail surveys
Pros--easy to administer and complete, no interviewer bias, good for personal topics
Cons—no control over order of completing, respondents can’t ask questions, low response rate
Personal interviews
Pros--control over questioning, asking for clarification, follow-up on ambiguous open-ended answers, higher response rate
Cons--cost and interviewer bias (e.g., how follow-up questions are asked)
13
Survey Methods (continued)
Telephone interviews
Pros—same pros of personal interviews, reach more people (better access), interviewers can be better supervised (if from one site)
Cons—selection bias (not everyone has a phone, some have multiple phones), interviewer bias, differential responding (faceless voice), low response rate (voicemail, caller ID)
Internet surveys
Pros--efficiency/cost, ability to survey cross culturally
Cons--response rate bias (lower than for mail and telephone), selection bias (have computer access), lack of control (Understand instructions? Answering conscientiously? Multiple submissions?)
Survey Research Designs
“Research design”
A plan for conducting a research project
Choose method best suited for answering a particular question
Three types of survey research designs
Cross-sectional design
Successive independent samples design
Longitudinal design
15
Cross-sectional survey design
Select sample from one or more populations at one time
Survey responses are used to
Describe population (descriptive statistics)
Compare responses across two or more populations
Make predictions for the population (correlations)
at that one point in time
Cannot assess change over time
Survey Research Designs, continued
16
Survey Research Designs, continued
Successive independent samples design (presidential approval; college student values survey)
A series of cross-sectional surveys over time
A different sample from the population completes the survey each time.
Each sample is selected from the same population.
Responses from each sample are used to describe changes in the population over time.
Problem: noncomparable successive samples
17
Survey Research Designs, continued
Longitudinal survey design (Ex: teachers)
Same sample of individuals completes the survey at different points in time
Assess how individuals change over time
Responses from the sample are generalized to describe changes over time in the population.
Problems: attrition and reactivity (try to look consistent; trying to hide problems)
18
Measures in Survey Research
Questionnaires
Most frequently used to collect survey data
Measure different types of variables
Demographic variables using checklists (e.g., ethnicity, age, SES)
Preferences, opinions, and attitudes
Self-report rating scales (assume interval level of measurement)
All measures must be reliable and valid.
19
Reliability and Validity
Reliability refers to consistency of measurement.
Test-retest reliability
Administer measure two times to same sample
High correlation between the two sets of scores indicates good reliability (r > .80)
20
Reliability and Validity, continued
How to improve reliability?
More items
Greater variability among individuals on the factor being measured (soccer example)
Testing situation free of distractions
Clear instructions
Unambiguously worded items
A measure can be reliable but not valid.
21
Reliability and Validity, continued
Validity refers to the truthfulness of a measure. (class test item not addressing material covered in class—not valid?)
Assesses what it is intended to measure
Construct validity
Instrument measures the theoretical construct it was designed to measure.
22
Reliability and Validity, continued
Establishing construct validity:
Convergent validity
Extent to which two measures of the same construct are correlated (go together)
Discriminant validity
Extent to which two measures of different constructs are not correlated (do not go together)
\
23
Constructing a Questionnaire
Best choice for selecting a measure
Use measure already shown to be reliable and valid in previous research.
If no suitable measure is found, create a questionnaire or measure.
Creating a reliable and valid questionnaire is not easy.
24
Constructing a Questionnaire, continued
Important first steps
Decide what information should be sought.
Decide how to administer the questionnaire.
Write a first draft of the questionnaire.
Reexamine and revise questionnaire based upon expert advice (survey and content experts).
Pretest the questionnaire.
Review results and edit the questionnaire.
25
Constructing a Questionnaire, continued
Next steps: Establish reliability and validity
Reliability
Test and re-test questionnaire using sample and conditions similar to planned survey.
Validity
Convergent: Administer questionnaire with measures of theoretically related constructs
Discriminant: Administer questionnaire with measures of theoretically unrelated constructs
26
Constructing a Questionnaire, continued
Guidelines for Writing Survey Questions
Choose how participants will respond
Free-response (more expressive, difficult to score) or closed questions (easy to score, responses don’t capture one’s view)
Use simple, familiar vocabulary
Write clear, specific questions
Avoid double-barreled questions
Place conditional phrases at the beginning of sentences (If you were forced to leave your job…)
Avoid leading questions and loaded questions
Avoid response bias (positive and negative wording)
27
Constructing a Questionnaire, continued
Ordering of questions
Self-administered questionnaires
Place most interesting questions first
Personal and telephone interviews
Demographic questions first
Use funnel questions and filter questions as needed.
28
Correspondence Between Reported and Actual Behavior
Survey responses may not be truthful.
Reactivity
Social desirability
Accept people’s responses as truthful unless there’s reason to suspect otherwise.
Use a multimethod approach to answering research questions (check records/archives to see if survey results match behavior).
Thinking Critically About Survey Research
29
Thinking Critically About Survey Research, continued
Correlation and causality
“Correlation does not imply causality”
Three possible causal inferences for any correlation
A causes B
B causes A
Variable C causes both A and B
30
