Thesis Proposal
Chapter 8
Quantitative Methods
*
Introduction
- Designing quantitative methods for a research proposal
- Survey and experimental designs
- Careful measurement, parsimonious variables, theory-guided
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
*
Defining Surveys and Experiments
- Survey design
Quantitative description of trends, attitudes, or opinions of a population
Testing association
Studying a sample of that population
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Defining Surveys and Experiments
- Experimental design
Systematic manipulation of one or more variables to evaluate an outcome
Holds other variables constant to isolate effects
Generalize to a broader population
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
- The survey design
- The population and sample
- Instrumentation
- Variables in the study
- Data analysis and interpretation
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Components of a Survey Method Plan
Creswell, Research Design 5e SAGE Publishing, 2018
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Components of a Survey Method Plan
Creswell, Research Design 5e SAGE Publishing, 2018
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Components of a Survey Method Plan
Creswell, Research Design 5e SAGE Publishing, 2018
Components of a Survey Method Plan
The survey design:
- Provide a purpose for using survey research
- Indicate why the survey method is preferred
- Indicate the type of survey design
Cross-sectional (data collected at one point in time)
Longitudinal (data collected over time)
- Specify the form of data collection – telephone, mail, Internet personal/group interviews) and rationale
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of a Survey Method Plan
The population and sample:
- Identify the population including size and sampling frames
- Specify the sampling design
Single-stage
Multi-stage (clustering)
- Type of sampling
Probability
Nonprobability
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of a Survey Method Plan
The population and sample:
- Indicate if the study involves stratification – ensuring specific population characteristics (e.g. gender) are represented
- Indicate number in the sample and procedure to determine
- Use a power analysis if you plan to detect significant associations
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of a Survey Method Plan
The population and sample:
- Power analysis involves
Estimating the size of correlation
An alpha value (type I error rate)
A beta value (type II error rate)
Conducting the power analysis (e.g., G*Power)
- Conduct power analysis during planning
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of a Survey Method Plan
Instrumentation:
- Name the survey instrument used to collect data
- Indicate how instrument was developed
- Describe the established validity scores from past use
Content validity
Predictive or concurrent validity
Construct validity
- Describe reliability of scores from past use
Internal consistency
Test-retest
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of a Survey Method Plan
Instrumentation:
- When modifying or combining instruments, the original validity and reliability may not hold
- Include sample items from the instrument
- Indicate major content sections in the instrument
Cover letter
Items – demographics, attitude items, behavior items, factual items
Closing instructions
Type of scale for responses
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of a Survey Method Plan
Instrumentation:
- Discuss pilot testing or field-testing
Rationale for plans
Content validity and reliability
Improve question
- Steps for administering for a mailed survey
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of a Survey Method Plan
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of a Survey Method Plan
Data analysis:
- Computer programs used for analysis
- Data analysis steps
Step 1. Number who did and did not respond
Step 2. Method to determine response bias
Step 3. Plan to provide descriptive analyses
Step 4. Calculate total scale scores
Step 5. Statistics and program for inferential statistical analyses
Step 6. Present results in figures or tables and interpret
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Interpreting results and writing a discussion section:
- Report how the results answered the research question or hypothesis
- Practical evidence in terms of effect size and confidence interval
- Discuss implications
- Consistent with, refute, extent previous studies
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Components of a Survey Method Plan
Creswell, Research Design 5e SAGE Publishing, 2018
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Components of a Survey Method Plan
Creswell, Research Design 5e SAGE Publishing, 2018
Components of an Experimental Study Method Plan
- Experimental method plan
Participants
Variables
Instrumentation and materials
Experimental procedures
Threats to validity
Data analysis
Interpreting results and writing a discussion
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of an Experimental Study Method Plan
Participants:
- Describe procedures for recruiting participants
- Describe the selection of participants as either
Random
Nonrandom (convenience)
- True experiment – individuals randomly assigned to groups
- Quasi-experiment – partial or no control over random assignment
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of an Experimental Study Method Plan
Participants:
- May measure second predictor variables
- Conduct and report power analysis
- End with formal experimental design statement
“The experiment consisted of a one-way two-groups design comparing burnout symptoms between full-time and part-time nurses”
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Components of an Experimental Study Method Plan
Creswell, Research Design 5e SAGE Publishing, 2018
Components of an Experimental Study Method Plan
Variables:
- Specify the variables and describe in detail
Identify the independent variables
Include a manipulation check measure
Identify dependent variable
Identify other variables measured
- Participant demographics
- Measure variables that contribute noise
- Potential confounding variables
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of an Experimental Study Method Plan
Instrumentation and materials:
- Describe the instrument(s) participants complete in the experiment
Development, items, and scales
Reliability and validity reports of past uses
- Thoroughly discuss materials used for the treatment
- Cover story to explain procedures if deception is used
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of an Experimental Study Method Plan
Experimental procedures:
- Identify the type of experiment
Pre-experimental, true experiment, quasi-experiment, single subject design
- Identify the type of comparisons – within-group or between-subject
Provide a visual model to illustrate the research design used
- X = treatment
- O = observation
- R = random assignment
See Examples 8.2–8.5
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of an Experimental Study Method Plan
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of an Experimental Study Method Plan
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of an Experimental Study Method Plan
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of an Experimental Study Method Plan
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of an Experimental Study Method Plan
Threats to validity:
- Internal validity – procedures, treatments, or experiences of the participants that threaten inferences in experiments
- External validity – drawing incorrect inferences from sample data to other persons, settings, situations
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of an Experimental Study Method Plan
Threats to validity:
- Statistical conclusion validity – inadequate statistical power or violation of statistical assumptions
- Construct validity – inadequate definitions and measures of variables
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of an Experimental Study Method Plan
History
Maturation
Regression
Selection
Mortality
Diffusion of treatment
Compensatory/resentful demoralization
Compensatory rivalry
Testing
Instrumentation
Creswell, Research Design 5e
SAGE Publishing, 2018
*
- Threats to internal validity
Creswell, Research Design 5e SAGE Publishing, 2018
Components of an Experimental Study Method Plan
- Threats to external validity
Interaction of selection and treatment
Interaction of setting and treatment
Interaction of history and treatment
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of an Experimental Study Method Plan
The procedure:
- Administer measures of the dependent variable or a variable closely correlated
- Assign participants to matched pairs
- Randomly assign one member of each pair to the control and experimental group
- Expose experimental group to the treatment
- Administer measures of dependent variables
- Compare performance of the experimental and control groups
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of an Experimental Study Method Plan
Data analysis:
- Report descriptive statistics (e.g., means, standard deviations, ranges)
- Indicate inferential statistical tests (e.g., t test, ANOVA, ANCOVA, or MANOVA)
- Report confidence intervals and effect sizes in addition to statistical tests
- Use line graphs for single subject designs
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Components of an Experimental Study Method Plan
Interpreting results and writing a discussion section:
- Interpret findings in light of hypotheses and research questions
- Whether supported or refuted
- Why results significant or not, literature
- Indicate implications
- Suggest future research
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018
Summary
- The methodological approach to a survey or experiment
- Surveys – purpose, population and sample, instruments, relationship, research questions, items, analysis
- Experiments – identify participants, variables, instruments, type of experiment, validly, analysis
*
Creswell, Research Design 5e
SAGE Publishing, 2018
Creswell, Research Design 5e SAGE Publishing, 2018