steps
steoan221a
rm2.5_research_design.pptxChapter 5: Research Design
Overview:
Research Design
Basic Concepts for Research Design
Issues of Validity and Inference
Research Design Types
Ethics and Research Design
Chapter 5 Learning Objectives
Understand core concepts related to the design of an evaluative project
Understand the different types of research design
Experimental
Quasi-experimental
Identify key issues of validity
Across a range of research design types
Identify ethical issues with the research design choices
Research Design
Basic challenge:
Develop an understanding of exactly how to approach the data collection process when attempting to evaluate or assess some program or policy
Research design:
Refers to the general process by which data gathering efforts are structured and defined; that is, what is to be studied and how, what variables are to be included in the study, how they are measured in relation to one another, and how those data are gathered
Basic Concepts for Research Design
Experiment:
An activity where a researcher controls or manipulates the conditions under which some sort of subject is examined in order to observe and measure a specific cause-and-effect relationship
Treatment:
A variable or condition that the researcher introduces into the experiment in order to see whether it has an effect on the subjects
Control group:
Those subjects in the experiment that do not receive the treatment
Issues of Validity and Inference
Internal validity:
Did you actually measure (accurately and validly) the concepts and variables you intended to?
Did you account for, or rule out, competing explanations?
Understanding the nature of internal validity, and threats to it, are critical to drawing causal inferences
Threats to Internal Validity
Threat of history
Threat of maturation
Threat of testing
Threat of instrumentation
Threat of regression to the mean
Threat of selection bias
Threat of mortality
Threats to External Validity
External validity:
Can the results can be generalized to other settings beyond the specific sample gathered for the purposes of conducting a given study?
Threats to external validity:
Involve interaction of the nature of the study (and its treatment) with specific aspects of conducting an experiment (i.e., an interaction between treatment and testing)
Research Design Types
True experiments
Quasi-experiments
Nonexperiments
Ethics and Research Design
Issues of potential bias and conflicts of interest in the conduct of a research or evaluation study is of paramount concern
Issues of subject selection and subject inclusion
American Evaluation Association and the guiding principles for the conduct of evaluation research
