CONCEPTUAL DRAFT OF CHAPTER 2 (LITERATURE REVIEW) INSTRUCTIONS

Chapter7740.pdf

Home >Law homework help >Criminal homework help >CONCEPTUAL DRAFT OF CHAPTER 2 (LITERATURE REVIEW) INSTRUCTIONS

Chapter 7:

Experimental and Quasi- Experimental Designs

Learning Objectives

• Recognize that experiments are well suited for the controlled testing of causal processes and for some evaluation studies

• Describe how the classical experiment tests the effect of an experimental stimulus on some dependent variable through the pretesting and posttesting of experimental and control groups

• Understand that a group of experimental subjects need not be representative of some larger population but that experimental and control groups must be similar to each other

• Describe how random assignment is the best way to achieve comparability in the experimental and control groups

• Describe how the classical experiment with random assignment of subjects guards against most of the threats to internal invalidity

Learning Objectives, cont. • Understand that the controlled conditions under which experiments take

place may restrict our ability to generalize results to real-world constructs or to other settings

• Recognize how the classical experiment may be modified by changing the number of experimental and control groups, the number and types of experimental stimuli, and the number of pretest or posttest measurements

• Know the reasons that quasi-experiments are conducted when it is not possible or desirable to use an experimental design, and be able to describe different categories of quasi-experiments

• Understand the differences between case-oriented and variable-oriented research

• Be able to describe how experiments and quasi-experiments can be customized by using design building blocks to suit particular research purposes

Introduction

• Experimentation is an approach to research best suited for explanation and evaluation

• An experiment is “a process of observation, to be carried out in a situation expressly brought about for that purpose”

• Experiments involve: – Taking action – Observing the consequences of that action

• Especially suited for hypothesis testing

The Classical Experiment

• Variables, time order, measures, and groups are the central features of the classical experiment

• Involves three major pairs of components: – Independent and dependent variables – Pretesting and posttesting – Experimental and control groups

Independent Variables

• The Independent Variable takes the form of a dichotomous stimulus that is either present or absent

• It varies (i.e., is independent) in our experimental process

• “The Cause”

Dependent Variables

• The outcome, the effect we expect to see

• Depends on the Independent Variable

• Might be physical conditions, social behavior, attitudes, feelings, or beliefs

• “The Effect”

Pretesting and Posttesting

• Subjects are initially measured in terms of the Dependent Variable prior to association with the Independent Variable (pretested)

• Then, they are exposed to the Independent Variable

• Then, they are remeasured in terms of the Dependent Variable (posttested)

• Differences noted between the measurements on the Dependent Variable are attributed to influence of the Independent Variable

Discussion Question 1

What if you took part in a social science experiment? What assurances would you expect from the administrators of the experiment, if any?

Experimental and Control Groups

• Experimental group: Exposed to whatever treatment, policy, or initiative we are testing

• Control group: Very similar to experimental group, except that they are NOT exposed

• If we see a difference, we want to make sure it is due to the Independent Variable, and not due to a difference between the two groups

Hawthorne Effect

• Pointed to the necessity of control groups • Independent Variable: improved working

conditions (better lighting) • Dependent Variable: improvement in employee

satisfaction and productivity • Workers were responding more to the attention

than to the improved working conditions

Placebo

• We often don’t want people to know if they are receiving treatment or not

• We expose our control group to a “dummy” Independent Variable just so we are treating everyone the same

• Medical research: Participants don’t know what they are taking

• Ensures that changes in Dependent Variable actually result from Independent Variable and are not psychologically based

Double-Blind Experiment

• Experimenters may be more likely to “observe” improvements among those who received drug

• In a Double-Blind experiment, neither the subjects nor the experimenters know which is the experimental group and which is the control group – Broward County Florida and Portland, Oregon domestic

violence policing units study: “keeping safe” strategies

Discussion Question 2

Would you ever participate in a double- blind experiment? Why or why not?

Selecting Subjects

• First, must decide on target population, the group to which the results of your experiment will apply

• Second, must decide how to select particular members from that group for your experiment

• Cardinal rule: ensure that Experimental and Control groups are as similar as possible

• Randomization aims to achieve this

Random Assignment • “Randomization” • Central feature of the classical experiment

– Produces experimental and control groups that are statistically equivalent

• Farrington and associates: – “Randomization insures that the average unit in the

treatment group is approximately equivalent to the average unit in another group before the treatment is applied”

• “All Other Things Are Equal”

Discussion Question 3

How difficult is it to randomize an experiment? Is it costly? Can any researcher do it?

Experiments and Causal Inference

• Experiments potentially control for many threats to the validity of causal inference

• Experimental design ensures: – Cause precedes effect via taking posttest – Empirical correlation exists via comparing pretest to

posttest – No spurious third variable influencing correlation via

posttest comparison between experimental and control groups, and via randomization

Threats to Internal Validity

• Conclusions drawn from experimental results may not reflect what went on in experiment

• History: External events may occur during the course of the experiment

• Maturation: People constantly are growing • Testing: The process of testing and retesting

Threats to Internal Validity, cont.

• Instrumentation: Changes in the measurement process

• Statistical regression: Extreme scores regress to the mean

• Selection biases: The way in which subjects are chosen (use random assignment)

• Experimental mortality: Subjects may drop out prior to completion of experiment

Threats to Internal Validity, slide 3

• Causal time order: Ambiguity about order of stimulus and Dependent Variable— which caused which?

• Diffusion/Imitation of treatments: Experimental group may pass on elements to Control group when communicating

• Compensatory treatment: Control group is deprived of something considered to be of value

Threats to Internal Validity, slide 4

• Compensatory Rivalry: Control group deprived of the stimulus may try to compensate by working harder

• Demoralization: Feelings of deprivation among control group result in subjects giving up

Generalizability and Threats to Validity

• Potential threats to internal validity are only some of the complications faced by experimenters; they also have the problem of generalizing from experimental findings to the real world

• Two dimensions of generalizability: – Construct Validity – External Validity

Threats to Construct Validity

• Concerned with generalizing from experiment to actual causal processes in the real world

• Link construct and measures to theory • Clearly indicate what constructs are

represented by what measures • Decide how much treatment is required

to produce change in Dependent Variable

Threats to External Validity

• Significant for experiments conducted under carefully controlled conditions rather than more natural conditions

• Reduces internal validity threats • John Eck (2002): "diabolical dilemma." • Suggestion:

– explanatory studies -> internal validity – applied studies -> external validity

Threats to Statistical Conclusion Validity • Becomes an issue when findings are based

on small samples • More cases allows you to reliably detect small

differences; less cases result in detection of only large differences

• Finding cause-and-effect relationships through experiments depends on two related factors: – Number of Subjects – Magnitude of posttest differences between the experimental

and control groups

Variations in the Classical Experimental Design

• Four basic building blocks present in experimental designs: – The number of experimental and control groups – The number and variation of experimental stimuli – The number of pretest and posttest measurements – The procedures used to select subjects and assign them

to groups

• Variations on the classical experiment can be produced by manipulating the building blocks of experiments

Quasi-Experimental Designs

• When randomization isn’t possible for legal or ethical reasons

• Renders them subject to Internal Validity threats

• Quasi = “to a certain degree” • Two categories:

– nonequivalent-groups designs – time-series designs

Nonequivalent-Groups Designs

• When we cannot randomize, we cannot assume equivalency; hence the name

• We take steps to make groups as comparable as possible

• Match subjects in Experimental and Control groups using important variables likely related to Dependent Variable under study

• Aggregate matching: comparable average characteristics

Cohort Designs

• Cohort: Group of subjects who enter or leave an institution at the same time – Ex: A class of police officers who graduate from a

training academy at the same time; all persons who were sentenced to probation in May

• Necessary to ensure that two cohorts being examined against one another are actually comparable

Time-Series Designs

• Longitudinal Studies – Examine a series of observations over time

• Interrupted: Observations compared before and after some intervention (used in cause-and-effect studies)

• Instrumentation threat to internal validity is likely because changes in measurements may occur over a long period of time – Often use measures produced by CJ organizations

Scientific Realism • A large number of variables are studied for a

small number of cases or subjects • Case-oriented research: Many cases are

examined to understand a small number of variables (e.g., Boston Gun Project)

• Variable-oriented research: A large number of variables are studied for a small number of cases or subjects – Case Study Design: Centered on an in-depth

examination of one or a few cases on many dimensions