discussion board questions/ experimental design activity

missruf

VSUResearchchap5Ch_5.ppt

Home >Law homework help >Criminal homework help >discussion board questions/ experimental design activity

Experimental and Quasi-Experimental Designs

Chapter 5

Introduction

Experiments is a mode of scientific observation

best suited for explanation and evaluation research

Experiments involve:

Taking action – select a group of subjects & do something to them

Observing the consequences of that action

Especially suited for hypothesis testing see

CJ experiments are usually conducted in field settings outside laboratories

The Classical Experiment

Classical experiment: is a specific way of structuring research
Involves three major components:

Independent variable and dependent variable

Pretesting and posttesting

Experimental group and control group

Independent and Dependent Variables

The independent variable takes the form of an experimental stimulus that is either present or absent

IV is the cause

The dependent variable is the outcome, the effect we expect to see

Eg. how often subjects used alcohol is the dependent variable

Exposure to a video about alcohol’s negative effects is the independent variable

Thus we might say that watching the video causes a change in alcohol use

Pretesting and Posttesting

Subjects are initially measured in terms of the DV prior to association with the IV (pretested)

Then, they are exposed to the IV

Then, they are remeasured in terms of the DV (posttested)

Differences noted between the measurements on the DV are attributed to influence of IV

Problem of validity: if subjects change their answers during post testing due to becoming aware of the exp. Not as a result of actual effects of the video

Experimental and Control Groups see diagram in pg 113 for basic experimental design

Experimental group: exposed to whatever treatment, policy, initiative we are testing

Control group: very similar to experimental group, except that they are NOT exposed to treatment

Can involve more than one experimental or control group

If we see a difference, we want to make sure it is due to the IV, and not to a difference between the two groups see note on diagram

Placebo

We often don’t want people to know if they are receiving treatment or not

We expose our control group to a “dummy” independent variable just so we are treating everyone the same

Medical research: participants don’t know what they are taking

Ensures that changes in DV actually result from IV and are not psychologically based

Double-Blind Experiment

Sometimes experimenters have the tendency to prejudge results

In med. research, Experimenters may be more likely to “observe” improvements among those who received drug

Double-blind experiment eliminates this possibility because…

In a double-blind experiment, neither the subjects nor the experimenters know which is the experimental group and which is the control group

Another researcher knows which subjects are in which group

Selecting Subjects

First, must decide on target population – the group to which the results of your experiment will apply

Second, must decide how particular members of the target population will be selected for the experiment

The methods used to select subjects must meet the scientific norm of generalizability

Cardinal rule – ensure that experimental and control groups are as similar as possible

Randomization

Randomization is a central feature of the classical experiment
Randomization: produces an experimental and control group that are statistically equivalent
You can randomize by:

assigning the odd numbered subjects to the experimental group and the even-numbered subjects to the control, group

Eliminates systematic bias in assigning subjects to group

Experiments and Causal Inference

Experimental design ensures:

Cause precedes effect by taking posttest measurement

Empirical correlation exists via comparing pretest to posttest

A change in pretest to posttest measures demonstrates correlation

The observed correlation between cause and effect is not due to the influence of a third variable

Note: if the correlation b/w the IV and the DV is due to some other factor, then the two post test will be similar

Example of Research Using an Experimental Design

Researchers at the University of Maryland conducted an evaluation of the Baltimore Drug Court using an experimental design. For their research, eligible offenders were randomly assigned to either the drug court or to ‘”treatment as usual”. The results of the randomization process were given to the judges as a recommendation. In most cases, the judges, who agreed to participate in the study beforehand, sentenced offenders in accordance with the randomization. The results showed that participants in the drug court were less likely to recidivate than those in the control group.

For more information see Gottfredson, D.C., Najaka, S.S. & Kearley, B. (2003). Effectiveness of drug treatment courts: Evidence from a randomized trial. Criminology, 2(2), 171-196.

Internal Validity Threats

Internal Validity threats: refers to the possibility that conclusions drawn from experimental results may not reflect what went on in experiment

History: external events may occur during the course of the experiment

Maturation: people constantly are growing eg in long term experiment, the fact that subjects grow older may have effect

Testing: the process of testing and retesting influence’s peoples behavior

Instrumentation: Changes in the measurement process eg using different questionnaire about alcohol use (DV) during the pretest and posttest

Change in procedure/measurement process

Internal Validity Threats: Continued

Selection bias: the way in which subjects are chosen may threaten validity

The effect of using Volunteers for experiments may not represent the population

Experimental mortality: subjects may drop out prior to completion of experiment & that can affect stat comparison

Eg. suppose the heavy drinkers in the experiment group are so turned off by the video & decides to leave

While the less heavy drinkers stayed for the posttest, the result will reflect substantial decrease in alcohol use

Generalizability and Threats to Validity

Researchers also face problems with generalizing results from experiments

Generalizability:

Even if results of experiments are an accurate measure of what happened during that experiment but ….

do the results of an experiment really tell us what would happen in the real world?

Two Dimensions of Generalizability: Construct & External Validity

Construct validity: is the correspondence between the empirical test of a hypothesis and the underlying causal process that the experiment is intended to represent

In other words: Do measures correspond with the fundamental concept/construct/idea we wish to measure?

Validity Issue: Eg. is the educational video a reasonable way to cause people to reduce excessive drinking?

The educational video is reasonable but incomplete cos there are other ways

Such as personal experience, talking to people, taking courses, reading books

Other validity issues: video may be poorly produced or too technical d/by making it inadequate to represent the construct we are interested in – ie. educating abt the health effects of alcohol

Showing a single video may not be adequate

External Validity Threats

External validity: whether the results from experiments in one setting will be obtained in other settings/population

Eg. can a probation program shown to be successful in Houston achieve similar result in Richmond

Threats for external validity are greater for experiments conducted under carefully controlled conditions like the video rather than more natural conditions

Field experiments takes place under real world conditions, results are more likely to be valid in other real world settings

Threats to internal validity are reduced by conducting experiments under controlled conditions.

Field experiments (FE) have greater external validity than internal validity cos FE are more difficult to control

Variations in the Classical Experimental Design see pg. 121

Classical Experiment:

Both pretest and posttest measure are used

Measures are taken for the control group at t1 & t3

Experimental stimulus is not given to the control group

Post-test Only design

No pretest measure is used

Used when pretest might bias results but can’t measure change

Factorial Design

Two experimental groups that received different treatment

Useful for comparing the effects of different/amt treatments. See design

In-class Activity

You are about to conduct an experiment:

1.State your hypothesis

2. State and Design the type of experiment you want to conduct

Quasi-Experimental Designs

When randomization is not possible

quasi = “to a certain degree”

Quasi-Experiment: an experiment to a certain degree

Do not randomly assign subjects, may suffer from the internal validity threats such as selection bias

Two categories of Quasi-Experiment:

Non-equivalent-groups designs and

Time series designs

Nonequivalent-Groups Designs see pg. 123

When we cannot randomize, we cannot assume equivalency; hence the name

We take steps to make groups as comparable as possible

We can Match subjects in the Experimental group with subjects in the Comparison groups

Comparison group is used rather than the control grp to highlight the nonequivalence of groups

It involves Aggregate matching – comparable average characteristics or variables like age, gender, SES, race

Cohort Designs

Cohort study can be viewed as a type of nonequivalent-groups design

Cohort – group of subjects who enter or leave an institution at the same time

EX. if you are interested in whether probationers who receive community service sentences are charged with fewer probation violations

You can compare persons sentenced to probation in May with such sentence (exp. grp) and those sentenced to probation in June without such sentence (compa. grp)

Necessary to ensure that two cohorts being examined against one another are actually comparable

Time-Series Designs

It examines a series of observations over time

Eg. examining the trends in arrests for drunk driving over time to see whether the number of arrest is increasing or decreasing or constant

Interrupted Time-Series Design – series of observations are compared before and after some intervention ex. Observations b/4 govt intervention to reduce terrorism and observations after the intervention

Instrumentation threat to internal validity is likely because changes in measurements may occur over a long period of time

Case & Variable-Oriented Research – another way of time series design

Case-oriented research: many cases are examined to understand a small number of variables
EX. studying in-mates to in-mates assaults in correctional facility

We might send a questionnaire to 500 correctional facilities to get info about assaults, facility design, inmate xteristics & housing conditions

Here we are gathering information on a few variables from a large number of cases

Variable-oriented research

Variable-oriented research: a large number of variables are studied for a small number of cases

Eg. we might visit one or a few facilities to conduct in-depth interviews with staffs, observe the condition of facilities and gather info from institutional records

Here we are collecting info on a wide range of variables from a small number of institutions

Eg. Case studies: researcher centers on an in-depth examination of one or few cases on many dimensions