Research Method

11/15/2020

1

CHAPTER 10

Introduction to Simple Experiments

Chapter Overview

▪ Example of simple experiments

▪ Experimental variables

▪ Why experiments support causal claims

▪ Experimental design ▪ Independent-groups designs

▪ Within-groups designs

▪ Interrogating causal claims with the four validities

▪ Independent-groups design (aka between-subjects design or between-groups design) → different groups of participants placed at different levels (conditions) of IV

▪ Within-groups design (aka within-subjects design) → only one group of participants and each participant in presented with all levels (conditions) of the IV

Independent-Groups vs. Within-Groups Design

11/15/2020

2

Independent Groups: Posttest-Only Design

▪ Posttest-Only Design (aka equivalent groups)→ participants are

randomly assigned to IV groups and are tested on the DV just once

Independent Groups: Pretest/Posttest Design

▪ Pretest/Posttest Design → participants are randomly assigned to different

IV groups and are tested on the DV twice—once before and once after

exposure to the IV

Independent Groups: Which Design Is Better?

▪ The situation determines which design is better.

▪ In some situations, it is problematic to use a pretest/posttest design.

▪ In other situations, a pretest/posttest design makes sense.

▪ Posttest-only designs can still be very powerful.

11/15/2020

3

CHAPTER 10

Introduction to Simple Experiments

▪ Independent-groups design (aka between-subjects design or between-groups design) → different groups of participants placed at different levels (conditions) of IV

▪ Within-groups design (aka within-subjects design) → only one group of participants and each participant in presented with all levels (conditions) of the IV

Independent-Groups vs. Within-Groups Design

Within-Groups Designs

▪ Repeated Measures Design → participants are measured on the DV more

than once (after exposure to each level of the IV)

11/15/2020

4

▪ Concurrent Measures Design → participants are exposed to all levels of

the IV at roughly the same time, and a single preference is the DV

Within-Groups Designs

Advantages of Within-Groups Designs

1. Participants in your groups are equivalent because they are the same participants and serve as their own controls.

2. These designs give researchers more power to notice differences between conditions.

▪ Power = ability of a study to show a statistically significant result

3. Within-groups designs require fewer participants than other designs.

▪ Covariance: there is a manipulated IV and comparison conditions

▪ Temporal precedence: the IV comes before the DV

▪ Internal validity: are there third variable explanations?

▪ Order effects (i.e., practice effects and carryover effects)

Do within-groups designs fulfill the three causal criteria?

11/15/2020

5

Internal Validity: Controlling for Order Effects

▪ Order effect → when being exposed to one condition affects how participants respond to other conditions

▪ Practice effects → when participants either get better at a task from practice

▪ Fatigue effects → when participants get worse at a task due to fatigue

▪ Carryover effects → when there is contamination carrying over from one condition to the next

How do these

hurt internal

validity?

Avoiding Order Effects by Counterbalancing

▪ Counterbalancing →present levels of the IV to participants in different orders

▪ Two types of counterbalancing

1. Full counterbalancing → when all possible condition orders are presented

2. Partial counterbalancing (example: Latin square) → when only some of the possible condition orders are used

Disadvantages of Within-Groups Designs

1. Potential for order effects ◦ Solution: counterbalancing

2. Might not be practical or possible

3. Experiencing all levels of the IV changes the way participants act (demand characteristics)

11/15/2020

6

CHAPTER 10

Introduction to Simple Experiments

Chapter Overview

▪ Example of simple experiments

▪ Experimental variables

▪ Why experiments support causal claims

▪ Experimental design ▪ Independent-groups designs

▪ Within-groups designs

▪ Interrogating causal claims with the four validities

Interrogating Causal Claims with the Four Validities

▪ Construct validity: How well were the variables measured and manipulated?

▪ External validity: To whom or what can the causal claim generalize?

▪ Statistical validity: How well do the data support the causal claim?

▪ Internal validity: Are there alternative explanations for the outcome?

11/15/2020

7

Example: Note-taking Style

Experimental Design

▪ Participants invited to a classroom

▪ ½ the time the classroom = laptops, ½ the time = notebooks/pens

▪ Participants watched a TED Talk & took notes

▪ Then spent 30 min doing a different activity

▪ Then tested on TED Talk material

Construct Validity: How Well Were the Variables Measured and Manipulated?

1. Dependent variables: How well were they measured?

2. Independent variables: How well were they manipulated?

▪ Manipulation check → an extra dependent variable that researchers can insert into an experiment to convince them that their manipulation worked

▪ Pilot study → a simple study, using a separate group of experiments, that is usually completed before the study of interest to confirm the effectiveness of their manipulations

External Validity: To Whom or What Can the Causal Claim Generalize?

▪ Generalizing to other people

▪ Generalizing to other situations

▪ What if external validity is poor?

11/15/2020

8

Statistical Validity: How Well Do the Data Support the Causal Claim?

▪ Is the difference statistically significant?

▪ How large is the effect?

Internal Validity: Are There Alternative Explanations for the Outcome?

Three fundamental questions about internal validity

1. Were there any design confounds?

2. If an independent-groups design was used, did researchers control for selection effects using random assignment or matching?

3. If a within-groups design was used, did researchers control for order effects by counterbalancing?