Psychology 302

1

Null

Hypothesis

Significance

Testing

Hypothesis Testing  Method to decide if an observed result is unlikely to have

occurred by chance

 Assumes data were obtained using a random sampling procedure  Probability associated with test may be wrong if sample is

not random

 In most real studies, p < .05 probably is not really p < .05!

Hypotheses  Research Hypothesis

 Outcome you expect if your theory is true

 Statistical Hypothesis

 A research hypothesis stated in terms of the distribution under investigation.

 Two types:

 Null Hypothesis (H0)

 Alternative Hypothesis (H1)

Null Hypothesis (H0)  States that the population parameter is some particular

value

 Makes a prediction opposite of the research hypothesis

 Rejecting H0 supports the research hypothesis

Alternative Hypothesis (H1)  States that the population parameter is some alternative

range of values

 The opposite of H0  Makes a prediction consistent with the research

hypothesis

Examples of Null and Alternate Hypotheses

 H0 :  = 0 (treatment had no effect)

 Ha :   0 (treatment did have an effect)

 H0 :   100 (no effect or negative effect)

 Ha :   100 (positive effect)

 H0 : r = 0 (no linear relationship)

 Ha : r  0 (linear relationship is not zero)

2

Nondirectional Hypothesis  Two-tailed

 Null will be rejected if test statistic is much higher OR lower than prediction

Directional Hypothesis  One-tailed

 Decide ahead of time: Null will be rejected if test statistic is much higher than prediction  (or much lower, depending on your prediction!)

 Use only if willing to ignore an extreme value in opposite direction from what is expected

How to test a null Hypothesis

 Assume that H0 is true, determine sampling distribution

 Draw a random sample from the population

 What is the probability that this sample could have been drawn if H0 was actually true?

Declaring Statistical Significance

 If the observed results are very unlikely to have occurred just by chance if H0 was actually true…

 You conclude that H0 is probably NOT true – you REJECT the null

Statistical Significance

 H0 is rejected

 The observed effect is greater than we would expect due to chance if the null were true

Practical Significance

 The observed difference has conceptual or practical meaning

 statistically significant may not be practically meaningful (and vice-versa)

3

Test Statistic

 Numeric summary of how far an observed estimate is from the parameter specified in H0

 Common ones are z, t, c2, F

Critical Value

 Value of a test statistic that cuts off the desired alpha level (region of rejection)

 Usually a = .05

 Look this up in a table

Hypothesis Testing: Formal Steps 1. Generate H0 and H1 2. Select statistical procedure (z, t, etc)

3. Select a

4. Calculate observed statistic for your data

5. Determine critical value

6. Compare (4) and (5)

7. If (4) exceeds (5), reject H0 8. Otherwise, fail to reject H0

Truth of the Universe H0 True (no effect) H0 False(effect exists)

Do not reject H0 (say there is no effect)

Type I error

a

Correct Decision

1 - b

Type II error

b

Correct Decision

1 - a

Your Decision

Reject H0 (say the effect exists)

 Probability is associated with your decision

 Probability is NOT associated with the truth of the universe

 Within each possible reality, decision probabilities sum to 1.0

An analogy with the legal system  US legal system is based on the premise “innocent until

proven guilty”

 Jury tests the null hypothesis of “didn’t commit the crime”

 Rejecting the null = “guilty”

 Failing to reject the null = “not guilty”  (notice that we don’t say innocent)

 What is a?

4

What really happened H0 True (Didn’t do it) H0 False (did it)

Do not reject H0 (Not guilty)

Innocent person is convicted

Type I error

Guilty person is convicted

Correct decision

Guilty person gets off

Type II error

Innocent person goes home

Correct decision

Jury Decision

Reject H0 (guilty verdict)