Statistics questions (The Basics of Hypothesis Testing)

MTH 245 Lesson 18 Notes The Basics of Hypothesis Testing

The basic idea underlying hypothesis testing is the Rare Event Rule of Inferential Statistics:

If, under a given assumption, we observe a result that is statistically significant with respect to that assumption, then we can conclude that the assumption is probably not correct.

A hypothesis is a formal statement about an assumption or claim we’ve made about some aspect of a population. It usually takes the form of a symbolic statement about a population parameter (e.g., "𝜇𝜇 = 0"). A hypothesis test (or test of significance) is a structured mathematical procedure for evaluating such a claim. It is a direct application of the Rare Event Rule. There are three methods for testing a hypothesis: p-value, critical value, and confidence interval. We'll focus on the p-value method in this section; the confidence interval method will be covered in Section 8.1. Every hypothesis test has the same five steps:

1. Structure formal hypotheses that identify the assumptions made about the population parameter.

2. Determine 𝛼𝛼, the significance level of the hypothesis test. 3. Calculate the p-value. 4. Compare the p-value to 𝛼𝛼 and make a decision regarding the

assumptions made in Step 1. 5. Interpret the decision made in Step 4 in light of the claim originally

made about the population parameter.

Structuring Hypotheses Every hypothesis test starts with two complementary hypotheses:

− The null hypothesis, denoted by 𝐻𝐻0, is the assumption to be tested. It is always formulated as an equality.

− The alternate hypothesis, denoted by 𝐻𝐻𝐴𝐴 (or sometimes 𝐻𝐻𝑎𝑎 or 𝐻𝐻1), is a counterclaim that the parameter in question is different from the value assumed under 𝐻𝐻0. It is always formulated as a strict inequality (<, >, or ≠).

The goal of the process is to decide if there is sufficient statistical evidence to reject 𝐻𝐻0, given a particular significance level (more on this later). If there isn't enough evidence, we will fail to reject 𝐻𝐻0.

Note: We never use the term "accept" in reference to either hypothesis, because that connotes "proof" that the hypothesis is true. A hypothesis test can't be used to "prove" anything, because there's always a chance that we could make an incorrect decision about the parameter based on the results of the test.

Note that the original claim made about the population parameter might wind up as either 𝐻𝐻0 or 𝐻𝐻𝐴𝐴, depending on the nature of the investigation and the way the problem is stated.

− If the original claim is a strict inequality (<, >, or ≠), it automatically becomes 𝐻𝐻𝐴𝐴.

− If the original claim is an equality (=) or a weak inequality (≤, ≥), then its complement becomes 𝐻𝐻𝐴𝐴.

The following table lists relational operators and their complements:

Operator Complement = ≠ ≤ > ≥ <

Example 1: Convert the following original claims into formal hypotheses.

a. The mean pulse rate for adult females is 76 beats per minute or lower.

Define 𝜇𝜇 as the mean pulse rate. The original claim is 76 beats per minute or lower, which translates to 𝜇𝜇 ≤ 76. Since this is a weak inequality (see above), it becomes 𝐻𝐻0, and its complement—𝜇𝜇 > 12—becomes 𝐻𝐻𝐴𝐴:

𝐻𝐻0: 𝜇𝜇 ≤ 76

𝐻𝐻𝐴𝐴: 𝜇𝜇 > 76

Unfortunately, as noted earlier, the null hypothesis must always take the form of an equality (=). However, we can simply convert the ≤ symbol in 𝐻𝐻0 to an equal sign (refer to this lesson's video for further information), which gives us

𝐻𝐻0: 𝜇𝜇 = 76

𝐻𝐻𝐴𝐴: 𝜇𝜇 > 76

b. Fewer than 20% of adult Americans smoke tobacco.

Define 𝑝𝑝 as the proportion of all adult Americans smoke tobacco. The original claim is that 𝑝𝑝 < 0.200. Since this is already a strict equality, this becomes 𝐻𝐻𝐴𝐴, and its complement—𝑝𝑝 ≥ 0.200—becomes 𝐻𝐻0. Using the same reasoning as in Part b above, we convert 𝐻𝐻0 to an equality, which gives

𝐻𝐻0: 𝑝𝑝 = 0.200

𝐻𝐻𝐴𝐴: 𝑝𝑝 < 0.200

c. The average weight of the liquid in a drink container is 12 fluid ounces.

Define 𝜇𝜇 as the average weight. The original claim is that 𝜇𝜇 = 12. Since this is already an equality, this becomes 𝐻𝐻0, and its complement—𝜇𝜇 ≠ 12— becomes 𝐻𝐻𝐴𝐴:

𝐻𝐻0: 𝜇𝜇 = 12

𝐻𝐻𝐴𝐴: 𝜇𝜇 ≠ 12

Testing a Set of Hypotheses Once we've structured the hypotheses, we collect data and use it to calculate the appropriate sample statistics we need for the test. For tests related to a population mean 𝜇𝜇, we calculate �̅�𝑥. For tests related to a population proportion 𝑝𝑝, we calculate �̂�𝑝.

Applying the Rare Event Rule, if we observe a value of �̅�𝑥 that is different from the value of 𝜇𝜇 we assumed in the null hypothesis (call it 𝜇𝜇0), then 𝐻𝐻0 is probably not true and should be rejected. By "different," we mean far enough away from 𝜇𝜇0 to not have occurred by random chance; in other words, �̅�𝑥 needs to be statistically significantly different from 𝜇𝜇0. (The same reasoning applies to comparing �̂�𝑝 to 𝑝𝑝.)

The criterion for statistical significance is α, is the significance level. (This is the same α discussed in Section 6.3.) Using α, it is possible to determine a set of critical values and use them to determine if the observed sample statistic is significant using the methods of Chapter 6.

However, the most common (and easier) way of determining significance is to calculate the p-value, which is the probability of observing �̅�𝑥 (or �̂�𝑝), or a more extreme value, assuming the null hypothesis is true. If �̅�𝑥 (or �̂�𝑝) is farther away from 𝜇𝜇0 than the critical value, then the p-value will be smaller than α. The figure below illustrates this relationship.

Note that in the above figure, the significance region (the area to the right of the significance threshold, is in the right tail of the curve. This is referred to as a right-tailed test. In this case, 𝐻𝐻𝐴𝐴 will contain the operator ">", since the intent of the test is to determine if the observed sample statistic is significantly higher than the parameter value assumed in 𝐻𝐻0. If 𝐻𝐻𝐴𝐴 contains the operator the operator "<", the significance region is on the left side of the graph, and the test is left-tailed. A two-tailed test contains the operator "≠", in which the significance region is evenly split between the distribution's two tails. The most commonly chosen value of α is 0.05. However, as discussed in Section 6.3, this is an arbitrary value, and in practice, α can take on any value the researcher feels is reasonable. Depending on the context, α could be as high as 0.20 or as low as 0.000001, although values greater than 0.10 or less than 0.01 are uncommon.

Once we calculate the p-value, we compare it to 𝛼𝛼. This process tells us whether the value of the sample statistic (�̅�𝑥 or �̂�𝑝) is statistically significant with respect to the assumptions of 𝐻𝐻0.

− If the p-value ≤ 𝛼𝛼, the observed evidence contradicts the assumptions of 𝐻𝐻0, so we reject it in favor of 𝐻𝐻𝐴𝐴. In this case, the results are statistically significant for the given value of 𝛼𝛼.

− If the p-value > 𝛼𝛼, the evidence is not strong enough to contradict 𝐻𝐻0, so we fail to reject it in favor of 𝐻𝐻𝐴𝐴. In this case, the results are not statistically significant for the given value of 𝛼𝛼.

Note: the p-value is a probability, so round it to three decimal places unless told otherwise.

Example 2:

a. Suppose when we conduct the hypothesis test in Example 1a, we calculate a p-value of 0.078. If we assume 𝛼𝛼 = 0.05, what decision should we make relative to 𝐻𝐻0?

Since the p-value 0.078 > 𝛼𝛼, fail to reject 𝐻𝐻0. b. Suppose when we conduct the hypothesis test in Example 1b, we

calculate a p-value of 0.023. If we assume 𝛼𝛼 = 0.05, what decision should we make relative to 𝐻𝐻0?

Since the p-value 0.023 < 𝛼𝛼, reject 𝐻𝐻0.

Example 3: What type of tests were conducted in:

a. Example 2a?

Since 𝐻𝐻𝐴𝐴 is 𝜇𝜇 > 76, the significance region lies to the right, and the test is therefore a right-tailed test.

b. Example 2b?

Since 𝐻𝐻𝐴𝐴 is 𝑝𝑝 < 0.200, the significance region lies to the left, and the test is therefore a left-tailed test.

Interpreting Results When we report the results of the test, our interpretation needs to be stated from the point of view of the original claim. The following table demonstrates the relationships between the original claim, the decision, and the proper interpretation:

Reject 𝐻𝐻0

(p-value ≤ 𝛼𝛼) Fail to reject 𝐻𝐻0

(p-value > 𝛼𝛼)

Original claim is 𝐻𝐻0 Sufficient evidence to

reject the original claim Insufficient evidence to reject the original claim

Original claim is 𝐻𝐻𝐴𝐴 Sufficient evidence to support the original

claim

Insufficient evidence to support the original

claim Example 4:

a. Interpret the decision made in Example 2a in terms of the original claim from Example 1a.

Since the original claim is 𝐻𝐻0 and we have failed to reject 𝐻𝐻0, there is insufficient evidence to reject the claim that the mean pulse rate of adult females is 76 bpm or lower.

b. Interpret the decision made in Example 2b in terms of the original

claim from Example 1b.

Since the original claim is 𝐻𝐻𝐴𝐴 and we have rejected 𝐻𝐻0, there is sufficient evidence to support the claim that fewer than 20 percent of adult Americans smoke tobacco.

Errors in Hypothesis Testing Because a hypothesis test is a statistical procedure, there's a risk of making an incorrect decision with respect to 𝐻𝐻0. These errors can be categorized as follows:

𝑯𝑯𝟎𝟎 true 𝑯𝑯𝟎𝟎 false

Reject 𝑯𝑯𝟎𝟎 Type I Error

(probability = 𝛼𝛼) Correct decision

Fail to reject 𝑯𝑯𝟎𝟎 Correct decision

(probability = 1 − 𝛽𝛽) Type II Error

(probability = 𝛽𝛽) The quantity 1 − 𝛽𝛽 (the probability of correctly failing to reject 𝐻𝐻0) is referred to as the power of the hypothesis test.

It is the accepted practice to choose 𝛼𝛼 before conducting the hypothesis test, and to allow 𝛽𝛽 to find its own level. This is because for a given 𝛼𝛼, there is no single corresponding value of 𝛽𝛽. The calculation of 𝛽𝛽 depends on a variety of factors; usually, it is only possible to define a range of values of 𝛽𝛽 for any given 𝛼𝛼. The probabilities 𝛼𝛼 and 𝛽𝛽 are inversely related: when one increases, the other decreases. To better understand the implications of this relationship, suppose that instead of conducting a hypothesis test, we are instead judging guilt or innocence in a court of criminal law. Assuming our "null hypothesis" is that the defendant is innocent, our decision options are:

𝑰𝑰𝑰𝑰𝑰𝑰𝑰𝑰𝑰𝑰𝑰𝑰𝑰𝑰𝑰𝑰 𝑮𝑮𝑮𝑮𝑮𝑮𝑮𝑮𝑰𝑰𝑮𝑮 Convict False conviction Justice served Acquit Justice served False acquittal

As the rate of false convictions drops, more guilty people will be acquitted, and vice versa.

Example 5: Suppose we conduct the hypothesis test in Example 1c (𝐻𝐻0: 𝜇𝜇 = 12 fl oz). How would we describe the Type I and Type II errors for this test in words?

If the mean weight of the fluid in the container truly is equal to 12 fl oz, but our sample data leads us to conclude otherwise by rejecting 𝐻𝐻0, we have committed a Type I error.

On the other hand, if the mean weight of the fluid in the container is different from 12 fl oz, but our sample data leads us to conclude otherwise by failing to reject 𝐻𝐻0, we have committed a Type II error.