One-Way ANOVA
wjm3774
Skillbuilderwk7MODULE15.docxMODULE 15 INTERPRETING ONE-WAY ANOVA RESULTS
Page navigation
· previous: Hypothesis Testing for Independent T-Tests Summary
· next: The Significance of the F test
· Go to page
Current Module | Pages 67 - 70
Learning Objectives
· Interpret the results from a one-way ANOVA
What you Need to Know
Before reading this skill builder, be sure to review the following concepts:
· Steps in testing hypotheses
· Null and alternative hypothesis
· Alpha level
· Type I error
· Type II error
· t-test for two independent groups
· Categorical variables
· Continuous variables
Introduction to One-Way ANOVA
When researchers conduct a one-way ANOVA, the goal is to examine whether there are mean differences among two or more groups. The one-way ANOVA is a useful statistical analysis for research designs that involve only one independent grouping variable (called a factor in experimental design) and a single continuous dependent variable. That is, a one-way ANOVA is appropriate in scenarios that meet the following criteria:
· There is one independent variable
· There is one dependent variable
· The independent variable has two or more levels
· The dependent variable can be considered to be continuous
In an ANOVA, just like in a t-test, a “level" is a group or category.
Example:
Imagine you want to see if cultural groups have different attitudes toward old people and decide to use the implicit Association Test (IAT), which is a computerized procedure for measuring attribute discrimination (see https://faculty.washington.edu/agg/iat_materials.htm# ). (You can participate in a study using the IAT approach at https://implicit.harvard.edu/implicit/). Using latency of responses to different kinds of stimuli, the test has been used to study attitudes towards age, race, skin-tone, religion sexuality, and weight, among other attributes ( https://faculty.washington.edu/agg/pdf/Lane%20et%20al.UUIAT4.2007.pdf ).
You might develop an IAT that measures attitudes toward old people and use it to compare four cultural groups: American Indians living on a reservation, inner city legal residents, suburban legal residents, and undocumented residents. Your interest in comparing four different groups will lead you to consider the one-way ANOVA. In your one-way ANOVA, you would test the following null hypothesis:
HO : μAmerican Indians=μinner city=μsuburban=μundocumentedHO : μAmerican Indians=μinner city=μsuburban=μundocumented
The alternative hypothesis would be:
HA : not HOHA : not HO
The null states that all four populations have the same mean for the IAT test: that is, that all four populations have the same attitudes, on average, toward old people. The alternative states that one or more of the population means differs from the others: that is, that the four populations are not equivalent in their attitudes, on average, toward old people.
In our IAT research scenario, for example, there are four levels: American Indians, inner city, suburban, and undocumented. Note that although we can use a one-way ANOVA if we have just two levels, a t -test is probably the simpler approach in this case. One-way ANOVAs are typically used when there are more than two levels.
Also note that, although the independent variable in an ANOVA can technically be either categorical or continuous, researchers typically use an ANOVA in cases in which there are not an excessive number of levels. When there are too many levels, there may be very few participants in each level (in each group) and there will likely not be enough statistical power to discern any differences between the groups. Hence, the one-way ANOVA is usually applied when the independent variable has relatively few levels, and most of the time, the independent variable in a one-way ANOVA is categorical.
Learn by Doing
Hints, displayed below
For each proposed research study, indicate whether the one-way ANOVA is an appropriate choice for analyzing the data.
|
Table of multiple choice questions |
||
|
|
Appropriate |
Not appropriate |
|
A true experiment is conducted, and there are two independent variables (factors). The first has three levels, and the second has two levels. The researcher randomly assigns each of 60 participants to one of the six cells (3 x 2) in the experiment. Each participant is assigned to one level of the first factor and one level of the second factor. |
|
|
|
A researcher wants to see if IQ and sound decision-making are related. The data will consist of an IQ score for each participant along with a score that is obtained by evaluating each participant’s decisions using a scale of 1 to 10 for a set of 10 decisions. |
|
|
|
A food producer wishes to compare profitability of sales of fresh vegetables using three sales methods: big-box grocery chains, specialty grocery stores, and home delivery. The producer randomly samples 10 facilities of each type and calculates profitability (total sales costs) for one month for each facility. |
|
|
|
A group of high school teachers wants to predict which students will pass their final exam and which students will fail the exam. In order to make this prediction, they ask their students how many hours they spend studying during a typical week. |
|
|
MODULE 15 INTERPRETING ONE-WAY ANOVA RESULTS
Page navigation
· previous: Introduction to One-Way ANOVA
· Go to page
Current Module | Pages 67 - 70
The Significance of the F test
Learning Objectives
· Interpret the results from a one-way ANOVA
The basic logic of the t -test for two independent groups can be applied to the F- test for the one-way ANOVA. The one-way ANOVA can be thought of as an extension of the t- test. In a t- test, researchers examine mean differences between two groups. In the one-way ANOVA, researchers use an F -test to examine whether there are mean differences among two or more groups. When conducting a t -test, researchers use a sampling distribution for the t -test to determine whether the t- test is statistically significant. Significance for the F -test is determined by examining a sampling distribution for the F- statistic under the assumption that the null hypothesis is true. For example, if you were to conduct the study described above involving the four different cultures and the IAT and had 100 total participants, the sampling distribution for the test statistic would have the distribution shown in figure 1 below.
Figure 1. F with 3 and 96 df
Just as with the t -test, the hypothesis test must account for a type I error (i.e., rejecting the null if the null is true). In this example, the researcher will reject the null hypothesis if the value of F based on the data (the observed value of F ) is greater than the critical value of F (2.69939) that has been determined by alpha along with the degrees of freedom for the F statistic. If alpha has been set at .05, when the observed value of F exceeds the critical value of F , the p -value will be less than .05. With a p -value less than .05, researchers would reject the null hypothesis. In figures 2 and 3, note how the shape of the F distribution changes with different degrees of freedom.
Figure 2. F with 1 and 98 df
Figure 3. F with 7 and 92 df
Also note that alpha is only in the upper tail of the test statistic’s sampling distribution. With the t test for two independent groups, alpha is divided in half for a two–tailed test, with one half being used in the upper tail of the test statistic’s sampling distribution and the other half in the lower tail. With ANOVA, even though the alternative hypothesis does not specify a direction for the inequalities, only one tail of the F distribution is used to determine whether to reject the null.
The good news for modern day researchers is that SPSS and other statistical programs compute p-values for the F- test. Recall that the p-value is the probability of obtaining a value of the test statistic that is more extreme than the observed value if the null hypothesis is true.
Learn by Doing
"Fill in the blank" question: select the correct answer.
Consider the initial example of comparing the attitudes toward the elderly using the IAT. Imagine that 25 people from each of the four cultures were recruited randomly and took the IAT. To test the null hypothesis of equivalent attitudes toward old people across the four groups, a one-way ANOVA was conducted using SPSS, and the following results were obtained. (Note: Data used for this analysis are not real).
ANOVA
|
Dependent |
Sum of Squares |
df |
Mean Square |
F |
Sig. |
|
Between Groups Within Groups Total |
2919.216 10239.868 13159.084 |
3 96 99 |
973.072 106.665 |
9.123 |
.000 |
What SPPS has labeled as Sig is what is commonly called the p -value.
If alpha is equal to 0.05, the researcher Hint, displayed below-Select- the null hypothesis.
Post-Hoc Tests
Learning Objectives
· Interpret the results from a one-way ANOVA
If the overall F -test leads the researcher to reject the null, belief in the null hypothesis is transferred to the alternative. But the alternative merely states that some of the means may not be equal to the others. How can the researcher decide which population means are likely to differ from the others?
A tempting, but ill-advised, approach is to conduct t tests involving every possible pair of means using a .05 alpha probability. In general, you would conduct a t-test comparing groups 1 and 2, 1 and 3, 1 and 4, 2 and 3, 2 and 4, and 3 and 4. The problem with doing the multiple t-tests is controlling the probability of a type I error. With six independent hypothesis tests and alpha equal to .05 for each individual test, the probability of one or more type I errors for the six tests is actually about 0.26. Hence, if all four population means are equal, the probability of deciding at least two of the means differ from each other is about one in four—certainly much greater than one in twenty, as would be the case with a .05 alpha level.
To avoid what is called the inflated probability of a type I error, there are a number of approaches that can be used after an overall F -test is found to be significant. One of the easiest ways of approaching the problem is to use a Bonferroni approach. The approach is rather simple and involves dividing alpha for the family or set of comparisons by the number of hypothesis tests within the set. Thus, if the set of comparisons involves 10 hypothesis tests, divide .05, the probability for one or more type I errors in the set of 10 hypothesis test, by 10 to get .005. Perform the individual tests using alpha equal to.005 for each test.
The Bonferroni approach is the basis for one type of post-hoc analysis offered in many statistical packages like SPSS. Post-hoc analyses are examined only if the initial F -test for a one-way ANOVA is found to be statistically significant (i.e., the null is rejected). Post-hoc analyses allow researchers to see which means are significantly different from one another. Some statisticians, however, feel that the Bonferroni approach overcorrects for the inflated risk of type I error and therefore is somewhat lacking in statistical power. There are several alternatives to the Bonferroni approach listed in SPSS. They apply different models to control the probability of a type I error for the set of comparisons. The models vary in statistical power and make different assumptions. The focus here will be on interpreting the SPSS output rather than an in-depth discussion of the various types of post-hoc tests.
The results shown in Table 1 are sample means for each of the four cultures. Table 2 shows the results for Tukey’s Honestly Significant Difference Test (HSD), one type of post-hoc analysis. The results displayed here are based on the same data used in the activity above in which the null hypothesis that four population means are equal was rejected.
To interpret the results in Table 2, first look at the number -4.80308 . This is the difference in sample means for the American Indians sample and the Inner City sample (remember, the data are fictitious and only being used to illustrate how SPSS results are used). To verify the calculation, subtract 58.6210 (the Inner City sample mean) from 53.6210 (the American Indian mean). Note the result is negative because the Inner City mean is greater than is the American Indian mean.
Now look at the number 359 . This value is the p -value for the individual comparison. Recall that the HSD procedure is controlling alpha (.05) for the set of all comparisons, but you can reject the null for this individual comparison if the p -value (Sig) is less than .05. Hence the decision in the first row is to retain the null that the population mean for American Indians and Inner City groups are equal; that is, you would not reject the null hypothesis and do not have evidence that the means for those two groups are different from one another.
Finally, look at the first row in the columns labeled 95% confidence interval,
You can see that the value of 0, which would indicate no mean difference between the two groups, is contained within the confidence interval. This is another indication that there is not evidence that the two means differ from one another.
Learn by Doing
Hints, displayed below
Decide whether each statement below is consistent with the post-hoc results from SPSS.
Interpreting One-Way ANOVA Summary
Summary
In thinking about whether to conduct a one-way ANOVA for your research question, consider first your research goal. If you want to examine mean differences across several groups, a one-way ANOVA may be appropriate for you to use. When you conduct a one-way ANOVA, you will want to first see whether the overall F -test for the ANOVA is statistically significant. If the F -test is, in fact, significant, you will examine the results from your post-hoc analyses to examine which means are different from one another. There are many different types of post-hoc analyses, and you will want to read more about the various post-hoc approaches to determine the best fit for your data.
Before You Continue
Evaluate your ability to perform each of the following tasks. In other words, how well can you do each task?
|
Table of multiple choice questions |
|||||
|
|
Not at all yet |
With a lot of support |
With some support |
With minimal support |
On my own |
|
Interpret the results from a one-way ANOVA* |
|
|
|
|
|
* Required questions
What concept or topic is the least clear to you at this point?
What other questions do you have?
Interpreting One-Way ANOVA Results
68
67
