data analysis

The Analysis of Variance is used to determine the statistical significance between more than two group means. This statistic identified differences across groups in average scores. The statistic generated from using the Analysis of Variance (ANOVA) is the F-test (or F-Ratio). The F is for Sir Ronald Alymer Fisher, its developer.

Brief History of Sir R.A. Fisher:

Fisher is regarded as the founder of modern statistics. He also developed the experimental method we use today in his book, Statistical Methods for Research Workers (1925). Fisher was a mathematician who turned his work toward Biometrics. A means of mathematically proving Darwin’s Theory of Evolution. Fisher would use Analysis of Variance to account for the possibility of chance occurrences that Mendelian biologists struggled to explain. Fisher began to recognize that heredity and genes are not blended together as they are passed from parents to children; rather they are distinct and separate traits that may not be revealed for several generations, or may show up in the next generation. [Think of a deck of cards – rather than paint being blended together.]

The One-Way Analysis of Variance (ANOVA) is used when one factor or one treatment variable (such as different group memberships) is being investigated, and this factor has more than two levels. It is called analysis of variance because the calculation of the ANOVA involves comparing the amount of variance (sums of squares) that exists within groups is compared to the amount of variances (sums of squares) that exist between groups. These two types of variances are compared and produce a ratio: Thus the F-Ratio. The F-Ratio will produce a test statistic that indicates the ratio of variances within groups to variances between groups. (See pp. 247-249).

The ANOVA is similar to the independent t-test; except the comparison is made between more than two groups (there are more than two means). This statistic is used when:

1. There is only one dimension or treatment

2. There are more than two levels of the group factor (more than two groups)

3. We are looking at differences across groups in mean scores.

We follow the same 8 steps addressed previously in testing a null or research hypothesis. The F-Ratio statistic will generate an obtained value that is indicated in the Table of Critical Values. Once again, we consider sample size (degrees of freedom of the number of groups – e.g. k-1 or 3-1 = 2 and N-K or 30-3 = 27.) We then identify the obtained value or F-Ratio to determine if the differences are statistically significant or due to chance. (See page 252 for a diagram of the source table that indicates a comparison of variance – sum of squares. Notice the Mean Sum of Squares of 566.54 (between groups) and the Mean Sum of Squares 64.39 (within groups), then the F of 8.799. This is the ratio comparison between the two sums of squares. A ratio of almost 9:1 of variances between compared to variance within. This difference between groups could not have been due to chance or the amount of variance within the groups. So, an F(2,27) = 8.80, p<.05 indicates that the F-ratio with this sample was statistically significant at an Alpha level of p<.05.

The text presents a formula for calculating Effect Size for the ANOVA. Note this calculation is different from the previous t-tests we saw earlier. It is the calculation for a value called eta squared η2. (See formula on page 257). Basically it is the between groups variance – sums of squares divided by the total sums of squares. It provides the similar numerical comparison as Cohen’s d in terms of small, medium, and large effect sizes.

However, most statisticians tend to forego the eta squared and perform a post hoc (after the fact) comparison to isolate where the actual variance exists – to see where the difference lies. The reason for this additional step is due to the fact that the ANOVA simply indicates there are differences in variances between the three groups; however, it does not indicate where these differences are.

A common post hoc computation that is used is called THE BONFERRONI. There are others, but the author likes this one. The SPSS can actually calculate the BONFERRONI for us. (See pp. 257-258). This statistical procedure will isolate where the true differences are. In the case presented in the text (see Page 258, Table 13.6), we can see that the differences are between the 5-hour group and the 20-hour group in this study. This study compared the effects on language development on three groups: Group 1 received 5 hours per week, Group 2 received 10 hours per week, and Group 3 received 20 hours per week. The dependent variable was a test of Language Development.

The text also mentioned the possibility of another treatment or independent variable that could have been considered. In this case – gender within the three groups. This research question would have generated a FACTORAL DESIGN or, in this case, a Two-Way ANOVA because we would have had two factors to consider – the treatment (number of hours), and the gender within the groups. Chapter 14 addresses the various ANOVAs as we increase the possible independent variables.

The ANOVA is one of the more powerful inferential statistics we use. It considers not only the difference between means, but also the amount of variances within each group. It is sometimes referred to as an OMNIBUS ROBUST statistics. We do not consider whether it is a one-tailed or two tailed test because more than two groups are being analyzed, and we are simply looking at the differences between means, so a specific direction or difference does not matter.

Key Terms:

Analysis of Variance: (ANOVA) A test for the differences between two or more group means by calculating the amount of variances (sums of squares) that exist within the groups to the amount of variance (sums of squares) that exist between the groups to determine if the differences between the groups are statistically significant. The ANOVA generates a statistic called the F-Ratio (named after Sir Ronald Fisher).

One-Way Analysis of Variance (One-Way ANOVA) This ANOVA determines if differences exist between two or more groups based on a single treatment variable.

Factorial Analysis: An ANOVA that compares the effects of more than one treatment or independent variable on two or more groups’ mean scores. Example: A comparison of three different reading programs on the reading skills of third graders – also comparing the effects of these programs one boys and girls. (Two independent variables = reading programs, and gender)

Post hoc comparisons: Following the running of an ANOVA, a method to determine at specific group comparisons to isolate the possible cause for overall differences between the three or more groups.

eta squared η2: The measure of effect size used for ANOVA F-tests.

The ANOVA is the holy grail of statistics. Once you have developed an understanding of this statistical analysis, you should be able to handle any statistical test that comes your way.