anova complex
Steven911
08CH_Sukal_Statistics.pdf
iStockphoto/Thinkstock
chapter 8
Factorial and Mixed-Factorial Analysis of Variance
Chapter Learning Objectives
After reading this chapter, you will be able to. . .
1. explain factorial and mixed-factorial designs.
2. relate sum of squares to factorial models.
3. compare, contrast, and identify various factorial designs.
4. demonstrate how to determine the main and interaction effects in factorial designs using multiple variables.
5. explain the combination of between- and within-group variability to create mixed designs.
6. explain the use of partial-eta-squared (partial-h2) in ANOVA.
7. interpret results of factorial and mixed-factorial designs and draw conclusions on these findings.
8. present relevant factorial and mixed results in APA format.
9. explain more complex design as a transition into advanced statistical courses.
CN
CO_LO
CO_TX
CO_NL
CT
CO_CRD
suk85842_08_c08.indd 287 10/23/13 1:41 PM
CHAPTER 8Section 8.1 Factorial Analysis of Variance
Building on the concepts of Chapters 6 and 7 and the statistical calculations of analysis of variance, we now consider more complex between-group designs called factorial ANOVA and a combination of between- and within-groups designs known as mixed- factorial ANOVA. The goal here is to explore the main effects, which are the influence of the independent variable on the dependent variable in testing a hypothesis, and to consider the combination of IVs influencing the DV known as interaction effects. We will continue to build on the magnitude of variance of the IV on a DV, or effect size that was introduced in Chapter 5 with Cohen’s d and in Chapters 6 and 7 with h2 and v2. Here we will add another effect size measure, partial-h2, to the list of effect size types.
The current chapter will also introduce even more complex designs such as MANOVA (multiple analysis of variance), ANCOVA (analysis of covariance), and MANCOVA (mul- tiple analysis of covariance). By the end, you will have a basic understanding of factorial designs and consider examples of these calculations using statistical software.
8.1 Factorial Analysis of Variance
Before we consider factorial analysis of variance (ANOVA), we first need to have a brief introduction to what are called factorial designs. In the language of statistics, a fac- tor is an independent variable, and a factorial ANOVA is one that includes multiple IVs (or factors) on one DV. Each of these relationships (i.e., an IV-DV relationship) is called a main effect.
As previously discussed, fluctuations in scores that are not explained by the IV(s) in the model emerge as error variance or unsystematic/unexplained variance because it has not been included in the experimental condition. Specifically, any variability in the IV(s) that are not related to the subjects’ DV becomes part of SS error (SSerror) and then the MS within (MSwith), which is a calculation of SSerror divided by the degrees of freedom (df ).
Building on the concept of sum of squares, in the factorial ANOVA, multiple IVs (or fac- tors) can be included. As long as the researcher has data for each variable, there is no theo- retical limit to the number of factors. For each factor, a sum-of-squares value is calculated and divided by its degrees of freedom (df ) to produce a mean square (MSbet). Each MSbet is divided by the same MSwith (or error) value to produce F so that there is a separate F for each factor.
In addition, sometimes factors in combination affect the DV differently than they do indi- vidually. This is called an interaction. The F values are also calculated for interactions of these factors. For example, if a researcher wanted to examine the impact of marital status and college graduation on subjects’ optimism regarding the economy, data would be gathered on subjects’ marital status (married or not married) and their college educa- tion (graduated or did not graduate). Each of these combinations of levels on their level of optimism (the DV) would then be explored. Thus, the main concept of using multiple factor designs is to explore these interaction effects.
H1
TX_DC
BLF
TX
BL
BLL
suk85842_08_c08.indd 288 10/23/13 1:41 PM
CHAPTER 8Section 8.1 Factorial Analysis of Variance
As with all the ANOVAs discussed in prior chapters, we have explored the concept of variability and how that is broken down into each facet of systematic variance (the vari- ance that we know) and unsystematic variance (the variance that we do not know or error). Simply, the greater the ratio of systematic to unsystematic variance, the higher the F value and the stronger the overall ANOVA model. The Factorial ANOVA model breaks down the Total variability into between treatments (systematic variance) and within treatments (unsystematic/unexplained variance or error). Subsequently, the between- treatments portion is further portioned into factor A variability, factor B variability, and the interaction of these factors, A 3 B variability.
SS values, MS values, and F ratios would be calculated for all variables that are similar to between-group designs such as a one-way ANOVA:
• for the first factor (IV1) • for a second factor (IV2) • for the two factors (IVs) in combination or the interaction
The procedures involved in calculating a factorial ANOVA are numerous and complex (see Table 8.1). Happily they are no longer calculated by hand. Software programs includ- ing SPSS can perform an ANOVA with two factors (a two-way ANOVA), three factors (a three-way ANOVA), and so on. As you can see, the numbers in each of these terms rep- resent the factors or IVs, hence the term factorial designs. Perhaps you have also noticed that as you add factors the analysis becomes more multifaceted because more interaction effects between the factors will occur. This is not necessarily more complicated, but simply more information in the output to interpret.
Table 8.1: Formula of the factorial ANOVA model
Source SS df MS
Factor A SS(A) (a 2 1) MS(A) 5 SS(A)/ (a 2 1)
Factor B SS(B) (b 2 1) MS(B) 5 SS(B)/(b 2 1)
Interaction AB SS(AB) (a 2 1)(b 2 1) MS(AB) 5 SS(AB)/ (a 2 1)(b 2 1)
Error SSE (N 2 ab) SSE/(N 2 ab)
Total (Corrected) (N 2 1)
Source: NIST/SEMATECH e-Handbook of Statistical Methods, http://www.itl.nist.gov/div898/handbook/, April, 2012. Published by The National Institute of Standards and Technology (NIST), an agency of the U.S. Department of Commerce.
Factorial designs can be elegantly described using the number of groups and levels of each variable. For instance, the simplest two-way ANOVA will have two factors with two
suk85842_08_c08.indd 289 10/23/13 1:41 PM
CHAPTER 8Section 8.1 Factorial Analysis of Variance
The Research Methods Knowledge Base is an informative website that taps more into the design of experiments, including factorial designs. You can learn more about factorial designs by visiting the link provided below.
http://www.social researchmethods.net /kb/expfact.php
Try It! levels (or four conditions/cells). This is known as a 2 3 2 factorial ANOVA. If we had two variables with three levels, then this will be a 3 3 3 factorial design (nine conditions/cells). If we had three variables as in a three-way ANOVA, one with two levels, another with five lev- els, and the third with 10 levels, then this would be a 2 3 3 3 10 facto- rial design (60 conditions/cells). You will commonly see this notation used in journal articles and textbooks.
The conditions or cells are calculated by simply multiplying the levels of each factor. Moreover, as you continue to add factors, it becomes more complex with main and interaction effects. So for instance if there was at least a sample size of 10 per cell for a 2 3 2 3 3 ANOVA then that will be a sample size of n 5 120. This is a general rule in cal- culating appropriate sample sizes for factorial designs, albeit not the best approach. Instead GPower3 or SPSS Sample Power 3 should be employed.
The Statistical Hypotheses in a Two-Way ANOVA
In the previous section, we discussed the simplest factorial designs (i.e., a two-way ANOVA where each factor has two levels). In this situation, how would the hypotheses read? As noted, a 2 3 2 ANOVA is essentially a combination of two factors on a DV, so there are three hypotheses that will be explored.
As an example, the simplest factorial design is a 2 3 2 ANOVA, with two factors, each with two levels: Gender (Female, Male) and Martial Status (Married, Never Married) and a dependent variable, work-life balance (WLB). The design will have three null and three alternative hypotheses. In fact, when you are performing a factorial design, there will always be at least three hypotheses consisting of two main effects and one interaction effect.
For this example, the two main effects are:
H01: There is no statistically significant difference of Gender on Work-Life Balance
mmale_WLB 5 mfemale_WLB
Ha1: There is a statistically significant difference of Gender on Work-Life Balance
mmale_WLB ? mfemale_WLB
H02: There is no statistically significant difference of Marital Status on Work-Life Balance
mmarried_WLB 5 mnever married_WLB
Ha2: There is a statistically significant difference of Marital Status on Work- Life Balance
mmarried_WLB ? mnever married_WLB
suk85842_08_c08.indd 290 10/23/13 1:41 PM
CHAPTER 8Section 8.1 Factorial Analysis of Variance
The interaction hypothesis is
H03: There is no statistically significant interaction of Gender and Marital Status on Work-Life Balance
mmale/married_WLB 1 mmale/unmarried_WLB 5 mfemale/married_WLB 1 mfemale/unmarried_WLB
Ha3: There is a statistically significant interaction of Gender and Marital Status on Work-Life Balance
mmale/married 1 mmale/unmarried ? mfemale/married 1 mfemale/unmarried
Later in the chapter, we will test these hypotheses in performing the appropriate analysis using SPSS. See Example 1 in Section 8.6.
Effect Size Calculations for ANOVA
From our previous discussions of effect sizes, you will recall that an effect size for a one- way ANOVA is as such:
h2 5 SSeffect SStotal
Formula 8.1
The eta-squared (h2) calculation is straightforward as we have only one IV on the DV. This is certainly useful for one-way ANOVA designs but cannot be used in multiple-factor designs, as in a factorial ANOVA, as it cannot give effects based on each individual fac- tor’s variability nor error variability. A second drawback to h2 is that it tends to overes- timate the population variance as the calculation only involves the sums of squares as seen in the formula. Therefore, the solution to addressing these drawbacks in handling a two-way, three-way, or more complex ANOVA that involves factors simultaneously being analyzed with the DV is to first separate out the effects of each factor on the DV plus any additional effects of error, hence the term partial-H2 as compared to h2 in Chapter 6. In this instance, partial-h2 for each IV-DV is calculated by
Partial h2 5 SSeffect
SSeffect 1 SSerror Formula 8.2
Here the “partialing out” of the other variables is accomplished by each factor’s indi- vidual variance on the DV (SSeffect) and the error term (SSerror). This statistic is conveniently calculated using appropriate software (e.g., SPSS). In regards to Cohen’s (1988) effect size guidelines, small h2 5 .01, medium h2 5 .09, and large h2 5 .25. These are a good barome- ter to calculating effect sizes estimates using GPower3. The partial-h2 effect size will range from 0 to 1 with one drawback in that it cannot be summed across the variables to get a total effect size.
To address the second drawback of h2 is to calculate omega-squared (V2), which was intro- duced in Chapter 6. Omega is less biased in its calculation than h2 and is always lower in magnitude than its counterpart. In addition, it measures the overall effect that you cannot do with partial h2. The formula for this is
v2 5 SSeffect 2 1dfeffect2 1MSerror2
SStotal 1 MSerror Formula 8.3
suk85842_08_c08.indd 291 10/23/13 1:41 PM
CHAPTER 8Section 8.2 Applying Factorial-ANOVA Research Design
8.2 Applying Factorial-ANOVA Research Design
Factorial ANOVA can be used in an experiment where two factors, each with two levels, have created four independent conditions. This is a simple classic four-group design. Here four experimental groups are compared for significant differences. Therefore, such an experimental design is a 2 3 2 independent-design ANOVA where we have two factors with one of the factors (Time in Instruction) being the two time points (1 hour per week and 4 hours per week) and the other factor being the Setting (Pull-out and In-class). The DV or outcome variable is a measurement of Exam Performance (EP), which is the number of correct items of the final exam. So think of this analysis as executing two independent- samples t-test simultaneously with the added benefit of interaction effect between the factors (i.e., Time in Instruction and Setting).
For this 2 3 2 independent-group ANOVA the hypotheses will be
H01: There is no statistically significant difference of Time in Instruction groups on Exam Performance
m1hr_EP 5 m4hr_EP
Ha1: There is a statistically significant difference of Time in Instruction groups on Exam Performance
m1hr_EP ? m4hr_EP
H02: There is no statistically significant difference of Setting groups on Exam Performance
min-class_EP 5 mpull-out_EP
Ha2: There is a statistically significant difference of Setting groups on Exam Performance
min-class_EP ? mpull-out_EP
H03: There is no statistically significant interaction of Time in Instruction and Setting groups on Exam Performance
m1hr/in-class_EP 1 m1hr/pull-out_EP 5 m4hr/in-class_EP 1 m4hr/pull-out_EP
Ha3: There is a statistically significant interaction of Time in Instruction and Setting groups on Exam Performance
m1hr/in-class_EP 1 m1hr/pull-out_EP ? m4hr/in-class_EP 1 m4hr/pull-out_EP
Figure 8.1 illustrates this 2 3 2 factorial design as a comparison of the four group condi- tions on the dependent variable EP. In finding significant mean differences in EP between groups, the null hypotheses above can be rejected, and we can conclude that support was found for their respective alternative hypotheses.
suk85842_08_c08.indd 292 10/23/13 1:41 PM
CHAPTER 8Section 8.2 Applying Factorial-ANOVA Research Design
Figure 8.1: The classic four-group design experiment
Source: Trochim, William M. The Research Methods Knowledge Base, 2nd Edition. www.socialresearchmethods.net/kb/
More Complex Designs
To help you understand more complex designs and additional ANOVA concepts, we will demonstrate a condition/cell breakdown of the simplest three-way ANOVA, that is, the 2 3 2 3 2 ANOVA, which is three factors with two levels each. Labeling the three factors IV1, IV2, IV3, and the DV, the effects will be as such:
H1: Main effect: IV1 S DV
H2: Main effect: IV2 S DV
H3: Main effect: IV3 S DV
H4: Interaction effect: IV1 3 IV2 S DV
H5: Interaction effect: IV1 3 IV3 S DV
H6: Interaction effect: IV2 3 IV3 S DV
H7: Higher-order interaction effect: IV1 3 IV2 3 IV3 S DV
As you can see, there will be three main effects, and four interac- tion effects for this 2 3 2 3 2 factorial design. In addition, there will be a three-way interaction of factors on the dependent variable in what is known as a higher-order interaction effect. Now you can see that as more factors are added, the num- ber of higher-order interactions between factors will increase.
Group 1 average
In -c
la s
s P
u ll
-o u
t Group 2 average
Group 3 average
1 hour/week 4 hours/week
Group 4 average
Time In Instruction
S e tt
in g
Levels: Subdivisions
of Factors
Factors: Major Independent Variables
There are many applets on the Internet to help in data analysis on all of these statistical concepts. One useful website is the Statistics Online Computational Resource. Along with applets, there are other resources such as e-books, activities, and games to help you learn statistics—visit the link provided below:
http://www.socr.ucla .edu/SOCR.html
Try It!
suk85842_08_c08.indd 293 10/23/13 1:41 PM
CHAPTER 8Section 8.3 Determining the Results’ Practical Importance
To demonstrate conditions or cells by way of a simple example, assume a psychologist wants to study the effects of Gender (Male/Female), Relationship Status (Married/Not Married), and Age Range (Under 40/Over 40) on the subjects’ level of depression using the Beck’s Depression Inventory. In this scenario, a 2 3 2 3 2 factorial design, there will be eight conditions or cells:
Gender Relationship Status Age Range
Under 40 Over 40
Male Married C1 C2
Not Married C3 C4
Female Married C5 C6
Not Married C7 C8
C1: Male 3 married 3 under 40
C2: Male 3 Married 3 Over 40
C3: Male 3 Not Married 3 Under 40
C4: Male 3 Not Married 3 Over 40
C5: Female 3 Married 3 Under 40
C6: Female 3 Married 3 Over 40
C7: Female 3 Not Married 3 Under 40
C8: Female 3 Not Married 3 Over 40
Assumptions of Factorial Designs
As with all other parametric tests we have considered, the assumptions of factorial designs are the same in terms of linearity, homogeneity, and normality. In addition, since factorial designs are more complex compared to simpler designs, it requires a larger sample size as more factors are added. As we have mentioned, for more cells, with n 5 10 per cell, the number can increase with the addition of factors and factor level. This is a precau- tion in factorial designs that the researcher is able to ensure sample sizes are appropriate. Complex designs involving multiple factors will require larger data sets, which may not always be feasible, and a compromise in the sample size may lead to erroneous conclu- sions (i.e., the statistical conclusion validity).
8.3 Determining the Results’ Practical Importance
One of the main reasons for running factorial designs is to see the interaction effects of factors on each other and their influence on the DV. Figure 8.2 depicts how this can be determined visually. A general rule of thumb when using an ocular analysis (or eyeball
suk85842_08_c08.indd 294 10/23/13 1:41 PM
CHAPTER 8Section 8.3 Determining the Results’ Practical Importance
test) is to see if the lines of each of the levels of treatments are parallel to each other. Simply put, parallel 5 no interaction effect, and nonparallel 5 interaction effect. Keep in mind that this is a general rule of thumb. Relative to the sensitivity of the graph, lines may seem nonparallel, indicating an interaction effect; however, that may not be the case. In any event, these line graphs are used to corroborate the results based on the ANOVA table’s F and p-values.
Figure 8.2: Line graphs of interaction effects
Source: Yatani, K. (n.d.) Statistics for HCI research. Retrieved from http://yatani.jp/HCIstats/images/img/interactions.png
Examples that follow discuss interaction effects of analysis of real-world data.
N o
M a
in E
ff e
c t
No Interaction Interaction
M a in
E ff
e c t
No significant effects, no interactions A significant interaction
Factor A is significant
Factor B is significant
Factor A and B are significant
Significant main effects and a significant interaction
suk85842_08_c08.indd 295 10/23/13 1:41 PM
CHAPTER 8Section 8.4 Mixed-Factorial Analysis of Variance
8.4 Mixed-Factorial Analysis of Variance
Thus far, we have discussed both within and between group designs. Now we will describe the combination of both designs simultaneously performed in what is termed a mixed-factorial ANOVA. The simplest of such a design is a 2 3 2 mixed-design (four conditions) where both the experiment and the control groups are compared to each other over two points in time or treatments (pre- and postmeasure, for instance). Unlike the 2 3 2 independent four-group design in Section 8.2, the mixed design uses each group twice and, therefore, is also known as a two-group experimental design. Think of this analysis as executing both an independent-samples t-test and a dependent-samples t-test at the same time. Figure 8.3 gives an overview of the calculation of df based on the total sample size (N) and the number of treatment groups (k).
Figure 8.3: Mixed ANOVA df calculation
The sum of squares, mean squares, and calculation of F based on the mean squares are cal- culated with the formulas in Table 8.2. Again, these calculations are seldom done by hand with easy reliance on statistical programs. Calculations are based on Yijk, an individual score; Y.jk, a cell mean; Y.j., a mean for a level of factor A (the between-subjects factor); Y..k, a mean for factor B (pre- or posttest); Yi.., the mean for an individual student; and Y. . ., the grand mean.
Table 8.2: Mixed ANOVA model
Description SS (definitional formula) df MS F
A main effect (between-subjects) SSA 5 1b2 1n2 a 1Y.j. 2 Y...2 2 a 2 1
SSA dfA
MSA MSS/A
Error term for A main effects SSs/A 5 ba a 1Y.jk 2 Y.j.2 2 a(n 2 1)
SSs/A dfs/A
B main effects (within-subjects) SSB 5 1a2 1n2 a a 1Y..k 2 Y...2 2 b 2 1
SSB dfB
MSB MSB3S/A
dfbetween treatments k � 1
dfwithin treatments N � k
dftotal N � 1
(continued)
suk85842_08_c08.indd 296 10/23/13 1:41 PM
CHAPTER 8Section 8.4 Mixed-Factorial Analysis of Variance
Table 8.2: Mixed ANOVA model (continued)
Description SS (definitional formula) df MS F
Interaction SSA3B 5 na a a 1Y.jk 2 Y.j. 2 Y..k 1 Y...2 2 (a 2 1)(b 2 1)
SSA3B dfA3B
MSA3B MSB3S/A
Error term for B main effect and
interaction SSB3S/A 5 a a a 1Yijk 2 Y.jk 2 Yi.. 2 Y.j.2 2 a(b 2 1)(n 2 1)
SSB3S/A dfB3S/A
Total SST 5 c 1Yijk 2 Y...2 2 (a)(b)(n) 2 1
Source: Adapted from Myers, J.L., & Well, A.D., & Lorch, R.F.,Jr. (2010). Research design and statistical analysis (3rd Edition). Mahwah, NJ: Erlbaum.
A Mixed-Factorial ANOVA Example
Consider the same scenario used above, but this time we will be using two groups instead of four groups. Each group (In-class and Pull-out) will be subjected to two different con- ditions/treatments (Time in Instruction, e.g., 1 hour/week and 4 hours/week). Conse- quently, each of the two groups will be evaluated with an exam at two different time points.
This will be a 2 3 2 mixed-design ANOVA with the hypotheses as follows:
H01: There is no statistically significant difference of Time in Instruction on Exam Performance
m1hr_EP 5 m4hr_EP
Ha1: There is a statistically significant difference of Time in Instruction on Exam Performance
m1hrEP ? m4hr_EP
H02: There is no statistically significant difference of Setting groups on Exam Performance
min-class_EP 5 mpull-out_EP
Ha2: There is a statistically significant difference of Setting groups on Exam Performance
min-class_EP ? mpull-out_EP
H03: There is no statistically significant interaction of Setting groups based on Time in Instruction on Exam Performance
m1hr/pull-out_EP 1 m1hr/pull-out_EP 5 m4hr/in-class_EP 1 m4hr/pull-out_EP
suk85842_08_c08.indd 297 10/23/13 1:41 PM
CHAPTER 8Section 8.4 Mixed-Factorial Analysis of Variance
Ha3: There is a statistically significant interaction of Setting groups based on Time in Instruction on Exam Performance
m1hr/pull-out_EP 1 m1hr/pull-out_EP ? m4hr/in-class_EP 1 m4hr/pull-out_EP
Figure 8.4 illustrates this 2 3 2 mixed-factorial ANOVA design, which is a comparison of within group conditions (1 hour/week and 4 hours/week) as well as difference between groups (In-group and Pull-out). Within-group differences, specifically that EP was sig- nificantly different across groups having 1 hour/week versus 4 hour/week instruction, allows us to reject H01: m1hr_EP 5 m4hr_EP and find support for Ha1: m1hr_EP ? m4hr_EP or the alternative hypothesis main effect.
Establishing significant mean differences of EP between group Settings regardless of time conditions such that H02: min-class_EP 5 mpull-out_EP can be rejected in support for Ha2: min-class_EP ? mpull-out_EP, which demonstrates the second main effect.
Finally, establishing both within-group differences over conditions (1 hour/week and 4 hours/week) and between-group differences (In-class and Pull-out) allows us to reject H03: m1hr/pull-out_EP 1 m1hr/pull-out_EP 5 m4hr/in-class_EP 1 m4hr/pull-out_EP and find support for Ha3 : m1hr/pull-out_EP 1 m1hr/pull-out_EP ? m4hr/in-class_EP 1 m4hr/pull-out_EP.
Figure 8.4: The classic mixed-design experiment using two groups
Source: Adapted from Trochim, William M. The Research Methods Knowledge Base, 2nd Edition. www.socialresearchmethods.net/kb/
Group 1 average
In -c
la s s
P u
ll -o
u t
Group 2 average
Group 1 average
1 hour/week 4 hours/week
Group 2 average
Time In Instruction
S e tt
in g
Levels: Subdivisions
of Factors
Factors: Major Independent Variables
suk85842_08_c08.indd 298 10/23/13 1:41 PM
CHAPTER 8Section 8.5 Presenting Results
Assumptions of Mixed Designs
For mixed designs, all of the aforementioned parametric assumptions must be met includ- ing normality, homogeneity, and linearity. Recall that sphericity is the homogeneity of differences between pairs of treatments across subjects. Ideally, there should not be sig- nificant differences. Testing for this entails the use of the Mauchly’s W-test. The degree of such differences or the level of sphericity is further quantified by another measure called epsilon (E). This measure ranges from 0 to 1 with 1 being the highest measure that indi- cates sphericity has been met. As e , 1, sphericity decreases, which will lead to a violation of sphericity based on the Mauchly’s W-test.
When there is a violation of sphericity, two adjustments are made called the Greenhouse- Geisser and Huynh-Feldt corrections, which were introduced in Chapter 7. When using SPSS these e-values will be calculated with the appropriate df adjustments based on the simple formula of multiplying the e-values by the df. For instance, if
df 5 4
e 5 .8
Then,
Adjusted df 5 4 3 .8 5 3.2
In Chapter 7, we considered an example of a repeated-measures design where decisions based on the Mauchly’s W-test are made. In this chapter is an example of performing this analysis in SPSS and specifically examining sphericity with a closer look at e-values.
8.5 Presenting Results
Each of the following examples presents steps for completing a different type of ANOVA using SPSS. SPSS Example 1: Steps for a Simple ANOVA
To start with the simplest ANOVA example using public data from Pew Research (2010) data set, the Changing American Family, a 2 3 2 factorial design will be conducted using gender (sex) and marital status (mar_status). Here sex has two levels (male and female) and mar_status has two as well (married, never married). We will perform an analysis of these two factors on participants work balance (How good a job have you done balancing your job, your marriage or partnership, and being a parent? Would you say excellent, very good, good, only fair, or poor?).
suk85842_08_c08.indd 299 10/23/13 1:41 PM
CHAPTER 8Section 8.5 Presenting Results
To perform the analysis, go to Analyze S General Linear Model S Univariate. Input sex and mar_status into the Fixed Factor(s) box and q28 into the Dependent Variable box (your screen should look like that in Figure 8.5). Next, click on Plots and move sex into Separate Lines and mar_status into Horizontal axis; then click Add and Continue. Then click Options and check Descriptive statistics and Estimates of effect size. Then click Continue and OK. Analysis results are presented in Figures 8.6 and 8.7.
Figure 8.5: SPSS steps for the 2 3 2 ANOVA
Source: Data from Pew Research: Social and Demographic Trends. (2010). Changing American family. Retrieved from http://www .pewsocialtrends.org/category/datasets/
suk85842_08_c08.indd 300 10/23/13 1:41 PM
CHAPTER 8Section 8.5 Presenting Results
Figure 8.6: SPSS output for the 2 3 2 ANOVA
Source: Data from Pew Research: Social and Demographic Trends. (2010). Changing American family. Retrieved from http://www .pewsocialtrends.org/category/datasets/
Between-Subjects Factors
SEX [ENTER RESPONDENT’S SEX:]
Marital Status
1
5
1
2 Female
Value Label
Male
Married
Never married
N
365
706
93
434
Descriptive Statistics Dependent Variable: Q.28 How good a job have you done balancing your job, your (IF MARITAL = 1: marriage/IF MARITAL = 2: partnership) and being a parent? Would you say excellent, very good, good, only fair, or poor?
Male
SEX [ENTER RESPONDENT’S SEX:] Marital Status
Female
Total
Never married
Mean
Married
Total
NStd. Deviation
Never married
Married
Total
Never married
Married
Total
2.57
2.40
2.08
2.38
2.28
2.11
2.43
2.25
2.27
1.331
1.040
0.783
0.999
0.886
0.798
1.136
0.919
0.948
47
434
319
387
46
365
93
706
799
Tests of Between-Subjects Effects Dependent Variable: Q.28 How good a job have you done balancing your job, your (IF MARITAL = 1: marriage/IF MARITAL = 2: partnership) and being a parent? Would you say excellent, very good, good, only fair, or poor?
Source df FMean Square Sig. Partial Eta Squared
1782.957
19.990a
7.072
.000
3.163
696.693
716.683
4826.000
Intercept
Corrected Model
sex
sex * mar_status
mar_status
Error
Total
Corrected Total
a. R Squared = 0.028 (Adjusted R Squared = 0.024)
1
1
1
3
1
795
798
799
1782.957
7.072
3.163
6.663
0.000
0.876
2034.541
8.070
3.609
7.604
0.000
0.000
0.005
0.058
0.000
0.987
0.719
0.010
0.005
0.028
0.000
Type III Sum of Squares
suk85842_08_c08.indd 301 10/23/13 1:41 PM
CHAPTER 8Section 8.5 Presenting Results
Figure 8.7: SPSS output graph for the 2 3 2 ANOVA
Source: Data from Pew Research: Social and Demographic Trends. (2010). Changing American family. Retrieved from http://www .pewsocialtrends.org/category/datasets/
SPSS Example 2: Steps for a Complex ANOVA
Next, we will take on a more complex ANOVA example, again using public data from Pew Research (2010) data set, the Changing American Family. Keep in mind that when working with real-world data, there may be issues of missing values and small sample sizes. The current examples demonstrate that these issues may arise and that researchers need to be realistic about the type of design they can perform, as well as the statistical con- clusions they can draw based on the quality of the data set. That said, the design using this data is a 2 3 5 3 7 factorial design that will be conducted using gender (sex), age range (ls2), and marital status (marital). Here sex has two levels, ls2 has five levels, and marital has seven levels. We will perform an analysis of these three factors on participants’ quality of life (Please tell me how satisfied you are with your life overall).
suk85842_08_c08.indd 302 10/23/13 1:41 PM
CHAPTER 8Section 8.5 Presenting Results
To perform the analysis, go to Analyze S General Linear Model S Univariate. Input ls2, sex, and marital into the Fixed Factor(s) box and q28 into the Dependent Variables box. On the right click on Plots and move sex into Separate Lines and marital into Horizontal axis; then click Add and Continue. Then, move sex into Separate Lines and move ls2 into Horizontal axis (your screen should look like that in Figure 8.8). Then click on Post Hoc and move marital and ls2 into Post Hoc Tests for; check Tukey (your screen should look like that in Figure 8.9). Click Continue. Then click Options and check Descriptive statistics and Estimates of effect size. Finally, click Continue and OK. Analysis results are presented in Figures 8.10, 8.11, and 8.12.
Figure 8.8: SPSS steps in a factorial ANOVA design
Source: Data from Pew Research: Social and Demographic Trends. (2010). Changing American family. Retrieved from http://www .pewsocialtrends.org/category/datasets/
suk85842_08_c08.indd 303 10/23/13 1:41 PM
CHAPTER 8Section 8.5 Presenting Results
Figure 8.9: SPSS steps in a factorial ANOVA design
Source: Data from Pew Research: Social and Demographic Trends. (2010). Changing American family. Retrieved from http://www .pewsocialtrends.org/category/datasets/
Figure 8.10: SPSS output in a factorial ANOVA design
Between-Subjects Factors
LS2. To make sure that our survey includes many different kinds of people, I need to ask your age. Just stop me when I get to the category that includes your age, as of your last birthday. Are you… (READ)
MARITAL Are you currently married, living with a partner, divorced, separated, widowed, or have you never been married?
SEX [ENTER RESPONDENT'S SEX:]
Value Label N
1
3
2
1
2
1
3
2
4
6
5
9
30 to 49
18 to 29
50 to 64, OR
Female
Male
Living with a partner
Married
Divorced
Widowed
Separated
Never been married
Don’t know/ refused (VOL.)
396
526
134
653
403
32
19
200
3
34
125
643
suk85842_08_c08.indd 304 10/23/13 1:41 PM
CHAPTER 8Section 8.5 Presenting Results
Figure 8.10: SPSS output in a factorial ANOVA design (continued)
Descriptive Statistics Dependent Variable: Q.1 First, please tell me how satisfied you are with your life overall— would you say you are [READ IN ORDER]
Mean Std. Deviation
N
Male
18 to 29
Total
Female
Male
30 to 49
Total
Female
LS2. To make sure that our survey includes many different kinds of people, I need to ask your age. Just stop me when I get to the category that includes your age, as of your last birthday. Are you… (READ)
SEX [ENTER RESPONDENT'S SEX:]
MARITAL Are you currently married, living with a partner, divorced, separated, widowed, or have you never been married?
Married 0.527 101.50 Living with a partner 0.577 41.50 Separated 0.000 11.00 Never been married 0.834 381.82 Total 0.769 531.72 Married 0.583 261.50 Living with a partner 0.535 71.57 Divorced 0.707 21.50 Separated 0.000 11.00 Never been married 1.215 451.98 Total 1.003 811.77 Married 0.561 361.50 Living with a partner 0.522 111.55 Divorced 0.707 21.50 Separated 0.000 21.00 Never been married 1.055 831.90 Total 0.915 1341.75 Married 0.583 1101.56 Living with a partner 0.577 41.50 Divorced 0.632 112.00 Separated 0.577 31.67 Widowed 0.000 12.00 Never been married 0.759 291.83 Total 0.629 1581.65 Married 0.539 1521.48 Living with a partner 2.406 102.70 Divorced 0.701 281.75 Separated 3.391 53.00 Widowed 1.225 52.00 Never been married 1.382 372.08 Don’t know/Refused (VOL.)
3.00 0.000 1
Total 1.71 1.054 238 Married 1.52 0.559 262 Living with a partner 2.36 2.098 14 Divorced 1.82 0.683 39 Separated 2.50 2.673 8 Widowed 2.00 1.095 6 Never been married 1.97 1.150 66 Don’t know/Refused (VOL.)
3.00 0.000 1
Total 1.68 0.908 396
suk85842_08_c08.indd 305 10/23/13 1:41 PM
CHAPTER 8Section 8.5 Presenting Results
Figure 8.10: SPSS output in a factorial ANOVA design (continued)
Descriptive Statistics (continued) Dependent Variable: Q.1 First, please tell me how satisfied you are with your life overall— would you say you are [READ IN ORDER]
Mean Std. Deviation
N
Male
50 to 64, OR
Total
Female
LS2. To make sure that our survey includes many different kinds of people, I need to ask your age. Just stop me when I get to the category that includes your age, as of your last birthday. Are you… (READ)
SEX [ENTER RESPONDENT'S SEX:]
MARITAL Are you currently married, living with a partner, divorced, separated, widowed, or have you never been married?
Married 0.919 1341.66 Living with a partner 1.069 71.86 Divorced 2.035 262.69 Separated 1.000 42.50 Widowed 0.000 12.00 Never been married 0.759 201.95 Total 1.171 1921.85 Married 0.759 2111.59 Living with a partner 0.707 22.50 Divorced 0.670 581.72 Separated 0.837 51.80 Widowed 1.645 252.04 Never been married 0.845 311.77 Don’t know/Refused (VOL.)
1.00 0.000 2
Total 1.67 0.856 334 Married 1.61 0.824 345 Living with a partner 2.00 1.000 9 Divorced 2.02 1.326 84 Separated 2.11 0.928 9 Widowed 2.04 1.612 26 Never been married 1.84 0.809 51 Don’t know/Refused (VOL.)
1.00 0.000 2
Total 1.74 0.986 526
Male
Total
Total
Female
Married 0.776 2541.61 Living with a partner 0.816 151.67 Divorced 1.758 372.49 Separated 0.926 82.00 Widowed 0.000 22.00 Never been married 0.785 871.85 Total 0.945 4031.75 Married 0.671 3891.54 Living with a partner 1.821 192.26 Divorced 0.673 881.73 Separated 2.328 112.27 Widowed 1.564 302.03 Never been married 1.183 1131.96 Don’t know/Refused (VOL.)
1.67 1.155 3
Total 1.69 0.950 653 Married 1.57 0.715 643 Living with a partner 2.00 1.477 34 Divorced 1.95 1.156 125 Separated 2.16 1.834 19 Widowed 2.03 1.513 32 Never been married 1.91 1.028 200 Don’t know/Refused (VOL.)
1.67 1.155 3
Total 1.72 0.948 1056
suk85842_08_c08.indd 306 10/23/13 1:41 PM
CHAPTER 8Section 8.5 Presenting Results
Figure 8.10: SPSS output in a factorial ANOVA design (continued)
Tests of Between-Subjects Effects Dependent Variable: Q.1 First, please tell me how satisfied you are with your life overall— would you say you are [READ IN ORDER]
Mean SquareSource
Type III Sum of Squares F
Partial Eta SquaredSig.df
1 285.925 336.806 0.000 0.248285.925Intercept
34 2.399 2.826 0.000 0.08681.583aCorrected Model
2 2.178 2.566 0.077 0.0054.357Is2
6 2.545 2.998 0.007 0.01715.270marital
1 0.064 0.075 0.784 0.0000.064sex
10 0.876 1.032 0.414 0.0108.757Is2 * marital
2 1.664 1.961 0.141 0.0043.329Is2 * sex
5 2.205 2.597 0.024 0.01311.025sex * marital
1021 0.849866.757Error
8 1.068 1.258 0.262 0.0108.542Is2 * sex * marital
10564061.000Total
1055948.340Corrected Total
a. R Squared = 0.086 (Adjusted R Squared = 0.056)
suk85842_08_c08.indd 307 10/23/13 1:41 PM
CHAPTER 8Section 8.5 Presenting Results
Figure 8.10: SPSS output in a factorial ANOVA design (continued)
Source: Data from Pew Research: Social and Demographic Trends. (2010). Changing American family. Retrieved from http://www .pewsocialtrends.org/category/datasets/
Multiple Comparisons Dependent Variable: Q.1 First, please tell me how satisfied you are with your life overall— would you say you are [READ IN ORDER] Tukey HSD
Mean Difference
(I-J)
Married
(I) MARITAL Are you currently married, living with a partner, divorced, separated, widowed, or have you never been married?
(J) MARITAL Are you currently married, living with a partner, divorced, separated, widowed, or have you never been married?
95% Confidence Interval
Std. Error
Lower Bound
Upper Bound
Sig.
Living with a partner Divorced Separated Widowed Never been married Don’t know/Refused (VOL.)
Living with a partner
Married Divorced Separated Widowed Never been married Don’t know/Refused (VOL.)
Divorced
Married Living with a partner Separated Widowed Never been married Don’t know/Refused (VOL.)
Separated
Married Living with a partner Divorced Widowed Never been married Don’t know/Refused (VOL.)
Widowed
Married Living with a partner Divorced Separated Never been married Don’t know/Refused (VOL.)
Never been married
Married Living with a partner Divorced Separated Widowed Don’t know/Refused (VOL.)
Don’t know/Refused (VOL.)
Married Living with a partner Divorced Separated Widowed Never been married
�0.43 �0.38* �0.59 �0.46 �0.34*
�0.10
0.43 0.05
�0.16 �0.03
0.09
0.33
0.38* �0.05 �0.21 �0.08
0.04
0.29
0.59 0.16 0.21 0.13 0.25
0.49
0.46 0.03 0.08
�0.13 0.12
0.36
0.34* �0.09 �0.04 �0.25 �0.12
0.24
0.10 �0.33 �0.29 �0.49 �0.36 �0.24
0.162 0.090 0.214 0.167 0.075
0.533
0.162 0.178 0.264 0.227 0.171
0.555
0.090 0.178 0.227 0.183 0.105
0.538
0.214 0.264 0.227 0.267 0.221
0.572
0.167 0.227 0.183 0.267 0.175
0.556
0.075 0.171 0.105 0.221 0.175
0.536
0.533 0.555 0.538 0.572 0.556 0.536
0.108 0.000 0.087 0.081 0.000
1.000
0.108 1.000 0.997 1.000 0.998
0.997
0.000 1.000 0.971 0.999 1.000
0.998
0.087 0.997 0.971 0.999 0.922
0.978
0.081 1.000 0.999 0.999 0.993
0.995
0.000 0.998 1.000 0.922 0.993
0.999
1.000 0.997 0.998 0.978 0.995 0.999
�0.91 �0.65 �1.22 �0.96 �0.56
�1.67
�0.05 �0.48 �0.94 �0.70 �0.41
�1.31
0.12 �0.57 �0.88 �0.62 �0.27
�1.30
�0.04 �0.62 �0.46 �0.66 �0.41
�1.20
�0.03 �0.64 �0.46 �0.91 �0.40
�1.28
0.12 �0.59 �0.35 �0.90 �0.64
�1.34
�1.48 �1.97 �1.88 �2.18 �2.01 �1.83
0.05 �0.12
0.04 0.03
�0.12
1.48
0.91 0.57 0.62 0.64 0.59
1.97
0.65 0.48 0.46 0.46 0.35
1.88
1.22 0.94 0.88 0.91 0.90
2.18
0.96 0.70 0.62 0.66 0.64
2.01
0.56 0.41 0.27 0.41 0.40
1.83
1.67 1.31 1.30 1.20 1.28 1.34
Based on observed means. The error term is Mean Square(Error) = 0.849. * The mean difference is significant at the 0.05 level.
suk85842_08_c08.indd 308 10/23/13 1:41 PM
CHAPTER 8Section 8.5 Presenting Results
Figure 8.11: SPSS output graph in a factorial ANOVA design
Source: Data from Pew Research: Social and Demographic Trends. (2010). Changing American family. Retrieved from http://www .pewsocialtrends.org/category/datasets/
Figure 8.12: SPSS output graph in a factorial ANOVA design
Source: Data from Pew Research: Social and Demographic Trends. (2010). Changing American family. Retrieved from http://www .pewsocialtrends.org/category/datasets/
suk85842_08_c08.indd 309 10/23/13 1:41 PM
CHAPTER 8Section 8.5 Presenting Results
SPSS Example 3: Steps for a Mixed-Factorial ANOVA
The last example is a mixed-factorial ANOVA where a researcher would like to compare groups in a longitudinal study. As discussed earlier in the chapter, this involves both a between-group and within-group design simultaneously. Using the data set available from the Statistical Consulting Group at the University of California Los Angeles, Stress_ Data_dataset (Figure 8.13), an occupational therapist wishes to evaluate the effectiveness of two stress treatments over time. After each stress treatment the therapist gives a reliable and valid stress test to her patients that indicates higher stress based on a larger numerical value. In addition, she has a control group that is not going through any stress treatments.
Based on this scenario and the dataset provided, this will be a 3 3 3 mixed-ANOVA design with three measurement times of patients stress levels (measurement_time) and compari- son of the three groups (treatment_1, treatment_2, and control).
• Go to Analyze S General Linear Model. • Click Repeated Measures. • Type in Measurement_Time in the Within-Subject Factor Name box, 3 in the
Number of Levels box, and Stress_Level in the Measure Name box (your screen should look like that in Figure 8.14).
• Click Define. • As shown in Figure 8.15, put in the three measurement times (Time_1, Time_2,
and Time_3) in simultaneous order in the Within-Subjects Variables box. • Move group into the Between-Subjects Factor(s) box (your screen should look
like that in Figure 8.15). • On the right, click Plots and move Measurement_Time into the Horizontal axis. • Click Add. • Do the same for group, and then put Measurement_Time into the Horizontal
axis, and group into Separate Lines (your screen should look like that in Figure 8.16).
• Click Add. • Click Options and move Measurement_Time into Display Means for. • Click Compare Main Effects and select Sidak from the dropdown box just below. • Click on Post Hoc and move group into the Post Hoc Tests for and check Dun-
nett and Dunnett’s C (a commonly used post hoc test when comparing treatment groups to control groups).
• Click Options. • Click Descriptive statistics, Estimates of effect size, and Homogeneity tests. • Click Continue. • Click OK. Analysis results are presented in Figures 8.17, 8.18, 8.19, and 8.20.
suk85842_08_c08.indd 310 10/23/13 1:41 PM
CHAPTER 8Section 8.5 Presenting Results
Figure 8.13: SPSS steps for a mixed-factorial design
Source: Based on data from Introduction to SAS (2007). UCLA: Statistical Consulting Group. Retrieved from http://www.ats.ucla.edu /stat/spss/dae/manova1.htm
Figure 8.14: SPSS steps for a mixed-factorial design
Source: Based on data from Introduction to SAS (2007). UCLA: Statistical Consulting Group. Retrieved from http://www.ats.ucla.edu /stat/spss/dae/manova1.htm
suk85842_08_c08.indd 311 10/23/13 1:41 PM
CHAPTER 8Section 8.5 Presenting Results
Figure 8.15: SPSS steps for a mixed-factorial design
Source: Based on data from Introduction to SAS (2007). UCLA: Statistical Consulting Group. Retrieved from http://www.ats.ucla.edu /stat/spss/dae/manova1.htm
Figure 8.16: SPSS steps for a mixed-factorial design
Source: Based on data from Introduction to SAS (2007). UCLA: Statistical Consulting Group. Retrieved from http://www.ats.ucla.edu /stat/spss/dae/manova1.htm
suk85842_08_c08.indd 312 10/23/13 1:41 PM
CHAPTER 8Section 8.5 Presenting Results
Figure 8.17: SPSS output for a mixed-factorial design
Between-Subject Factors
Value Label N
treatment 2
treatment 1
control
11
11
11
group
1.00
2.00
3.00
Descriptive Statistics
Mean Std. Deviation N
treatment_2
treatment_1
control
11
11
11
group
Time_1
18.1182
15.5273
15.3455
3.90380
2.07562
3.13827
Total 3316.3303 3.29246
treatment_2
treatment_1
control
11
11
11 Time_2
6.1909
5.5818
5.3727
1.89971
2.43426
1.75903
Total 335.7152 2.01760
treatment_2
treatment_1
control
11
11
11 Time_3
8.6818
5.1091
5.6364
4.86309
2.53119
3.54691
Total 336.4758 3.98513
Mauchly’s Test of Sphericitya Measure: StressLevel
Mauchly’s WWithin Subject Effect
Approx. Chi-Square
0.750 0.500Measurement_Time
Tests the null hypothesis that the error covariance matrix of the orthonormalized transformed dependent variables is proportional to an identity matrix. a. Design: Intercept + group Within Subjects Design: Measurement_Time b. May be used to adjust the degrees of freedom for the averaged tests of significance. Corrected tests are displayed in the Tests of Within-Subjects Effects table.
0.532 18.298 2 0.000 0.681
Epsilonbdf Sig.
Greenhouse- Geisser
Huynh- Feldt
Lower Bound
suk85842_08_c08.indd 313 10/23/13 1:41 PM
CHAPTER 8Section 8.5 Presenting Results
Figure 8.17: SPSS output for a mixed-factorial design (continued)
Tests of Within-Subjects Effects Measure: StressLevel
Source
Measurement_Time
Measurement_Time * group
Error(Measurement_Time)
F
102.730
102.730
102.730
102.730
0.640
0.640
0.640
0.640
Sig.
0.000
0.000
0.000
0.000
0.636
0.579
0.594
0.535
Partial Eta
Squared
0.774
0.774
0.774
0.774
0.041
0.041
0.041
0.041
Type III Sum of Squares
2314.092
2314.092
2314.092
2314.092
28.815
28.815
28.815
28.815
675.776
675.776
675.776
675.776
df
2
1.362
1.500
1.000
4
2.725
3.000
2.000
60
40.874
45.005
30.000
Mean Square
1157.046
1698.457
1542.544
2314.092
7.204
10.575
9.604
14.407
11.263
16.533
15.015
22.526
Sphericity Assumed
Greenhouse-Geisser
Huynh-Feldt
Lower-Bound
Sphericity Assumed
Greenhouse-Geisser
Huynh-Feldt
Lower-Bound
Sphericity Assumed
Greenhouse-Geisser
Huynh-Feldt
Lower-Bound
Tests of Within-Subjects Contrasts Measure: StressLevel
Source
Measurement_Time
Measurement_Time
Measurement_Time * group
Error(Measurement_Time)
F
84.639
198.017
0.075
3.615
Sig.
0.000
0.000
0.928
0.039
Partial Eta
Squared
0.738
0.868
0.005
0.194
Type III Sum of Squares
1602.349
711.743
2.825
25.989
567.945
107.831
df
1
1
2
2
30
30
Mean Square
1602.349
711.743
1.413
12.995
18.932
3.594
Linear
Quadratic
Linear
Quadratic
Linear
Quadratic
Tests of Within-Subjects Effects Measure: StressLevel Transformed Variable: Average
Source
Intercept
group
Error
F
1571.235
9.650
Sig.
0.000
0.001
Partial Eta
Squared
0.981
0.391
Type III Sum of Squares
8948.055
109.914
170.848
df
1
2
30
Mean Square
8948.055
54.957
5.695
suk85842_08_c08.indd 314 10/23/13 1:41 PM
CHAPTER 8Section 8.5 Presenting Results
Figure 8.17: SPSS output for a mixed-factorial design (continued)
Source: Based on data from Introduction to SAS (2007). UCLA: Statistical Consulting Group. Retrieved from http://www.ats.ucla.edu /stat/spss/dae/manova1.htm
Figure 8.18: SPSS output graph in a mixed-factorial design
Source: Based on data from Introduction to SAS (2007). UCLA: Statistical Consulting Group. Retrieved from http://www.ats.ucla.edu/ stat/spss/dae/manova1.htm
Multiple Comparisons Measure: StressLevel
(I) group (J) group Mean Difference
(I-J)
Std. Error Sig.
0.001
0.996
0.56986
0.60904
0.56986
0.58290
0.60904
0.58290
0.58749
0.58749
2.2576*
2.2121*
�2.2576*
�0.0455
�2.2121*
0.0455
2.2121*
�0.0455
treatment_2
control
treatment_1
control
treatment_1
treatment_2
control
control
treatment_1
treatment_2
treatment 1
treatment 2
control
Dunnett C
Dunnett t (2-sided)b
95% Confidence Interval
0.6954
0.5426
�3.8197
�1.6434
�3.8817
�1.5524
0.8487
�1.4089
Lower Bound
3.8197
3.8817
�0.6954
1.5524
�0.5426
1.6434
3.5755
1.3180
Upper Bound
Based on observed means. The error term is Mean Square(Error) = 1.898. *. The mean difference is significant at the .05 level. b. Dunnett t-tests treat one group as a control, and compare all other groups against it.
suk85842_08_c08.indd 315 10/23/13 1:41 PM
CHAPTER 8Section 8.5 Presenting Results
Figure 8.19: SPSS output graph in a mixed-factorial design
Source: Based on data from Introduction to SAS (2007). UCLA: Statistical Consulting Group. Retrieved from http://www.ats.ucla.edu /stat/spss/dae/manova1.htm
Figure 8.20: SPSS output graph in a mixed-factorial design
Source: Based on data from Introduction to SAS (2007). UCLA: Statistical Consulting Group. Retrieved from http://www.ats.ucla.edu /stat/spss/dae/manova1.htm
suk85842_08_c08.indd 316 10/23/13 1:41 PM
CHAPTER 8Section 8.6 Interpreting Results
8.6 Interpreting Results
Refer to the most recent edition of the APA manual for specific detail on formatting statistics; Table 8.3 may be used as a quick guide in presenting the statistics covered in this chapter.
Table 8.3: Guide to APA formatting of F statistic results
Abbreviation or term Description
F F test statistic score
Partial-h2 An effect size based on part of the factorial design for one factor
W Mauchly’s test of sphericity
SS Sum of squares
MS Mean square
e Epsilon
M Box’s test of equality of covariance matrices
Source: Publication Manual of the American Psychological Association, 6th edition. © 2009 American Psychological Association, pp. 119–122.
Using the results from SPSS Example 1 of the 2 3 2 ANOVA, we present the results (Figure 8.6), in the following way:
• There is a significant main effect of sex on work balance F(1, 795) 5 8.070, p , .05, partial-h2 5 .01, specifically males (M 5 2.40, SD 5 1.04) reported a significantly higher mean than females’ scores (M 5 2.11, SD 5 .798) in regards to work balance.
• There is no significant main effect of marital status on work balance F(1, 795) 5 3.609, p 5 .058, partial-h2 5 .005, specifically married (M 5 2.25, SD 5 .919) reported no significant differences than never married (M 5 2.11, SD 5 .798) in regards to work balance.
• There is no interaction effect between sex and marital status in regards to the work balance F(1, 795) 5 .000, p 5 .987, partial-h2 5 .000. This is also corrobo- rated in the interaction line graph, shown in Figure 8.7, where both gender lines are parallel to each other across marital status on the x-axis.
Using the results from SPSS Example 2 of the 2 3 5 3 7 ANOVA, we present the results (Figure 8.10), in the following way:
• There is no significant main effect of age on quality of life F(2, 1021) 5 2.566, p 5 .077, partial-h2 5 .005
• There is no significant main effect of sex on quality of life F(1, 1021) 5 .075, p 5 .784, partial-h2 5 .000, specifically males (M 5 1.95, SD 5 1.156) reported a significantly higher mean than females (M 5 1.69, SD 5 .940) in regards to qual- ity of life.
• There is a significant main effect of marital status on quality of life F(1, 1021) 5 2.998, p 5 .007, partial-h2 5 .017, specifically divorce scores (M 5 1.95, SD 5 .940) reported a significantly higher mean than married (M 5 1.57, SD 5 .715) in regards to quality of life. Additionally never been married scores
suk85842_08_c08.indd 317 10/23/13 1:41 PM
CHAPTER 8Section 8.7 Other Factorial Designs
(M 5 1.91, SD 5 1.028) reported a significantly higher mean than married (M 5 1.57, SD 5 .715).
• There is no interaction effect between the age and sex on quality of life F(1, 1021) 5 1.961, p 5 .141, partial-h2 5 .004.
• There is no interaction effect between the age and marital on quality of life F(1, 1021) 5 1.032, p 5 .414, partial-h2 5 .010.
• There is an interaction effect between the sex and marital on quality of life F(1, 1021) 5 2.597, p , .05, partial-h2 5 .013. This is corroborated in the interac- tion line graph, shown in Figure 8.12, where both gender lines are nonparallel to each other across marital status on the x-axis. Looking at this graph a bit closer, you can see substantially lower quality of life scores for males living with a part- ner than women. The opposite is true for divorced males whose scores represent a higher quality of life than divorced females.
• There is no higher-order interaction effect between the age, sex, and marital on quality of life F(1, 1021) 5 1.258, p 5 .262, partial-h2 5 .010.
Using the results from SPSS Example 3, Figure 8.17, we could present the results in the following way:
• According to Mauchly’s test of Sphericity, W 5 .532, x2 5 18.30, p , .05; there- fore, a violation of sphericity has occurred. Therefore, using the Huynh-Feldt, the e 5 .750 adjustment, there is a significant difference in measurement times of stress within groups F(1.5, 45.00) 5 102.73, p , .05, partial-h2 5 .774, specifi- cally Time_1 (M 5 16.33) is significantly different from Time_2 (M 5 5.715) and Time_3 (M 5 6.476).
• There were also overall between-group differences, F(2, 30) 5 9.65, p , .05, partial-h2 5 .391, specifically Treatment_1 (M 5 10.99) was significantly different than Treatment_2 (M 5 8.74) and the control group (M 5 8.78).
• There were no interaction effects for the independent variables, F(3, 45.00) 5 .640, p , .535, partial-h2 5 .041.
8.7 Other Factorial Designs
As mentioned, the addition of variables to factorial designs will become more intri-cate, yet interesting with various interactions in the model, which is usually the core interest of researchers. This makes the analysis of theoretical models, involving multiple variables that may include between-, within-, and mixed-designs, possible. Some of these more complex designs, found in advanced (multivariate) statistical textbooks, are worth mentioning for your own knowledge, understanding, and perhaps a general love of sta- tistical techniques.
Multivariate Analysis of Variance (MANOVA)
Building upon factorial designs discussed thus far, we have started with one-way, two-way, and repeated-measures ANOVAs, and by now it should be clear at this point that there is a commonality among all of these in the use of only one dependent vari- able. That said, factorial designs could become even more complex with the addition
suk85842_08_c08.indd 318 10/23/13 1:41 PM
CHAPTER 8Section 8.7 Other Factorial Designs
of multiple DVs, which are known as a multivariate analysis of variance (MANOVA). The use of multiple DVs adds to increased complexity in regards to interaction effects and the use of multiple factors acting together accounting for systematic variance. As such, the MANOVA is a way to control for family-wise rate error (FWER) as discussed in Chapter 6, where, as in this case, running several separate ANOVAs will inflate the type I error. To combat this issue, the omnibus MANOVA test is used with consequential follow- up ANOVAs and post hoc tests.
When executing the MANOVA—as seen in Figure 8.21—the general modus operandi is to perform the overall MANOVA first with all of the variables (IVs and DVs) tested together, then followed by subsequent ANOVAs, where each of the DVs are tested separately. Con- sequently, if these ANOVAs are significant, then post hoc tests are involved to identify specific mean difference. If this is done by manual calculations, it will take a painfully long time, but with the use of software such as SPSS, R, or SAS, it will take just a few minutes to execute. That said, MANOVAs are quite commonly used for dissertations and reported in journal articles when testing theoretical models that involve many variables. To reiterate, the elegance of the MANOVA is that the variables can be testing simultane- ously to explore main and interaction effects using multiple IVs and DVs.
Figure 8.21: General modus operandi for executing the MANOVA
Analysis of Covariance (ANCOVA) and Multivariate Analysis of Covariance (MANCOVA)
This section on multivariate analysis takes a different approach to the addition of vari- ables. As background context, commonly in a laboratory setting as in biology, chemistry, and physics labs, control of unforeseen or nuisance variables that can affect the outcome of the experiment is imperative and standard in these disciplines. By controlling the envi- ronment (e.g., the lights, sounds, equipment), the experimenter will eliminate chances of these variables affecting the outcome. This control, which will ultimately lead to what is called internal validity, can also be consistent across experiments and with each partici- pant. Subsequently, high internal validity will lead to a strong conclusion on a factor’s influence on a DV that is called statistical conclusion validity.
On the other hand, psychological and business research involves collecting data in a field setting as in an organization, on the street, or in some other public venue. Such method- ologies make it very difficult and almost impossible to control all environmental factors that can affect the field experiment. Because of inherent limitations, known as nuisance variables or confounding variables, internal validity that can affect the IV-DV relation- ship and hence the statistical conclusion validity is compromised. To combat these extra variables, the researcher must think about what these variables are and then how to mea- sure them. If they are measured, they can be added to the analysis as a covariate to gauge
MANOVA ANOVA Post hoc test
suk85842_08_c08.indd 319 10/23/13 1:41 PM
CHAPTER 8Key Terms
their level of influence on the IV-DV relationship. One of the common ways to do this is by running an analysis of covariance (ANCOVA). By being knowledgeable about con- founding influences or covariates, the researcher can decide whether they may have had a covarying influence with the IV on the DV. If these covariates are discovered to have an influence, then limitations of the study are duly reported as best-practice research.
As in other types of ANOVAs, including ANCOVA, the commonality is one DV. Conse- quently, if there are multiple DVs, then a multiple analysis of covariance (MANCOVA) may be used.
Summary The chapter provided a wrap-up of the comparison of between-group designs that started with t-tests in Chapter 5 to ANOVAs in Chapter 6; and then comparison of groups over time and treatments, as in within-groups designs from Chapter 7. The current chapter explored factorial designs including the analysis of multiple IVs on a DV as well as mixed designs that are both between-group and within-group designs performed simultaneously (Objective 1). Breakdowns of the sum of squares variances were shown for both types of designs as a way to compare and contrast both models (Objectives 2, 3, and 5). Main and interaction effects were explained as an important component to factorial designs using multiple variables (Objective 4). New methods of effect sizes such as partial-h2 (Objective 6) were discussed and calculated. In addition, we presented the SPSS steps of these analyses, interpreted, and reported the results in proper APA formatting (Objectives 7 and 8). We ended the chapter with a brief overview of even more complex multivariate designs that include ANCOVA, MANOVA, and MANCOVA (Objective 9).
Key Terms
analysis of covariance (ANCOVA) The analysis used for the influence of a factor (IV) or multiple factors on a single depen- dent variable while exploring the influence of covariate variables and their influence on the IV-DV relationship.
covariate A distinction used for a third variable that covaries with the independent variable to affect the dependent variable.
epsilon (E) A measure of sphericity that ranges from 0 to 1 with 1 being the maxi- mum level. Values of e , 1 may lead to a violation of sphericity that is detected using the Mauchly’s W-test.
factorial designs Research design involv- ing one (one-way ANOVA) or multiple (e.g., two-way or three-way ANOVA) fac- tors on a dependent variable or multiple dependent variables (MANOVA).
suk85842_08_c08.indd 320 10/23/13 1:41 PM
CHAPTER 8Chapter Exercises
higher-order interaction effect The inter- action of three or more factors on the dependent variable. As more factors are added, more higher-order interactions will occur.
interaction The influence of one factor (IV) on another factor manifested in their com- bined influence on the dependent variable.
main effects The influence of a factor (IV) on the dependent variable in a factorial design.
multivariate analysis of covariance (MANCOVA) The analysis used for the influence of a factor (IV) or multiple fac- tors on multiple dependent variables while exploring the influence of covariate vari- ables and their influence on each IV-DV relationship.
multivariate analysis of variance (MANOVA) The analysis used for the influence of a factor (IV) or multiple factors on a multiple dependent variable.
omega-squared (V2) Measures the overall effects size and is less biased than e2 in its calculation and is always lower in magni- tude than e2.
partial-H2 A measure of effect size where a partial measurement is given for each fac- tor on the dependent variable, that is, each IV-DV effect size for the factorial design. These values cannot be summed to give a total effect size but rather omega-squared (v2) must be calculated.
Chapter Exercises
Review Questions The answers to the odd-numbered items can be found in the answers appendix.
Again, using the data set available from the Statistical Consulting Group at the University of California Los Angeles, Drug_Treatment_dataset (see Figure 8.22), perform a mixed- methods ANOVA. Assume that you as a researcher want to evaluate the effects two treat- ment (Drug_1 and Drug_2) using a Treatment and two Control (control_1 and control_2) groups and answer the following questions.
1. What type of factorial design is this? Be specific (e.g., 2 3 2 factorial design).
2. What are the assumptions prior to running a factorial design?
3. List all the hypotheses for this factorial design.
After performing your factorial design analysis in Excel, SPSS, or other capable software program, answer the following questions.
suk85842_08_c08.indd 321 10/23/13 1:41 PM
CHAPTER 8Chapter Exercises
4. What are the main effect results? Are they significant? How do you know?
5. What are the interaction results? Are they significant? How do you know?
6. Report the effect sizes for each of the main and interaction effects. What do these sizes convey in regards to Cohen’s (1988) values?
7. Based on your analysis, write your overall conclusions in proper APA formatting.
Figure 8.22: Dataset for the Review Questions factorial analysis
Source: Based data from Introduction to SAS (2007). UCLA: Statistical Consulting Group. Retrieved from http://www.ats.ucla.edu/stat /spss/dae/manova1.htm
suk85842_08_c08.indd 322 10/23/13 1:41 PM
CHAPTER 8Chapter Exercises
Analyzing the Research Review the article abstracts provided below. You can then access the full articles via your university’s online library portal to answer the critical thinking questions. Answers can be found in the answers appendix.
Using Factorial Design for a Cognitive Strategies Study
Forys, K. L., & Dahlquist, L. M. (2007). The influence of preferred coping style and cogni- tive strategy on laboratory-induced pain. Health Psychology, 26(1), 22–29.
Article Abstract
Objective: To evaluate the effects of matching an individual’s coping style (low, mixed, or high monitoring) to an appropriate cognitive strategy (distraction or sensation monitor- ing) to improve pain management. Design: This study used a split-plot factorial design in a laboratory setting. Main Outcome Measures: Main outcomes were pain threshold, pain tolerance, pain intensity, pain affect, and anxiety. Results: The results of the 2 3 3 3 3 (Experimental Condition 3 Coping Style 3 Trial) analysis of variance (ANOVA) inter- action were significant for pain threshold scores, F(4, 178) 5 2.95, p , .01. Low moni- tors in the matched distraction trial had higher pain threshold scores than during base- line, t(15) 5 22.68, p , .017, and the mismatched sensation monitoring trial, t(15) 5 2.80, p , .014. High monitors’ pain threshold scores were higher than baseline only during the matched sensation monitoring trial, t(27) 5 22.75, p , .010. The results of the 2 3 3 3 3 ANOVA interaction were not significant for pain tolerance scores; however, when the mixed monitors were excluded, the 3-way interaction was significant, F(2, 124) 5 3.48, p , .05. The results were nonsignificant for pain intensity, pain affect, and anxiety. Conclusion: Results demonstrate that matching coping style to the appropriate cognitive strategy is important for improving pain threshold and pain tolerance; however, match- ing did not reduce pain intensity, pain affect, or anxiety. Future studies should explore the explanation for differential responses of high and low monitors and should test these hypotheses in a clinical setting.
Critical Thinking Questions
1. What is the factorial design in this study?
2. How many independent variables are used in this study?
3. Why does this study use a factorial analysis of variance?
4. In the study, it was determined there was a significant Coping Style 3 Experimental Condition 3 Trial interaction for pain threshold scores. What does this interaction show?
Using Mixed-Factorial ANOVA for a Performance Patterns Study
Oliver, R., & Williams, R. L. (2006). Performance patterns of high, medium, and low per- formers during and following a reward versus non-reward contingency phase. School Psychology Quarterly, 21(2), 119–147.
suk85842_08_c08.indd 323 10/23/13 1:41 PM
CHAPTER 8Chapter Exercises
Article Abstract
Three contingency conditions were applied to the math performance of 4th and 5th grad- ers: bonus credit for accurately solving math problems, bonus credit for completing math problems, and no bonus credit for accurately answering or completing math problems. Mixed ANOVAs were used in tracking the performance of high, medium, and low per- formers during the experimental phase across a mandatory follow-up phase and a choice follow-up phase. The two reward contingencies produced generally higher performance than the non-reward contingency (control condition) in the experimental phase, but all performance levels did better in the mandatory follow-up phase after the non-reward condition than after either reward contingency. Plus, high performers did substantially better in the choice phase following a non-reward contingency than following either reward contingency, most especially following the accuracy contingency. The pattern of results generally points to an overjustification effect for contingent bonus credit, with this effect more attributable to a perception of control than a perception of competency.
Critical Thinking Questions
1. What are the independent variables used in the study that made them choose to run a mixed-factorial ANOVA?
2. If the study was worried about violating the assumption of sphericity, what test should be run to test for this?
3. If there were a main effect, what would this mean? Rewrite the nonsignificant main effect to make it significant. F(5, 68) 5 .87, p 5 .51.
suk85842_08_c08.indd 324 10/23/13 1:41 PM
