statistics

Exercise 18

Understanding Analysis of Variance (ANOVA) and Post Hoc Analyses

Statistical Technique in Review

Analysis of variance (ANOVA) statistical technique is conducted to examine differences between two or more groups. There are different types of ANOVAs, with the most basic being the one-way ANOVA, which is used to analyze data in studies with one independent and one dependent variable. More details on the types of ANOVAs can be found in your research textbook and statistical texts (Grove, Burns, & Gray, 2013; Plichta & Kelvin, 2013). The outcome of ANOVA is a numerical value for the F statistic. The calculated F-ratio from ANOVA indicates the extent to which group means differ, taking into account the variability within the groups. Assuming the null hypothesis of no differences among the groups studied is true; the probability of obtaining an F-ratio as large as the obtained value in a given sample is determined by the calculated p value. If the p value is greater than the level of significance, or alpha (α), = 0.05 set for the study, then the study results are nonsignificant and the F-ratio will be less than the critical values for F in the statistical table (see Appendix C Critical Values of F for α = 0.05 and α = 0.01 at the back of this text). With nonsignificant results, researchers will accept the null hypothesis of no significant differences between the groups. In a study, if p = 0.01, this value is less than α = 0.05, which indicates the groups are significantly different and the null hypothesis is rejected. However, there is always a possibility that this decision is in error, and the probability of committing this Type I error is determined by α. When α = 0.05, there are 5 chances in 100 the results are a Type I error, or saying something is significant when it is not.

ANOVA is similar to the t-test because the null hypothesis (no differences between groups) is rejected when the analysis yields a smaller p value, such as p ≤ 0.05, than the α set for the study. Assumptions for the ANOVA statistical technique include the following:

1. The populations from which the samples were drawn or the random samples are normally distributed.

2. The groups should be mutually exclusive.

3. The groups should have equal variance, also known as homogeneity of variance.

4. The observations are independent.

5. The dependent variable is measured at the interval or ratio level (Plichta & Kelvin, 2013; Zar, 2010).

180

Researchers who perform ANOVA on their data record their results in an ANOVA summary table or in the text of a research article. An example of how an ANOVA result is commonly expressed is as follows:

where:

• F is the statistic.

• 2 is the group degrees of freedom (df) calculated by k − 1, where k = number of groups in the study. In this example, k − 1 = 3 − 1 = 2.

• 120 is the error degrees of freedom (df) that is calculated based upon the number of participants, or N − k. In this example, 123 subjects − 3 groups = 120 error df.

• 4.79 is the F-ratio or value.

• p indicates the significance of the F-ratio in this study or p = 0.01.

The simplest ANOVA is the one-way ANOVA, but many of the studies in the literature include more complex ANOVA statistical techniques. A commonly used ANOVA technique is the repeated-measures analysis of variance, which is used to analyze data from studies where the same variable(s) is(are) repeatedly measured over time on a group or groups of subjects. The intent is to determine the change that occurs over time in the dependent variable(s) with exposure to the independent or intervention variable(s).

Post Hoc Analyses Following ANOVA

When a significant F value is obtained from the conduct of ANOVA, additional analyses are needed to determine the specific location of the differences in a study with more than two groups. Post hoc analyses were developed to determine where the differences lie, because some of the groups might be different and others might be similar. For example, a study might include three groups, an experimental group (receiving an intervention), placebo group (receiving a pseudo or false treatment such as a sugar pill in a drug study), and a comparison group (receiving standard care). The ANOVA resulted in a significant F-ratio or value, but post hoc analyses are needed to determine the exact location of the differences. With post hoc analyses, researchers might find that the experimental group is significantly different from both the placebo and comparison groups but that the placebo and comparison groups were not significantly different from each other. One could conduct three t-tests to determine differences among the three groups, but that would inflate the Type I error. A Type I error occurs when the results indicate that two groups are significantly different when, in actuality, the groups are not different. Thus post hoc analyses were developed to detect the differences following ANOVA in studies with more than two groups to prevent an inflation of a Type I error. The frequently used post hoc analyses include the Newman-Keuls test, the Tukey Honestly Significant Difference (HSD) test, the Scheffé test, and the Dunnett test (Plichta & Kelvin, 2013).

With post hoc analyses, the α level is reduced in proportion to the number of additional tests required to locate the statistically significant differences. As the α level is decreased, reaching the level of significance becomes increasingly more difficult. The Newman-Keuls test compares all possible pairs of means and is the most liberal of the post hoc tests discussed here. “Liberal” indicates that the α is not as severely decreased. The Tukey HSD test computes one value with which all means within the data set are compared. It is considered more stringent than the Newman-Keuls test and requires approximately equal sample sizes in each group. The most conservative test is the Scheffé, but with the decrease in Type I error there is an increase in Type II error, which is saying something is not 181significant when it is. The Dunnett test requires a control group, and the experimental groups are compared with the control group without a decrease in α. Exercise 33 provides the step-by-step process for calculating ANOVA and post hoc analyses.

Research Article

Source

Mayland, C. R., Williams, E. M., Addington-Hall, J., Cox, T. F., & Ellershaw, J. E. (2014). Assessing the quality of care for dying patients from the bereaved relatives' perspective: Further validation of “Evaluating Care and Health Outcomes—for the Dying.” Journal of Pain and Symptom Management, 47(4), 687–696.

Introduction

The Liverpool Care Pathway (LCP) for the Dying Patient was created to address the need for better end of life care for both patients and families, which had been identified as an issue in the United Kingdom at the national level. “LCP is an integrated care pathway used in the last days and hours of life that aims to transfer the hospice principles of best practice into the acute hospital and other settings” (Mayland et al., 2014, p. 688). “Evaluating Care and Health Outcomes—for the Dying (ECHO-D) is a post-bereavement questionnaire that assesses quality of care for the dying and is linked with the Liverpool Care Pathway for the Dying Patient (LCP)” (Mayland et al., 2014, p. 687).

The purpose of this comparative descriptive study was to assess the internal consistency reliability, test-retest reliability, and construct validity of the key composite subscales of the ECHO-D scale. The study's convenience sample consisted of 255 next-of-kin or close family members of the patients with an anticipated death from cancer at either the selected hospice or hospital in Liverpool, United Kingdom. The sample consisted of three groups of family members based on where the patients received end of life care; the hospice, which used LCP; the hospital group that also used LCP; and another group from the same hospital that did not use LCP. The ECHO-D questionnaire was completed by all 255 study participants and a subset of self-selected participants completed a second ECHO-D 1 month after the completion of the first ECHO-D. Mayland and colleagues (2014) concluded their study provided additional evidence of reliability and validity for ECHO-D in the assessment of end of life care.

Relevant Study Results

“Overall, hospice participants had the highest scores for all composite scales, and ‘hospital without LCP’ participants had the lowest scores (Tables 2 and 3). The scores for the ‘hospital with LCP’ participants were between these two levels” (Mayland et al., 2014, p. 693). The level of significance was set at 0.05 for the study. One-way analysis of variance was calculated to assess differences among the hospice, hospital with LCP, and hospital without LCP groups. Post hoc testing was conducted with the Tukey HSD test. ANOVA and post hoc results are displayed in Tables 2 and 3.

TABLE 2

COMPARISON OF HOSPICE AND HOSPITAL PARTICIPANTS' SCORES FOR COMPOSITE SCALES WITHIN THE ECHO-D QUESTIONNAIRE

Composite Scale	Mean (SD) Range	ANOVA (p) a	Post Hoc Comparisons Using Tukey HSD Test b
	All Participants (n = 255)	Hospice (n = 109)	Hospital with LCP (n = 78)	Hospital without LCP (n = 68)		Hospice vs. Hospital with LCP	Hospice vs. Hospital without LCP	Hospital with LCP vs. Hospital without LCP
Ward environment	7.3 (2.7) 0–10	9.1 (1.2) 5–10	6.4 (2.6) 0–10	5.4 (2.7) 0–10	60.4 (<0.0001)	<0.0001	<0.0001	0.01
Facilities	7.3 (4.8) 0–18	10.5 (4.0) 2–18	4.5 (3.8) 0–18	4.1 (2.7) 0–18	76.7 (<0.0001)	<0.0001	<0.0001	0.85
Care	18.4 (6.4) 0–25	22.0 (3.75) 7–25	16.8 (0.66) 3–25	14.6 (7.33) 0–25	35.9 (<0.0001)	<0.0001	<0.0001	0.05
Communication	9.8 (3.7) 0–14	11.2 (3.2) 0–14	9.4 (3.5) 0–14	8.2 (3.8) 0–14	16.6 (<0.0001)	0.002	<0.0001	0.86

a One-way ANOVA (between-groups ANOVA with planned comparisons).

b Post hoc comparisons allow further exploration of the differences between individual groups using the Tukey HSD test, which assumes equal variances for the groups.

ECHO-D = Evaluating Care and Health Outcomes for the Dying; ANOVA = analysis of variance; HSD = honestly significant difference; LCP = Liverpool Care Pathway for the Dying Patient.

Mayland, C. R., Williams, E. M., Addington-Hall, J., Cox, T. F., & Ellershaw, J. E. (2014). Assessing the quality of care for dying patients from the bereaved relatives' perspective: Further validation of “Evaluating Care and Health Outcomes-for the Dying.” Journal of Pain and Symptom Management, 47(4), p. 691.

TABLE 3

COMPARISON OF HOSPICE AND HOSPITAL PARTICIPANTS' SCORES FOR COMPOSITE VARIABLES WITHIN THE ECHO-D QUESTIONNAIRE

Composite Variable	Mean (Range)	ANOVA (p)	Post Hoc Comparisons Using Tukey HSD Test
	All Participants (n = 255)	Hospice (n = 109)	Hospital with LCP (n = 78)	Hospital without LCP (n = 68)		Hospice vs. Hospital with LCP	Hospice vs. Hospital without LCP	Hospital with LCP vs. Hospital without LCP
Symptom Control Degree of affliction from symptoms commonly associated with dying patients: pain, restlessness, respiratory tract secretions, nausea and/or vomiting, and breathlessness. Scores range from 0 (all five symptoms present all of the time) to 10 (no symptoms present).	6.8 (0–10)	7.0 (0–10)	7.0 (2–10)	6.1 (1–10)	4.4 (0.01)	0.99	0.02	0.03
Symptom Management Reflecting whether more should have been done by staff to control symptoms. Scores range from 0 (not enough done by staff to control symptoms) to 6 (staff did all they could to control symptoms).	4.8 (0–6)	5.2 (2–6)	4.8 (0–6)	4.2 (0–6)	10.6 (<0.0001)	0.17	<0.0001	0.02
Spiritual Need—Patient Reflecting whether patients' spiritual and religious needs were met. Scores range from 0 (where need was not met at all) to 6 (where needs were extremely well met).	3.0 (1.9)	3.9 (0–6)	2.9 (0–6)	1.6 (0–6)	38.1 (<0.0001)	0.0001	0.0001	0.0001
Spiritual Need—Next-of-Kin Reflecting whether relatives' religious and spiritual needs were met. Scores range from 0 (where need was not met at all) to 7 (where needs were extremely well met).	2.7 (0–7)	3.5 (0–7)	2.6 (0–7)	1.5 (0–7)	22.6 (<0.0001)	0.006	0.0001	0.002

ECHO-D = Evaluating Care and Health Outcomes for the Dying; ANOVA = analysis of variance; HSD = Honestly Significant Difference; LCP = Liverpool Care Pathway for the Dying Patient.

Mayland, C. R., Williams, E. M., Addington-Hall, J., Cox, T. F., & Ellershaw, J. E. (2014). Assessing the quality of care for dying patients from the bereaved relatives' perspective: Further validation of “Evaluating Care and Health Outcomes-for the Dying.” Journal of Pain and Symptom Management, 47(4), p. 692.

Study Questions

1. What type of analysis was conducted in this study to examine group differences? What three groups were analyzed for differences?

2. What did the researcher set the level of significance, or alpha (α), at for this study?

3. State the null hypothesis for communication for the three groups. Should this null hypothesis be accepted or rejected? Provide a rationale for your answer.

4. What is the purpose of conducting post hoc analysis?

5. Identify the post hoc results for communication on Table 2 . Which results are statistically significant? What do these results mean?

6. What variable on Table 2 has the result F = 60.4 (p < 0.0001)? What does this result mean?

185

7. Mayland et al. (2014) reported means, standard deviations, and range on Table 2 and Table 3 . In your opinion, is this information helpful? Provide a rationale for your answer with documentation.

8. What is the F for spiritual need—next-of-kin? Is this result statistically significant? Provide a rationale for your answer.

9. What are the post hoc results for spiritual need—next-of-kin? Which results are statistically significant? What do the results mean?

10. Mayland et al. (2014) chose the dying patients' next-of-kin rather than the patients themselves as study participants to assess end of life care. In your opinion, was this an appropriate choice?

186

Answers to Study Questions

1. One-way analysis of variance (ANOVA) was used to analyze the differences among the three groups. The three groups analyzed were hospice with Liverpool care pathway for the dying patient (LCP), hospital with LCP, and hospital without LCP. Although not identified in Tables 2 and 3 in this study, the hospice setting did include the LCP. For consistency, the following answers will refer to just hospice as in the Tables 2 and 3 .

2. The level of significance or alpha (α) for this study was set at 0.05.

3. The null hypothesis is: The three care groups of hospice, hospital with LCP, and hospital without LCP have no difference in communication scores for the next-of-kin of patients who had died of cancer. According to Table 2, F = 16.6, p < 0.0001, for composite scale communication. This F value is statistically significant because the p value is less than α = 0.05 that was set for this study. The significant result means that there was a statistically significant difference among the three groups for communication; therefore, the null hypothesis was rejected.

4. A statistically significant F value for ANOVA indicates that a difference is present among the groups analyzed but post hoc testing is necessary to identify the location(s) of the differences when a study has more than two groups. Mayland et al. (2014 , p. 691) explains the purpose of post hoc testing in the footnotes of Table 2 as “Post hoc comparisons allow further exploration of the differences between individual groups.”

5. The post hoc results for communication are: p = 0.002 for hospice versus hospital with LCP and p = 0.0001 for hospice versus hospital without LCP. These results identify statistically significant differences for communication among these identified groups since the p values are less than α = 0.05. The post hoc result of p = 0.86 for hospital with LCP versus hospital without LCP is >0.05 and is not statistically significant. This result means there is no significant difference between the hospital with LCP and the hospital without LCP groups in terms of communication.

6. Ward environment is the variable on Table 2 that has the result F = 60.4 (p < 0.0001). This statistically significant result means there is a significant difference among the three groups of hospice, hospital with LCP, and hospital without LCP in terms of the ward environment.

7. Yes, these results are helpful. The means, standard deviations, and ranges are important for describing the variables in this study. The results from these analyses provide more information about the central tendencies and dispersion of data for the study variables. Additionally, means and standard deviations are essential for conducting power analyses to determine sample sizes for future studies and are important for conducting meta-analyses used to summarize the results from multiple studies to facilitate evidence-based practice ( Brown, 2014 ; Cohen, 1988 ; Melnyk & Fineout-Overholt, 2015 ).

8. The ANOVA statistic for spiritual need—next-of-kin is F = 22.6, p < 0.0001. Since α = 0.05 in this study, any results with a p (probability) of ≤0.05 is considered statistically significant; 187therefore, p < 0.0001 is statistically significant. This result means there is a statistically significant difference among the three groups of hospice, hospital with LCP, and hospital without LCP in terms of spiritual need—next-of-kin.

9. The post hoc results for spiritual need—next-of-kin were p = 0.006 for hospice versus hospital with LCP, p = 0.0001 for hospice versus hospital without LCP, and p = 0.002 for hospital with LCP versus hospital without LCP. All three groups are significantly different since the p values are less than α = 0.05 set for this study. These results mean there are statistically significant differences between the hospice and hospital with LCP groups, between the hospice and hospital without LCP, and between the hospital with LCP and hospital without LCP in terms of spiritual need—next-of-kin.

10. Answers may vary. End of life care extends beyond the patient and includes next-of-kin or patients' family members. Mayland et al. (2014) described the need to improve end of life care for both patients and family. With end of life care, “one recognized method of evaluation involves the use of bereaved relatives as ‘proxy’ measures to assess the quality of care” ( Mayland et al., 2014 , p. 688). For additional resources on end of life care, hospice care, and grief process, you might search the American Cancer Society website at http://www.cancer.org .

189

EXERCISE 18 Questions to Be Graded

Follow your instructor's directions to submit your answers to the following questions for grading. Your instructor may ask you to write your answers below and submit them as a hard copy for grading. Alternatively, your instructor may ask you to use the space below for notes and submit your answers online at http://evolve.elsevier.com/Grove/Statistics/ under “Questions to Be Graded.”

Name: _______________________________________________________ Class: _____________________

Date: ___________________________________________________________________________________

1. Mayland et al. (2014) do not provide the degrees of freedom (df) in their study. Use the degrees of freedom formulas provided at the beginning of this exercise to calculate the group df and the error df.

2. What is the F value and p value for spiritual need—patient? What do these results mean?

3. What is the post hoc result for facilities for the hospital with LCP vs. the hospital without LCP (see Table 2 )? Is this result statistically significant? In your opinion, is this an expected finding?

4. What are the assumptions for use of ANOVA?

190

5. What variable on Table 3 has the result F = 10.6, p < 0.0001? What does the result mean?

6. ANOVA was used for analysis by Mayland et al. (2014) . Would t-tests have also been appropriate? Provide a rationale for your answer.

7. What type of post hoc analysis was performed? Is the post hoc analysis performed more or less conservative than the Scheffé test?

8. State the null hypothesis for care for the three study groups (see Table 2 ). Should the null hypothesis be accepted or rejected? Provide a rationale for your answer.

9. What are the post hoc results for care? Which results are statistically significant? What do the results mean?

10. In your opinion, do the study findings presented in Tables 2 and 3 have implications for end of life care? Provide a rationale for your answer.