Calculating Analysis of Variance (ANOVA) and Post Hoc Analyses Following ANOVA
Analysis of variance (ANOVA) is a statistical procedure that compares data between two or more groups or conditions to investigate the presence of differences between those groups on some continuous dependent variable (see Exercise 18). In this exercise, we will focus on the one-way ANOVA, which involves testing one independent variable and one dependent variable (as opposed to other types of ANOVAs, such as factorial ANOVAs that incorporate multiple independent variables).
Why ANOVA and not a t-test? Remember that a t-test is formulated to compare two sets of data or two groups at one time (see Exercise 23 for guidance on selecting appropriate statistics). Thus, data generated from a clinical trial that involves four experimental groups, Treatment 1, Treatment 2, Treatments 1 and 2 combined, and a Control, would require 6 t-tests. Consequently, the chance of making a Type I error (alpha error) increases substantially (or is inflated) because so many computations are being performed. Specifically, the chance of making a Type I error is the number of comparisons multiplied by the alpha level. Thus, ANOVA is the recommended statistical technique for examining differences between more than two groups (Zar, 2010).
ANOVA is a procedure that culminates in a statistic called the F statistic. It is this value that is compared against an F distribution (see Appendix C) in order to determine whether the groups significantly differ from one another on the dependent variable. The formulas for ANOVA actually compute two estimates of variance: One estimate represents differences between the groups/conditions, and the other estimate represents differences among (within) the data.
Research Designs Appropriate for the One-Way ANOVA
Research designs that may utilize the one-way ANOVA include the randomized experimental, quasi-experimental, and comparative designs (Gliner, Morgan, & Leech, 2009). The independent variable (the “grouping” variable for the ANOVA) may be active or attributional. An active independent variable refers to an intervention, treatment, or program. An attributional independent variable refers to a characteristic of the participant, such as gender, diagnosis, or ethnicity. The ANOVA can compare two groups or more. In the case of a two-group design, the researcher can either select an independent samples t-test or a one-way ANOVA to answer the research question. The results will always yield the same conclusion, regardless of which test is computed; however, when examining differences between more than two groups, the one-way ANOVA is the preferred statistical test.
Example 1: A researcher conducts a randomized experimental study wherein she randomizes participants to receive a high-dosage weight loss pill, a low-dosage weight loss pill, or a placebo. She assesses the number of pounds lost from baseline to post-treatment 378for the three groups. Her research question is: “Is there a difference between the three groups in weight lost?” The independent variables are the treatment conditions (high-dose weight loss pill, low-dose weight loss pill, and placebo) and the dependent variable is number of pounds lost over the treatment span.
Null hypothesis: There is no difference in weight lost among the high-dose weight loss pill, low-dose weight loss pill, and placebo groups in a population of overweight adults.
Example 2: A nurse researcher working in dermatology conducts a retrospective comparative study wherein she conducts a chart review of patients and divides them into three groups: psoriasis, psoriatric symptoms, or control. The dependent variable is health status and the independent variable is disease group (psoriasis, psoriatic symptoms, and control). Her research question is: “Is there a difference between the three groups in levels of health status?”
Null hypothesis: There is no difference between the three groups in health status.
Statistical Formula and Assumptions
Use of the ANOVA involves the following assumptions (Zar, 2010):
1. de Winter & Dodou, 2010; Rasmussen, 1989).
The basic formula for the F without numerical symbols is:
F=Mean Square Between GroupsMean Square Within Groups
The term “mean square” (MS) is used interchangeably with the word “variance.” The formulas for ANOVA compute two estimates of variance: the between groups variance and the within groups variance. The between groups variance represents differences between the groups/conditions being compared, and the within groups variance represents differences among (within) each group’s data. Therefore, the formula is F = MS between/MS within.
Hand Calculations
Using an example from a study of students enrolled in an RN to BSN program, a subset of graduates from the program were examined (Mancini, Ashwill, & Cipher, 2014). The data are presented in Table 33-1. A simulated subset was selected for this example so that 379the computations would be small and manageable. In actuality, studies involving one-way ANOVAs need to be adequately powered (Aberson, 2010; Cohen, 1988). See Exercises 24 and 25 for more information regarding statistical power.
TABLE 33-1
Table 33-2).Appendix C) and compare it to our obtained F = 7.96 value. The critical F value for 2 and 24 df at α = 0.05 is 3.40, which indicates the F value in this example is statistically significant. Researchers report ANOVA results in a study report using the following format: F(2,24) = 7.96, p < 0.05. Researchers report the exact p value instead of “p < 0.05,” but this usually requires the use of computer software due to the tedious nature of p value computations.
Our obtained F = 7.96 exceeds the critical value in the table, which indicates that the F is statistically significant and that the population means are not equal. Therefore, we can reject our null hypothesis that the three groups spent the same amount of time completing the RN to BSN program. However, the F does not indicate which groups differ from one another, and this F value does not identify which groups are significantly different from one another. Further testing, termed multiple comparison tests or post hoc tests, is required to complete the ANOVA process and determine all the significant differences among the study groups.
Post Hoc Tests
Post hoc tests have been developed specifically to determine the location of group differences after ANOVA is performed on data from more than two groups. These tests were developed to reduce the incidence of a Type I error. Frequently used post hoc tests are the Newman-Keuls test, the Tukey Honestly Significant Difference (HSD) test, the Scheffé test, and the Dunnett test (Zar, 2010; see Exercise 18 for examples). When these tests are 381calculated, the alpha level is reduced in proportion to the number of additional tests required to locate statistically significant differences. For example, for several of the aforementioned post hoc tests, if many groups’ mean values are being compared, the magnitude of the difference is set higher than if only two groups are being compared. Thus, post hoc tests are tedious to perform by hand and are best handled with statistical computer software programs. Accordingly, the rest of this example will be presented with the assistance of SPSS.
SPSS Computations
The following screenshot is a replica of what your SPSS window will look like. The data for ID numbers 24 through 27 are viewable by scrolling down in the SPSS screen.
Table 33-2. This table contains an additional value that we did not compute by hand—the exact p value, which is 0.002. Because the SPSS output indicates that we have a significant ANOVA, post hoc testing must be performed.
Return to the ANOVA window and click “Post Hoc.” You will see a window similar to the one below. Select the “LSD” and “Tukey” options. Click “Continue” and “OK.”
The following output is added to the original output. This table contains post hoc test results for two different tests: the LSD (Least Significant Difference) test and the Tukey HSD (Honestly Significant Difference) test. The LSD test, the original post hoc test, explores all possible pairwise comparisons of means using the equivalent of multiple t-tests. However, the LSD test, in performing a set of multiple t-tests, reports inaccurate p values that have not been adjusted for multiple computations (Zar, 2010). Consequently, researchers should exercise caution when choosing the LSD post hoc test following an ANOVA.
The Tukey HSD comparison test, on the other hand, is a more “conservative” test, meaning that it requires a larger difference between two groups to indicate a significant difference than some of the other post hoc tests available. By requiring a larger difference between the groups, the Tukey HSD procedure yields more accurate p values of 0.062 to reflect the multiple comparisons (Zar, 2010).
Post Hoc Tests
Exercises 24 and 25 for more information regarding the consequences of low statistical power.
Final Interpretation in American Psychological Association (Apa) Format
The following interpretation is written as it might appear in a research article, formatted according to APA guidelines (APA, 2010). A one-way ANOVA performed on months to program completion revealed significant differences among the three groups, F(2,24) = 7.96, p = 0.002. Post hoc comparisons using the Tukey HSD comparison test indicated that the students in the Associate’s degree group took significantly longer to complete the program than the students in the Bachelor’s degree group (19.8 versus 13.9 months, respectively) (APA, 2010). However, there were no significant differences in program completion time between the Associate’s degree group and the Master’s degree group or between the Bachelor’s degree group and the Master’s degree group.
Study Questions
Mancini et al. (2014) example normally distributed? Provide a rationale for your answer.
2. Please see below and contact first appeared on Intellectpaper.
