11.2: The One-Way ANOVA Test
Learning Objectives
- Understand the shortcomings of comparing multiple means as pairs of hypotheses.
- Understand the steps of the ANOVA method and its advantages.
- Compare the means of three or more populations using the ANOVA method.
- Calculate the pooled standard deviation and confidence intervals as estimates of standard deviations of the populations.
Introduction
Previously, we have discussed analysis that allows us to test if the means and variances of two populations are equal. But let’s say that a teacher is testing multiple reading programs to determine the impact on student achievement. There are five different reading programs and her \begin{align*}31\end{align*} students are randomly assigned to one of the five programs. The mean achievement scores and variances for the groups are recorded along with the means and the variances for all the subjects combined.
We could conduct a series of \begin{align*}t\end{align*}-tests to test that all of the sample means came from the same population. However, this would be tedious and has a major flaw which we will discuss later. Instead, we use something called the Analysis of Variance (ANOVA) that allows us to test the hypothesis that multiple \begin{align*}(K)\end{align*} population means and variance of scores are equal. Theoretically, we could test hundreds of population means using this procedure.
Shortcomings of Comparing Multiple Means Using Previously Explained Methods
As mentioned, to test whether pairs of sample means differ by more than we would expect due to chance, we could conduct a series of separate \begin{align*}t\end{align*}-tests in order to compare all possible pairs of means. This would be tedious, but we could use the computer or TI-83/4 calculator to compute these easily and quickly. However, there is a major flaw with this reasoning.
When more than one \begin{align*}t\end{align*}-test is run, each at its own level of significance ( \begin{align*}\alpha =.10, .05, .01\end{align*}, etc.) the probability of making one or more Type I errors multiplies exponentially. Recall that a Type I error occurs when we reject the null hypothesis when we should not. The level of significance, \begin{align*}\alpha\end{align*}, is the probability of a Type I error in a single test. When testing more than one pair of samples, the probability of making at least one Type I error is \begin{align*}1-(1-\alpha)^c\end{align*} where \begin{align*}\alpha\end{align*} is the level of significance for each \begin{align*}t\end{align*}-test and \begin{align*}c\end{align*} is the number of independent \begin{align*}t\end{align*}-tests. Using the example from the introduction, if our teacher tested conducted separate \begin{align*}t\end{align*}-tests to examine the means of the populations, she would have to conduct \begin{align*}10\end{align*} separate \begin{align*}t\end{align*}-tests. If she performed these tests with \begin{align*}\alpha = .05\end{align*}, the probability of committing a Type I error is not \begin{align*}.05\end{align*} as one would initially expect. Instead, it would be \begin{align*}0.40\end{align*} – extremely high!
The Steps of the ANOVA Method
In ANOVA, we are actually analyzing the total variation of the scores including (1) the variation of the scores within the groups and (2) the variation between the group means. Since we are interested in two different types of variation, we first calculate each type of variation independently and then calculate the ratio between the two. We use the \begin{align*}F\end{align*}-distribution as our sampling distribution and set our critical values and test our hypothesis accordingly.
When using the ANOVA method, we are testing the null hypothesis that the means and the variances of our samples are equal. When we conduct a hypothesis test, we are testing the probability of obtaining an extreme \begin{align*}F\end{align*}-statistic by chance. If we reject the null hypothesis that the means and variances of the samples are equal, then we are saying that there is a small likelihood \begin{align*}\alpha\end{align*} that we would have obtained such an extreme \begin{align*}F\end{align*}-statistic by chance.
To test a hypothesis using the ANOVA method, there are several steps that we need to take. These include:
1. Calculating the mean squares between groups \begin{align*}(MS_B)\end{align*}. The \begin{align*}MS_B\end{align*} is the difference between the means of the various samples. If we hypothesize that the group means are equal \begin{align*}(\mu _1 = \mu_2 = \ldots = \mu_k)\end{align*}, then they must also equal the population mean. Under our null hypothesis, we state that the means of the different samples are all equal and come from the same population, but we understand that there may be fluctuations due to sampling error.
When we calculate the \begin{align*}MS_B\end{align*} , we must first determine the \begin{align*}SS_B\end{align*} , which is the sum of the differences between the individual scores and the means in each group. To calculate this difference, we use the formula:
\begin{align*}{SS_B} = \sum_{k=1}^k n_k(\bar X_k - \bar X)^2\end{align*}
where:
\begin{align*}k =\end{align*} the group number
\begin{align*}n_k =\end{align*} the sample size in group \begin{align*}k\end{align*}
\begin{align*}{\bar X_k} =\end{align*} the mean of group \begin{align*}k\end{align*}
\begin{align*}{\bar X} =\end{align*} mean of all individual observations
\begin{align*}k =\end{align*} the number of groups
When simplified, the formula becomes:
\begin{align*}{SS_B} = \sum_{k=1}^k \frac {T_k^2}{n_k} - \frac {T^2}{N}\end{align*}
where
\begin{align*}T_k =\end{align*} sum of the observations in group \begin{align*}K\end{align*}
\begin{align*}T =\end{align*} sum of all observations.
Once we calculate this value, we divide by the number of degrees of freedom \begin{align*}(K-1)\end{align*} to arrive at the \begin{align*}MS_B\end{align*}.
\begin{align*}{MS_B} = \frac {SS_B}{K-1}\end{align*}
2. Calculating the mean squares within groups \begin{align*}(MS_W)\end{align*}. The mean squares within groups calculation is also called the pooled estimate of the population variance. Remember that when we square the standard deviation of a sample, we are estimating population variance. Therefore, to calculate this figure, we sum of the squared deviations within each group and then divide by the sum of the degrees of freedom for each group.
To calculate the \begin{align*}MS_W\end{align*} we first find the \begin{align*}SS_W\end{align*}, which is calculated using the formula:
\begin{align*}\frac {\sum (X_{i1} - {\bar X_1})^2 + \textstyle \sum (X_{i2} - {\bar X_2})^2 +\ldots + \textstyle \sum (X_{ik} - {\bar X_k})^2} {(n_1-1) + (n_2-1)+\ldots+(n_k-1)}\end{align*}
Simplified, this formula states:
\begin{align*}{SS_W} = \sum_{k=1}^k \sum_{i=1}^{n_k} X^2_{ik} - \sum_{k=1}^k \frac {T_k^2}{n_k}\end{align*}
where
\begin{align*}T_k =\end{align*} sum of the observations in group \begin{align*}k\end{align*}
Essentially, this formula sums the squares of each observation and then subtracts the total of the observations squared divided by the number of observations. Finally, we divide this value by the total number of degrees of freedom in the scenario \begin{align*}(N-K)\end{align*}.
\begin{align*}{MS_w} = \frac {SS_w}{N-K}\end{align*}
3. Calculate the test statistic. The test statistic is as follows:
\begin{align*}F = \frac {MS_B}{MS_W}\end{align*}
4. Find the critical value on the \begin{align*}F\end{align*}- distribution. As mentioned above, \begin{align*}K-1\;\mathrm{degrees}\end{align*} of freedom are associated with \begin{align*}MS_B\end{align*} and \begin{align*}N-K \;\mathrm{degrees}\end{align*} of freedom are associated with \begin{align*}MS_W\end{align*}. The degrees of freedom for \begin{align*}MS_B\end{align*} are read across the columns and the degrees of freedom for \begin{align*}MS_W\end{align*} are read across the rows.
5. Interpret the results of the hypothesis test. In ANOVA, the last step is to decide whether to reject the null hypothesis and then provide clarification about what that decision means.
The primary advantage to using the ANOVA method is that it takes all types of variation into account so that we have an accurate analysis. In addition, we can use technological tools including computer programs (SAS, SPSS, Microsoft Excel) and the TI-83/4 calculator to easily conduct the calculations and test our hypothesis. We use these technological tools quite often when using the ANOVA method.
Let’s take a look at an example to help clarify.
Example:
Let’s go back to the example in the introduction with the teacher that is testing multiple reading programs to determine the impact on student achievement. There are five different reading programs and her \begin{align*}31\end{align*} students are randomly assigned to the five programs and she collects the following data:
Method
\begin{align*}& 1 && 2 && 3 && 4 && 5 \\ & 1 && 8 && 7 && 9 && 10 \\ & 4 && 6 && 6 && 10 && 12 \\ & 3 && 7 && 4 && 8 && 9 \\ & 2 && 4 && 9 && 6 && 11 \\ & 5 && 3 && 8 && 5 &&8 \\ & 1 && 5 && 5 &&&&\\ & 6 && && 7 &&&&\\ & &&&& 5 &&&&\end{align*} Please (1) compare the means of these different groups by calculating the mean squares between groups and (2) use the standard deviations from our samples to calculate the mean squares within groups and estimate the pooled variance of a population.
Solution:
To solve for \begin{align*}SS_B\end{align*} , it is necessary to calculate several summary statistics from the data above.
\begin{align*}& \text{Number} (n_k) && 7 && 6 && 8 && 5 && 5 && 31\\ & \text{Total} (T_k) && 22 && 33 && 51 && 38 && 50 &&= 194\\ & \text{Mean} (\bar X) && 3.14 && 5.50 && 6.38 && 7.60 && 10.00 && = 6.26\\ & \text{Sum of Squared Obs.} \left (\sum_{i=1}^{n_k} X^2_{ik}\right ) && 92 && 199 && 345 && 306 && 510 && = 1,452\\ & \frac{\text{Sum of Obs. Squared}}{\text{Number of Obs}} \left (\frac {T_k^2}{n_k}\right ) && 69.14 && 181.50 && 325.13 && 288.80 && 500.00 && = 1,364.57\end{align*}
Using this information, we find that the sum of squares between groups is equal to
\begin{align*}& {SS_B} = \sum_{k=1}^k \frac {T_k^2}{n_k} - {\frac {T^2}{N}}\\ & \approx 1,364.57 - \frac{(194)^2}{31} \approx {150.5}\end{align*}
Since there are four Degrees of Freedom for this calculation (the number of groups minus one), the mean squares between groups is
\begin{align*}MS_B=\frac{SS_B}{K-1}\approx \frac {150.5}{4} \approx 37.6\end{align*}
Next we calculate the mean squares within groups \begin{align*}(MS_W)\end{align*} which is also known as the estimation of the pooled variance of a population \begin{align*}(\sigma^2)\end{align*}.
To calculate the mean squares within groups, we use the formula
\begin{align*}{SS_W} = \sum_{k=1}^k \sum_{i=1}^{n_k}X^2_{ik} - \sum_{k=1}^k \frac {T_k^2}{n_k} \end{align*}
Using our summary statistics from above, we can calculate that the within groups mean square \begin{align*}(MS_W)\end{align*} is equal to:
\begin{align*}{SS_W} & = \sum_{k=1}^k \sum_{i=1}^{n_k}X^2_{ik} - \sum_{k=1}^k \frac {T_k^2}{n_k}\\ & \approx 1,452 - 1,364.57\\ & \approx 87.43\end{align*}
And so we have
\begin{align*}{MS_W} = \frac {SS_W}{N-K} \approx \frac {87.43}{26} \approx 3.36\end{align*}
Therefore, our \begin{align*}F\end{align*}-Ratio is
\begin{align*}F = \frac {MS_B}{MS_W}\approx \frac{37.6}{3.36}\approx 11.18\end{align*}
We would then analyze this test statistic against our critical value (using the \begin{align*}F\end{align*}-distribution table and a value of \begin{align*}(\alpha =.02)\end{align*}, we find our critical value equal to \begin{align*}4.14\end{align*}. Since our test statistic \begin{align*}(11.18)\end{align*} exceeds our critical value \begin{align*}(4.14)\end{align*}, we reject the null hypothesis. Therefore, we can conclude that not all of the population means of the five programs are equal and that obtaining an \begin{align*}F\end{align*}-ratio that extreme by chance is highly improbable.
Technology Note - Excel
Here is the procedure for performing a One-way ANOVA in Excel using this set of data.
- Copy and paste the table into an empty Excel worksheet
- Select Data Analysis from the Tools menu and choose “ANOVA: Single-factor” from the list that appears
- Place the cursor is in the “Input Range” field and select the entire table.
- Place the cursor in the “Output Range” and click somewhere in a blank cell below the table.
- Click “Labels” only if you have also included the labels in the table. This will cause the names of the predictor variables to be displayed in the table
- Click OK and the results shown below will be displayed.
Note: The TI-83/4 also offers a One-way ANOVA test.
Anova: Single Factor
Groups | Count | Sum | Average | Variance |
---|---|---|---|---|
Column 1 | \begin{align*}7\end{align*} | \begin{align*}22\end{align*} | \begin{align*}3.142857\end{align*} | \begin{align*}3.809524\end{align*} |
Column 2 | \begin{align*}6\end{align*} | \begin{align*}33\end{align*} | \begin{align*}5.5\end{align*} | \begin{align*}3.5\end{align*} |
Column 3 | \begin{align*}8\end{align*} | \begin{align*}51\end{align*} | \begin{align*}6.375\end{align*} | \begin{align*}2.839286\end{align*} |
Column 4 | \begin{align*}5\end{align*} | \begin{align*}38\end{align*} | \begin{align*}7.6\end{align*} | \begin{align*}4.3\end{align*} |
Column 5 | \begin{align*}5\end{align*} | \begin{align*}50\end{align*} | \begin{align*}10\end{align*} | \begin{align*}2.5\end{align*} |
Source of Variation | \begin{align*}SS\end{align*} | \begin{align*}df\end{align*} | \begin{align*}MS\end{align*} | \begin{align*}F\end{align*} | \begin{align*}P-\end{align*}value | \begin{align*}F\end{align*} crit |
---|---|---|---|---|---|---|
Between Groups | \begin{align*}150.5033\end{align*} | \begin{align*}4\end{align*} | \begin{align*}37.62584\end{align*} | \begin{align*}11.18893\end{align*} | \begin{align*}2.05E-05\end{align*} | \begin{align*}2.742594\end{align*} |
Within Groups | \begin{align*}87.43214\end{align*} | \begin{align*}26\end{align*} | \begin{align*}3.362775\end{align*} | |||
Total | \begin{align*}237.9355\end{align*} | \begin{align*}30\end{align*} |
Lesson Summary
- When testing multiple independent samples to determine if they come from the same populations, we could conduct a series of separate \begin{align*}t\end{align*}-tests in order to compare all possible pairs of means. However, a more precise and accurate analysis is the Analysis of Variance (ANOVA).
- In ANOVA, we analyze the total variation of the scores including (1) the variation of the scores within the groups and (2) the variation between the group means and the total mean of all the groups (also known as the grand mean).
- In this analysis, we calculate the \begin{align*}F\end{align*}-ratio, which is the total mean of squares between groups divided by the total mean of squares within groups.
- The total mean of squares within groups is also known as the estimate of the pooled variance of the population. We find this value by analysis of the standard deviations in each of the samples.
Review Questions
- What does the ANOVA acronym stand for?
- If we are tested whether pairs of sample means differ by more than we would expect due to chance using multiple \begin{align*}t\end{align*}-tests, the probability of making a Type I error would ___.
- In the ANOVA method, we use the ___ distribution.
- Student’s \begin{align*}t\end{align*}-
- normal
- \begin{align*}F\end{align*}-
- In the ANOVA method, we complete a series of steps to evaluate our hypothesis. Put the following steps in chronological order.
- Calculate the mean squares between groups and the means squares within groups
- Determine the critical values in the \begin{align*}F\end{align*}-distribution
- Evaluate the hypothesis
- Calculate the test statistic
- State the null hypothesis
A school psychologist is interested whether or not teachers affect the anxiety scores among students taking the AP Statistics exam. The data below are the scores on a standardized anxiety test for students with three different teachers.
Ms. Jones | Mr. Smith | Mrs. White |
---|---|---|
\begin{align*}8\end{align*} | \begin{align*}23\end{align*} | \begin{align*}21\end{align*} |
\begin{align*}6\end{align*} | \begin{align*}11\end{align*} | \begin{align*}21\end{align*} |
\begin{align*}4\end{align*} | \begin{align*}17\end{align*} | \begin{align*}22\end{align*} |
\begin{align*}12\end{align*} | \begin{align*}16\end{align*} | \begin{align*}18\end{align*} |
\begin{align*}16\end{align*} | \begin{align*}6\end{align*} | \begin{align*}14\end{align*} |
\begin{align*}17\end{align*} | \begin{align*}14\end{align*} | \begin{align*}21\end{align*} |
\begin{align*}12\end{align*} | \begin{align*}15\end{align*} | \begin{align*}9\end{align*} |
\begin{align*}10\end{align*} | \begin{align*}19\end{align*} | \begin{align*}11\end{align*} |
\begin{align*}11\end{align*} | \begin{align*}10\end{align*} | |
\begin{align*}13\end{align*} |
- State the null hypothesis.
- Using the data above, please fill out the missing values in the table below.
Ms. Jones | Mr. Smith | Mrs. White | Totals | |
---|---|---|---|---|
Number \begin{align*}(n_k)\end{align*} | \begin{align*}8\end{align*} | \begin{align*}=\end{align*} | ||
Total \begin{align*}(T_k)\end{align*} | \begin{align*}131\end{align*} | \begin{align*}=\end{align*} | ||
Mean \begin{align*}(\bar X)\end{align*} | \begin{align*}14.6\end{align*} | \begin{align*}=\end{align*} | ||
Sum of Squared Obs. \begin{align*}\textstyle (\sum_{i=1}^{n_k} X^2_{ik})\end{align*} | \begin{align*}=\end{align*} | |||
Sum of Obs. Squared/Number of Obs. \begin{align*}\left (\frac {T_k^2}{n_k}\right )\end{align*} | \begin{align*}= \end{align*} |
- What is the mean squares between groups \begin{align*}(MS_B)\end{align*} value?
- What is the mean squares within groups \begin{align*}(MS_W)\end{align*} value?
- What is the \begin{align*}F\end{align*}-ratio of these two values?
- Using a \begin{align*}\alpha = .05\end{align*}, please use the \begin{align*}F\end{align*}-distribution to set a critical value
- What decision would you make regarding the null hypothesis? Why?
Review Answers
- Analysis of Variance
- Increase or increase exponentially
- \begin{align*}C\end{align*}
- \begin{align*}E, A, D, B, C\end{align*}
- \begin{align*}H_0: \mu_1 = \mu_2 = \mu_3\end{align*}
Ms. Jones | Mr. Smith | Mrs. White | Totals | |
---|---|---|---|---|
Number \begin{align*}(n_k)\end{align*} | \begin{align*}10\end{align*} | \begin{align*}9\end{align*} | \begin{align*}8\end{align*} | \begin{align*}= 27\end{align*} |
Total \begin{align*}(T_k)\end{align*} | \begin{align*}109\end{align*} | \begin{align*}131\end{align*} | \begin{align*}137\end{align*} | \begin{align*}= 377\end{align*} |
Mean \begin{align*}(\bar X)\end{align*} | \begin{align*}10.9\end{align*} | \begin{align*}14.6\end{align*} | \begin{align*}17.1\end{align*} | \begin{align*}= 5,264\end{align*} |
Sum of Squared Obs. \begin{align*}\textstyle (\sum_{i=1}^{n_k} X^2_{ik})\end{align*} | \begin{align*}1,339\end{align*} | \begin{align*}2,113\end{align*} | \begin{align*}2,529\end{align*} | \begin{align*}= 5,981\end{align*} |
Sum of Obs. Squared/Number of Obs. \begin{align*}\left (\frac {T_k^2}{n_k}\right )\end{align*} | \begin{align*}1,188\end{align*} | \begin{align*}1,907\end{align*} | \begin{align*}2,346\end{align*} | \begin{align*}= 5,441\end{align*} |
- \begin{align*}26.35\end{align*}
- \begin{align*}4.03\end{align*}
- \begin{align*}6.54\end{align*}
- \begin{align*}3.40\end{align*}
- The calculated test statistic exceeds the critical value so we would reject the null hypothesis. Therefore, we could conclude that not all the population means are equal.
Notes/Highlights Having trouble? Report an issue.
Color | Highlighted Text | Notes | |
---|---|---|---|
Show More |