11.2: The One-Way ANOVA Test
Learning Objectives
- Understand the shortcomings of comparing multiple means as pairs of hypotheses.
- Understand the steps of the ANOVA method and the method's advantages.
- Compare the means of three or more populations using the ANOVA method.
- Calculate pooled standard deviations and confidence intervals as estimates of standard deviations of populations.
Introduction
Previously, we have discussed analyses that allow us to test if the means and variances of two populations are equal. Suppose a teacher is testing multiple reading programs to determine the impact on student achievement. There are five different reading programs, and her 31 students are randomly assigned to one of the five programs. The mean achievement scores and variances for the groups are recorded, along with the means and the variances for all the subjects combined.
We could conduct a series of \begin{align*}t\end{align*}-tests to determine if all of the sample means came from the same population. However, this would be tedious and has a major flaw, which we will discuss shortly. Instead, we use something called the Analysis of Variance (ANOVA), which allows us to test the hypothesis that multiple population means and variances of scores are equal. Theoretically, we could test hundreds of population means using this procedure.
Shortcomings of Comparing Multiple Means Using Previously Explained Methods
As mentioned, to test whether pairs of sample means differ by more than we would expect due to chance, we could conduct a series of separate \begin{align*}t\end{align*}-tests in order to compare all possible pairs of means. This would be tedious, but we could use a computer or a TI-83/84 calculator to compute these quickly and easily. However, there is a major flaw with this reasoning.
When more than one \begin{align*}t\end{align*}-test is run, each at its own level of significance, the probability of making one or more type I errors multiplies exponentially. Recall that a type I error occurs when we reject the null hypothesis when we should not. The level of significance, \begin{align*}\alpha\end{align*}, is the probability of a type I error in a single test. When testing more than one pair of samples, the probability of making at least one type I error is \begin{align*}1-(1-\alpha)^c\end{align*}, where \begin{align*}\alpha\end{align*} is the level of significance for each \begin{align*}t\end{align*}-test and \begin{align*}c\end{align*} is the number of independent \begin{align*}t\end{align*}-tests. Using the example from the introduction, if our teacher conducted separate \begin{align*}t\end{align*}-tests to examine the means of the populations, she would have to conduct 10 separate \begin{align*}t\end{align*}-tests. If she performed these tests with \begin{align*}\alpha=0.05\end{align*}, the probability of committing a type I error is not 0.05 as one would initially expect. Instead, it would be 0.40, which is extremely high!
The Steps of the ANOVA Method
With the ANOVA method, we are actually analyzing the total variation of the scores, including the variation of the scores within the groups and the variation between the group means. Since we are interested in two different types of variation, we first calculate each type of variation independently and then calculate the ratio between the two. We use the \begin{align*}F\end{align*}-distribution as our sampling distribution and set our critical values and test our hypothesis accordingly.
When using the ANOVA method, we are testing the null hypothesis that the means and the variances of our samples are equal. When we conduct a hypothesis test, we are testing the probability of obtaining an extreme \begin{align*}F\end{align*}-statistic by chance. If we reject the null hypothesis that the means and variances of the samples are equal, and then we are saying that the difference that we see could not have happened just by chance.
To test a hypothesis using the ANOVA method, there are several steps that we need to take. These include:
1. Calculating the mean squares between groups, \begin{align*}MS_B\end{align*}. The \begin{align*}MS_B\end{align*} is the difference between the means of the various samples. If we hypothesize that the group means are equal, then they must also equal the population mean. Under our null hypothesis, we state that the means of the different samples are all equal and come from the same population, but we understand that there may be fluctuations due to sampling error. When we calculate the \begin{align*}MS_B\end{align*}, we must first determine the \begin{align*}SS_B\end{align*}, which is the sum of the differences between the individual scores and the mean in each group. To calculate this sum, we use the following formula:
\begin{align*}SS_B=\sum^m_{k=1} n_k (\bar{x}_k-\bar{x})^2\end{align*}
where:
\begin{align*}k\end{align*} is the group number.
\begin{align*}n_k\end{align*} is the sample size of group \begin{align*}k\end{align*}.
\begin{align*}\bar{x}_k\end{align*} is the mean of group \begin{align*}k\end{align*}.
\begin{align*}\bar{x}\end{align*} is the overall mean of all the observations.
\begin{align*}m\end{align*} is the total number of groups.
When simplified, the formula becomes:
\begin{align*}SS_B=\sum^m_{k=1} \frac{T^2_k}{n_k}-\frac{T^2}{n}\end{align*}
where:
\begin{align*}T_k\end{align*} is the sum of the observations in group \begin{align*}k\end{align*}.
\begin{align*}T\end{align*} is the sum of all the observations.
\begin{align*}n\end{align*} is the total number of observations.
Once we calculate this value, we divide by the number of degrees of freedom, \begin{align*}m-1\end{align*}, to arrive at the \begin{align*}MS_B\end{align*}. That is, \begin{align*}MS_B=\frac{SS_B}{m-1}\end{align*}
2. Calculating the mean squares within groups, \begin{align*}MS_W\end{align*}. The mean squares within groups calculation is also called the pooled estimate of the population variance. Remember that when we square the standard deviation of a sample, we are estimating population variance. Therefore, to calculate this figure, we sum the squared deviations within each group and then divide by the sum of the degrees of freedom for each group.
To calculate the \begin{align*}MS_W\end{align*}, we first find the \begin{align*}SS_W\end{align*}, which is calculated using the following formula:
\begin{align*}\frac{\sum(x_{i1}-\bar{x}_1)^2+\sum (x_{i2}-\bar{x}_2)^2+ \ldots + \sum (x_{im}-\bar{x}_m)^2}{(n_1-1)+(n_2-1)+ \ldots + (n_m-1)}\end{align*}
Simplified, this formula becomes:
\begin{align*}SS_W=\sum^m_{k=1} \sum^{n_k}_{i=1} x^2_{ik}-\sum^m_{k=1} \frac{T^2_k}{n_k}\end{align*}
where:
\begin{align*}T_k\end{align*} is the sum of the observations in group \begin{align*}k\end{align*}.
Essentially, this formula sums the squares of each observation and then subtracts the total of the observations squared divided by the number of observations. Finally, we divide this value by the total number of degrees of freedom in the scenario, \begin{align*}n-m\end{align*}.
\begin{align*}MS_W=\frac{SS_W}{n-m}\end{align*}
3. Calculating the test statistic. The formula for the test statistic is as follows:
\begin{align*}F=\frac{MS_B}{MS_W}\end{align*}
4. Finding the critical value of the \begin{align*}F\end{align*}-distribution. As mentioned above, \begin{align*}m-1\end{align*} degrees of freedom are associated with \begin{align*}MS_B\end{align*}, and \begin{align*}n-m\end{align*} degrees of freedom are associated with \begin{align*}MS_W\end{align*}. In a table, the degrees of freedom for \begin{align*}MS_B\end{align*} are read across the columns, and the degrees of freedom for \begin{align*}MS_W\end{align*} are read across the rows.
5. Interpreting the results of the hypothesis test. In ANOVA, the last step is to decide whether to reject the null hypothesis and then provide clarification about what that decision means.
The primary advantage of using the ANOVA method is that it takes all types of variations into account so that we have an accurate analysis. In addition, we can use technological tools, including computer programs, such as SAS, SPSS, and Microsoft Excel, as well as the TI-83/84 graphing calculator, to easily perform the calculations and test our hypothesis. We use these technological tools quite often when using the ANOVA method.
Example: Let’s go back to the example in the introduction with the teacher who is testing multiple reading programs to determine the impact on student achievement. There are five different reading programs, and her 31 students are randomly assigned to one of the five programs. She collects the following data:
Method
\begin{align*}& 1 && 2 && 3 && 4 && 5 \\ & 1 && 8 && 7 && 9 && 10 \\ & 4 && 6 && 6 && 10 && 12 \\ & 3 && 7 && 4 && 8 && 9 \\ & 2 && 4 && 9 && 6 && 11 \\ & 5 && 3 && 8 && 5 &&8 \\ & 1 && 5 && 5 &&&&\\ & 6 && && 7 &&&&\\ & &&&& 5 &&&&\end{align*}
Compare the means of these different groups by calculating the mean squares between groups, and use the standard deviations from our samples to calculate the mean squares within groups and the pooled estimate of the population variance.
To solve for \begin{align*}SS_B\end{align*}, it is necessary to calculate several summary statistics from the data above:
\begin{align*}& \text{Number } (n_k) && 7 && 6 && 8 && 5 && 5 && 31\\ & \text{Total } (T_k) && 22 && 33 && 51 && 38 && 50 &&= 194\\ & \text{Mean } (\bar x) && 3.14 && 5.50 && 6.38 && 7.60 && 10.00 && = 6.26\\ & \text{Sum of Squared Obs. } \left (\sum_{i=1}^{n_k} x^2_{ik}\right ) && 92 && 199 && 345 && 306 && 510 && = 1,452\\ & \frac{\text{Sum of Obs. Squared }}{\text{Number of Obs}} \left (\frac {T_k^2}{n_k}\right ) && 69.14 && 181.50 && 325.13 && 288.80 && 500.00 && = 1,364.57\end{align*}
Using this information, we find that the sum of squares between groups is equal to the following:
\begin{align*}SS_B &= \sum^m_{k=1} \frac{T^2_k}{n_k}-\frac{T^2}{N}\\ & \approx 1, 364.57 - \frac{(194)^2}{31} \approx 150.5\end{align*}
Since there are four degrees of freedom for this calculation (the number of groups minus one), the mean squares between groups is as shown below:
\begin{align*}MS_B=\frac{SS_B}{m-1} \approx \frac{150.5}{4} \approx 37.6\end{align*}
Next, we calculate the mean squares within groups, \begin{align*}MS_W\end{align*}, which is also known as the pooled estimate of the population variance, \begin{align*}\sigma^2\end{align*}.
To calculate the mean squares within groups, we first use the following formula to calculate \begin{align*}SS_W\end{align*}:
\begin{align*}SS_W=\sum^m_{k=1} \sum^{n_k}_{i=1} x^2_{ik}-\sum^m_{k=1} \frac{T^2_k}{n_k}\end{align*}
Using our summary statistics from above, we can calculate \begin{align*}SS_W\end{align*} as shown below:
\begin{align*}SS_W &= \sum^m_{k=1} \sum^{n_k}_{i=1} x^2_{ik}-\sum^m_{k=1} \frac{T^2_k}{n_k}\\ & \approx 1, 452 - 1, 364.57\\ & \approx 87.43\end{align*}
This means that we have the following for \begin{align*}MS_W\end{align*}:
\begin{align*}MS_W=\frac{SS_W}{n-m} \approx \frac{87.43}{26} \approx 3.36\end{align*}
Therefore, our \begin{align*}F\end{align*}-ratio is as shown below:
\begin{align*}F=\frac{MS_B}{MS_W} \approx \frac{37.6}{3.36} \approx 11.19\end{align*}
We would then analyze this test statistic against our critical value. Using the \begin{align*}F\end{align*}-distribution table and \begin{align*}\alpha=0.02\end{align*}, we find our critical value equal to 4.140. Since our test statistic of 11.19 exceeds our critical value of 4.140, we reject the null hypothesis. Therefore, we can conclude that not all of the population means of the five programs are equal and that obtaining an \begin{align*}F\end{align*}-ratio this extreme by chance is highly improbable.
On the Web
http://preview.tinyurl.com/36j4by6 \begin{align*}F\end{align*}-distribution tables with \begin{align*}\alpha=0.02\end{align*}.
Technology Note: Calculating a One-Way ANOVA with Excel
Here is the procedure for performing a one-way ANOVA in Excel using this set of data.
Copy and paste the table into an empty Excel worksheet.
Select 'Data Analysis' from the Tools menu and choose 'ANOVA: Single-factor' from the list that appears.
Place the cursor in the 'Input Range' field and select the entire table.
Place the cursor in the 'Output Range' field and click somewhere in a blank cell below the table.
Click 'Labels' only if you have also included the labels in the table. This will cause the names of the predictor variables to be displayed in the table.
Click 'OK', and the results shown below will be displayed.
Anova: Single Factor
Groups | Count | Sum | Average | Variance |
---|---|---|---|---|
Column 1 | 7 | 22 | 3.142857 | 3.809524 |
Column 2 | 6 | 33 | 5.5 | 3,5 |
Column 3 | 8 | 51 | 6.375 | 2.839286 |
Column 4 | 5 | 38 | 7.6 | 4.3 |
Column 5 | 6 | 50 | 10 | 2.5 |
Source of Variation | \begin{align*}SS\end{align*} | \begin{align*}df\end{align*} | \begin{align*}MS\end{align*} | \begin{align*}F\end{align*} | \begin{align*}P\end{align*}-value | \begin{align*}F\end{align*} crit |
---|---|---|---|---|---|---|
Between Groups | 150.5033 | 4 | 37.62584 | 11.18893 | 2.05e-05 | 2.742594 |
Within Groups | 87.43214 | 26 | 3.362775 | |||
Total | 237.9355 | 30 |
Technology Note: One-Way ANOVA on the TI-83/84 Calculator
Enter raw data from population 1 into L1, population 2 into L2, population 3 into L3, population 4 into L4, and so on.
Now press [STAT], scroll right to TESTS, scroll down to 'ANOVA(', and press [ENTER]. Then enter the lists to produce a command such as 'ANOVA(L1, L2, L3, L4)' and press [ENTER].
Lesson Summary
When testing multiple independent samples to determine if they come from the same population, we could conduct a series of separate \begin{align*}t\end{align*}-tests in order to compare all possible pairs of means. However, a more precise and accurate analysis is the Analysis of Variance (ANOVA).
In ANOVA, we analyze the total variation of the scores, including the variation of the scores within the groups, the variation between the group means, and the total mean of all the groups (also known as the grand mean).
In this analysis, we calculate the \begin{align*}F\end{align*}-ratio, which is the total mean of squares between groups divided by the total mean of squares within groups.
The total mean of squares within groups is also known as the pooled estimate of the population variance. We find this value by analysis of the standard deviations in each of the samples.
Review Questions
- What does the ANOVA acronym stand for?
- If we are testing whether pairs of sample means differ by more than we would expect due to chance using multiple \begin{align*}t\end{align*}-tests, the probability of making a type I error would ___.
- In the ANOVA method, we use the ___ distribution.
- Student’s \begin{align*}t\end{align*}-
- normal
- \begin{align*}F\end{align*}-
- In the ANOVA method, we complete a series of steps to evaluate our hypothesis. Put the following steps in chronological order.
- Calculate the mean squares between groups and the mean squares within groups.
- Determine the critical values in the \begin{align*}F\end{align*}-distribution.
- Evaluate the hypothesis.
- Calculate the test statistic.
- State the null hypothesis.
- A school psychologist is interested in whether or not teachers affect the anxiety scores among students taking the AP Statistics exam. The data below are the scores on a standardized anxiety test for students with three different teachers.
Ms. Jones | Mr. Smith | Mrs. White |
---|---|---|
8 | 23 | 21 |
6 | 11 | 21 |
4 | 17 | 22 |
12 | 16 | 18 |
16 | 6 | 14 |
17 | 14 | 21 |
12 | 15 | 9 |
10 | 19 | 11 |
11 | 10 | |
13 |
(a) State the null hypothesis.
(b) Using the data above, fill out the missing values in the table below.
Ms. Jones | Mr. Smith | Mrs. White | Totals | |
---|---|---|---|---|
Number \begin{align*}(n_k)\end{align*} | 8 | \begin{align*}=\end{align*} | ||
Total \begin{align*}(T_k)\end{align*} | 131 | \begin{align*}=\end{align*} | ||
Mean \begin{align*}(\bar{x})\end{align*} | 14.6 | \begin{align*}=\end{align*} | ||
Sum of Squared Obs. \begin{align*}(\sum^{n_k}_{i=1} x^2_{ik})\end{align*} | \begin{align*}=\end{align*} | |||
Sum of Obs. Squared/Number of Obs. \begin{align*}(\frac{T^2_k}{n_k})\end{align*} | \begin{align*}=\end{align*} |
(c) What is the value of the mean squares between groups, \begin{align*}MS_B\end{align*}?
(d) What is the value of the mean squares within groups, \begin{align*}MS_W\end{align*}?
(e) What is the \begin{align*}F\end{align*}-ratio of these two values?
(f) With \begin{align*}\alpha=0.05\end{align*}, use the \begin{align*}F\end{align*}-distribution to set a critical value.
(g) What decision would you make regarding the null hypothesis? Why?