In this Concept, you will learn how to test the means and variances of multiple populations by using Analysis of Variance (ANOVA).
Watch This
For an example of a one-way ANOVA test, see statslectures, One-Way ANOVA (6:51).
Guidance
Previously, we have discussed analyses that allow us to test if the means and variances of two populations are equal. We will use the following example to demonstrate how to test the means and variances of multiple populations.
Example A
Suppose a teacher is testing multiple reading programs to determine the impact on student achievement. There are five different reading programs, and her 31 students are randomly assigned to one of the five programs. The mean achievement scores and variances for the groups are recorded, along with the means and the variances for all the subjects combined. How should we analyze this data?
Solution:
We could conduct a series of \begin{align*}t\end{align*}-tests to determine if all of the sample means came from the same population. However, this would be tedious and has a major flaw, which we will discuss shortly. Instead, we use something called the Analysis of Variance (ANOVA), which allows us to test the hypothesis that multiple population means and variances of scores are equal. Theoretically, we could test hundreds of population means using this procedure.
Shortcomings of Comparing Multiple Means Using Previously Explained Methods
As mentioned, to test whether pairs of sample means differ by more than we would expect due to chance, we could conduct a series of separate \begin{align*}t\end{align*}-tests in order to compare all possible pairs of means. This would be tedious, but we could use a computer or a TI-83/84 calculator to compute these quickly and easily. However, there is a major flaw with this reasoning.
When more than one \begin{align*}t\end{align*}-test is run, each at its own level of significance, the probability of making one or more type I errors multiplies exponentially. Recall that a type I error occurs when we reject the null hypothesis when we should not. The level of significance, \begin{align*}\alpha\end{align*}, is the probability of a type I error in a single test. When testing more than one pair of samples, the probability of making at least one type I error is \begin{align*}1-(1-\alpha)^c\end{align*}, where \begin{align*}\alpha\end{align*} is the level of significance for each \begin{align*}t\end{align*}-test and \begin{align*}c\end{align*} is the number of independent \begin{align*}t\end{align*}-tests. Using the example from the introduction, if our teacher conducted separate \begin{align*}t\end{align*}-tests to examine the means of the populations, she would have to conduct 10 separate \begin{align*}t\end{align*}-tests. If she performed these tests with \begin{align*}\alpha=0.05\end{align*}, the probability of committing a type I error is not 0.05 as one would initially expect. Instead, it would be 0.40, which is extremely high!
The Steps of the ANOVA Method
With the ANOVA method, we are actually analyzing the total variation of the scores, including the variation of the scores within the groups and the variation between the group means. Since we are interested in two different types of variation, we first calculate each type of variation independently and then calculate the ratio between the two. We use the \begin{align*}F\end{align*}-distribution as our sampling distribution and set our critical values and test our hypothesis accordingly.
When using the ANOVA method, we are testing the null hypothesis that the means and the variances of our samples are equal. When we conduct a hypothesis test, we are testing the probability of obtaining an extreme \begin{align*}F\end{align*}-statistic by chance. If we reject the null hypothesis that the means and variances of the samples are equal, and then we are saying that the difference that we see could not have happened just by chance.
To test a hypothesis using the ANOVA method, there are several steps that we need to take. These include:
1. Calculating the mean squares between groups, \begin{align*}MS_B\end{align*}. The \begin{align*}MS_B\end{align*} is the difference between the means of the various samples. If we hypothesize that the group means are equal, then they must also equal the population mean. Under our null hypothesis, we state that the means of the different samples are all equal and come from the same population, but we understand that there may be fluctuations due to sampling error. When we calculate the \begin{align*}MS_B\end{align*}, we must first determine the \begin{align*}SS_B\end{align*}, which is the sum of the differences between the individual scores and the mean in each group. To calculate this sum, we use the following formula:
\begin{align*}SS_B=\sum^m_{k=1} n_k (\bar{x}_k-\bar{x})^2\end{align*}
where:
\begin{align*}k\end{align*} is the group number.
\begin{align*}n_k\end{align*} is the sample size of group \begin{align*}k\end{align*}.
\begin{align*}\bar{x}_k\end{align*} is the mean of group \begin{align*}k\end{align*}.
\begin{align*}\bar{x}\end{align*} is the overall mean of all the observations.
\begin{align*}m\end{align*} is the total number of groups.
When simplified, the formula becomes:
\begin{align*}SS_B=\sum^m_{k=1} \frac{T^2_k}{n_k}-\frac{T^2}{n}\end{align*}
where:
\begin{align*}T_k\end{align*} is the sum of the observations in group \begin{align*}k\end{align*}.
\begin{align*}T\end{align*} is the sum of all the observations.
\begin{align*}n\end{align*} is the total number of observations.
Once we calculate this value, we divide by the number of degrees of freedom, \begin{align*}m-1\end{align*}, to arrive at the \begin{align*}MS_B\end{align*}. That is, \begin{align*}MS_B=\frac{SS_B}{m-1}\end{align*}
2. Calculating the mean squares within groups, \begin{align*}MS_W\end{align*}. The mean squares within groups calculation is also called the pooled estimate of the population variance. Remember that when we square the standard deviation of a sample, we are estimating population variance. Therefore, to calculate this figure, we sum the squared deviations within each group and then divide by the sum of the degrees of freedom for each group.
To calculate the \begin{align*}MS_W\end{align*}, we first find the \begin{align*}SS_W\end{align*}, which is calculated using the following formula:
\begin{align*}\frac{\sum(x_{i1}-\bar{x}_1)^2+\sum (x_{i2}-\bar{x}_2)^2+ \ldots + \sum (x_{im}-\bar{x}_m)^2}{(n_1-1)+(n_2-1)+ \ldots + (n_m-1)}\end{align*}
Simplified, this formula becomes:
\begin{align*}SS_W=\sum^m_{k=1} \sum^{n_k}_{i=1} x^2_{ik}-\sum^m_{k=1} \frac{T^2_k}{n_k}\end{align*}
where:
\begin{align*}T_k\end{align*} is the sum of the observations in group \begin{align*}k\end{align*}.
Essentially, this formula sums the squares of each observation and then subtracts the total of the observations squared divided by the number of observations. Finally, we divide this value by the total number of degrees of freedom in the scenario, \begin{align*}n-m\end{align*}.
\begin{align*}MS_W=\frac{SS_W}{n-m}\end{align*}
3. Calculating the test statistic. The formula for the test statistic is as follows:
\begin{align*}F=\frac{MS_B}{MS_W}\end{align*}
4. Finding the critical value of the \begin{align*}F\end{align*}-distribution. As mentioned above, \begin{align*}m-1\end{align*} degrees of freedom are associated with \begin{align*}MS_B\end{align*}, and \begin{align*}n-m\end{align*} degrees of freedom are associated with \begin{align*}MS_W\end{align*}. In a table, the degrees of freedom for \begin{align*}MS_B\end{align*} are read across the columns, and the degrees of freedom for \begin{align*}MS_W\end{align*} are read across the rows.
5. Interpreting the results of the hypothesis test. In ANOVA, the last step is to decide whether to reject the null hypothesis and then provide clarification about what that decision means.
The primary advantage of using the ANOVA method is that it takes all types of variations into account so that we have an accurate analysis. In addition, we can use technological tools, including computer programs, such as SAS, SPSS, and Microsoft Excel, as well as the TI-83/84 graphing calculator, to easily perform the calculations and test our hypothesis. We use these technological tools quite often when using the ANOVA method.
Example B
Let’s go back to the example in the introduction with the teacher who is testing multiple reading programs to determine the impact on student achievement. There are five different reading programs, and her 31 students are randomly assigned to one of the five programs. She collects the following data:
Method
\begin{align*}& 1 && 2 && 3 && 4 && 5 \\ & 1 && 8 && 7 && 9 && 10 \\ & 4 && 6 && 6 && 10 && 12 \\ & 3 && 7 && 4 && 8 && 9 \\ & 2 && 4 && 9 && 6 && 11 \\ & 5 && 3 && 8 && 5 &&8 \\ & 1 && 5 && 5 &&&&\\ & 6 && && 7 &&&&\\ & &&&& 5 &&&&\end{align*}
Compare the means of these different groups by calculating the mean squares between groups, and use the standard deviations from our samples to calculate the mean squares within groups and the pooled estimate of the population variance.
To solve for \begin{align*}SS_B\end{align*}, it is necessary to calculate several summary statistics from the data above:
\begin{align*}& \text{Number } (n_k) && 7 && 6 && 8 && 5 && 5 && 31\\ & \text{Total } (T_k) && 22 && 33 && 51 && 38 && 50 &&= 194\\ & \text{Mean } (\bar x) && 3.14 && 5.50 && 6.38 && 7.60 && 10.00 && = 6.26\\ & \text{Sum of Squared Obs. } \left (\sum_{i=1}^{n_k} x^2_{ik}\right ) && 92 && 199 && 345 && 306 && 510 && = 1,452\\ & \frac{\text{Sum of Obs. Squared }}{\text{Number of Obs}} \left (\frac {T_k^2}{n_k}\right ) && 69.14 && 181.50 && 325.13 && 288.80 && 500.00 && = 1,364.57\end{align*}
Using this information, we find that the sum of squares between groups is equal to the following:
\begin{align*}SS_B &= \sum^m_{k=1} \frac{T^2_k}{n_k}-\frac{T^2}{N}\\ & \approx 1, 364.57 - \frac{(194)^2}{31} \approx 150.5\end{align*}
Since there are four degrees of freedom for this calculation (the number of groups minus one), the mean squares between groups is as shown below:
\begin{align*}MS_B=\frac{SS_B}{m-1} \approx \frac{150.5}{4} \approx 37.6\end{align*}
Next, we calculate the mean squares within groups, \begin{align*}MS_W\end{align*}, which is also known as the pooled estimate of the population variance, \begin{align*}\sigma^2\end{align*}.
To calculate the mean squares within groups, we first use the following formula to calculate \begin{align*}SS_W\end{align*}:
\begin{align*}SS_W=\sum^m_{k=1} \sum^{n_k}_{i=1} x^2_{ik}-\sum^m_{k=1} \frac{T^2_k}{n_k}\end{align*}
Using our summary statistics from above, we can calculate \begin{align*}SS_W\end{align*} as shown below:
\begin{align*}SS_W &= \sum^m_{k=1} \sum^{n_k}_{i=1} x^2_{ik}-\sum^m_{k=1} \frac{T^2_k}{n_k}\\ & \approx 1, 452 - 1, 364.57\\ & \approx 87.43\end{align*}
This means that we have the following for \begin{align*}MS_W\end{align*}:
\begin{align*}MS_W=\frac{SS_W}{n-m} \approx \frac{87.43}{26} \approx 3.36\end{align*}
Therefore, our \begin{align*}F\end{align*}-ratio is as shown below:
\begin{align*}F=\frac{MS_B}{MS_W} \approx \frac{37.6}{3.36} \approx 11.19\end{align*}
We would then analyze this test statistic against our critical value. Using an F-distribution table for α=0.01 (equivalent to a two-tailed significance of 0.02), and also the numerator degrees of freedom of m-1=5-1=4 and the denominator degrees of freedom of m-n=31-5=26, we find our critical value equal to 4.140. Since our test statistic of 11.19 exceeds the critical value of 4.140, we reject the null hypothesis. We can conclude, therefore, that not all of the population means of the five programs are equal and that obtaining an F-ratio this extreme by chance is highly improbable.
Technology Note: Calculating a One-Way ANOVA with Excel
Here is the procedure for performing a one-way ANOVA in Excel using this set of data.
Example C
Perform a one-way ANOVA for the data in Example B, using Excel.
Solution:
Enter the table above (i.e., with data on how the 31 students divided into five groups performed on reading comprehension) into an empty Excel worksheet.
Click the Data choice on the toolbar, then select ’Data Analysis,’ and then choose ’Regression’ from the list that appears (Note, if Data Analysis does not appear as a choice on your Data page need to follow the add-in instructions below).
Place the cursor in the ’Input Y range’ field and select the third column.
Place the cursor in the ’Input X range’ field and select the first and second columns.
Place the cursor in the ’Output Range’ field and click somewhere in a blank cell below and to the left of the table.
Click ’Labels’ so that the names of the predictor variables will be displayed in the table.
Click 'OK', and the results shown below will be displayed.
Note: In Excel 2007, to add Data Analysis to your Data page, perform the following functions. Click the Microsoft Office Button in the upper left, then click on Excel Options. Click on Add-ins, then highlight the Analysis ToolPak, click Go, make sure the Analysis ToolPak box is checked off, and then click OK. The Data Analysis choice should now appear on your Excel Data page. Follow the remaining instructions above.
Anova: Single Factor
Groups | Count | Sum | Average | Variance |
---|---|---|---|---|
Column 1 | 7 | 22 | 3.142857 | 3.809524 |
Column 2 | 6 | 33 | 5.5 | 3.5 |
Column 3 | 8 | 51 | 6.375 | 2.839286 |
Column 4 | 5 | 38 | 7.6 | 4.3 |
Column 5 | 6 | 50 | 10 | 2.5 |
Source of Variation | \begin{align*}SS\end{align*} | \begin{align*}df\end{align*} | \begin{align*}MS\end{align*} | \begin{align*}F\end{align*} | \begin{align*}P\end{align*}-value | \begin{align*}F\end{align*} crit |
---|---|---|---|---|---|---|
Between Groups | 150.5033 | 4 | 37.62584 | 11.18893 | 2.05e-05 | 2.742594 |
Within Groups | 87.43214 | 26 | 3.362775 | |||
Total | 237.9355 | 30 |
On the Web
http://preview.tinyurl.com/36j4by6 \begin{align*}F\end{align*}-distribution tables with \begin{align*}\alpha=0.02\end{align*}.
Guided Practice
A panel of testers judged the flavor quality of different frozen vanilla desserts and measured them on a scale of 0 to 100. The data are from a Consumer Reports article “Low-fat frozen desserts: Better for you than ice cream?” (August, 1992). Here is most of the ANOVA output from the computer:
Source | \begin{align*}df\end{align*} | \begin{align*}SS\end{align*} | \begin{align*}MS\end{align*} | \begin{align*}F\end{align*} | \begin{align*}P\end{align*}-value |
---|---|---|---|---|---|
Type | 6364 | 3182 | |||
Error | 24 | 3031 | 126 | ||
Total | 9395 |
a) State the null and alternative hypotheses.
b) Complete the ANOVA table giving the F-Statistic, degrees of freedom and approximating the p-value.
c) What is your conclusion about the flavor quality of the different frozen vanilla desserts?
Solutions:
a. The null hypothesis is that the flavor quality of the different frozen vanilla desserts is the same. The alternative hypothesis is that the flavor qualities among the different frozen vanilla desserts are not all the same.
b. We can find the following and add them to the table:
The SS/MS = degrees of freedom
The total degrees of freedom = df type + df error
The F statistic is MSType/MSError
Source | \begin{align*}df\end{align*} | \begin{align*}SS\end{align*} | \begin{align*}MS\end{align*} | \begin{align*}F\end{align*} | \begin{align*}P\end{align*}-value |
---|---|---|---|---|---|
Type | 2 | 6364 | 3182 | 24.8 | 0 |
Error | 24 | 3031 | 126 | ||
Total | 26 | 9395 |
c. Based on the p-value (less than 0.01) we reject the null hypothesis and claim that the flavor qualities of the different frozen vanilla desserts are not all the same.
Explore More
- What does the ANOVA acronym stand for?
- If we are testing whether pairs of sample means differ by more than we would expect due to chance using multiple \begin{align*}t\end{align*}-tests, the probability of making a type I error would ___.
- In the ANOVA method, we use the ___ distribution.
- Student’s \begin{align*}t\end{align*}-
- normal
- \begin{align*}F\end{align*}-
- In the ANOVA method, we complete a series of steps to evaluate our hypothesis. Put the following steps in chronological order.
- Calculate the mean squares between groups and the mean squares within groups.
- Determine the critical values in the \begin{align*}F\end{align*}-distribution.
- Evaluate the hypothesis.
- Calculate the test statistic.
- State the null hypothesis.
- A school psychologist is interested in whether or not teachers affect the anxiety scores among students taking the AP Statistics exam. The data below are the scores on a standardized anxiety test for students with three different teachers.
Ms. Jones | Mr. Smith | Mrs. White |
---|---|---|
8 | 23 | 21 |
6 | 11 | 21 |
4 | 17 | 22 |
12 | 16 | 18 |
16 | 6 | 14 |
17 | 14 | 21 |
12 | 15 | 9 |
10 | 19 | 11 |
11 | 10 | |
13 |
(a) State the null hypothesis.
(b) Using the data above, fill out the missing values in the table below.
Ms. Jones | Mr. Smith | Mrs. White | Totals | |
---|---|---|---|---|
Number \begin{align*}(n_k)\end{align*} | 8 | \begin{align*}=\end{align*} | ||
Total \begin{align*}(T_k)\end{align*} | 131 | \begin{align*}=\end{align*} | ||
Mean \begin{align*}(\bar{x})\end{align*} | 14.6 | \begin{align*}=\end{align*} | ||
Sum of Squared Obs. \begin{align*}(\sum^{n_k}_{i=1} x^2_{ik})\end{align*} | \begin{align*}=\end{align*} | |||
Sum of Obs. Squared/Number of Obs. \begin{align*}(\frac{T^2_k}{n_k})\end{align*} | \begin{align*}=\end{align*} |
(c) What is the value of the mean squares between groups, \begin{align*}MS_B\end{align*}?
(d) What is the value of the mean squares within groups, \begin{align*}MS_W\end{align*}?
(e) What is the \begin{align*}F\end{align*}-ratio of these two values?
(f) With \begin{align*}\alpha=0.05\end{align*}, use the \begin{align*}F\end{align*}-distribution to set a critical value.
(g) What decision would you make regarding the null hypothesis? Why?
- What are the assumptions of the ANOVA test? That is, what condition are the data supposed to meet?
- In each situation, determine whether on-way analysis of variance is an appropriate method for analyzing the data describe. Explain you response.
- A researcher compares the mean blood pressures of women over 60 years old for four different ethnic groups. She samples 200 women in each ethnic group, and measures their blood pressure.
- 100 individuals all listen to six songs and rate each song on a scale of 1 to 10. The mean scores for the six songs are compared.
- A researcher is interested in studying the connection between body mass index and age and report the following: “The p-value was 0.04 for a one-way analysis of variance done to compare body mass index values for the four age groups.”
- What was the null hypothesis that the researcher was testing? Write the hypothesis in words and using statistical symbols.
- Explain the conclusion that can be made about the comparison between body mass index values for the four age groups.
- A researcher was interested in comparing a students height with their preferred choice of seating in a classroom – front, middle or back. The p-value for an F-test that compared the mean heights of the students in the three different seating locations was 0.002. In the context of this situation what conclusion can be drawn?
- For each of the following situations use an F table to find the critical value and then state a conclusion for an F-test of the null hypothesis of equal population means.
- F statistic = 6.27 with 2 and 12 degrees of freedom;
- F-statistic = 3.27, with 3 and 20 degrees of freedom
- Give a value for each of the missing elements in the following ANOVA table:
Source | \begin{align*}Df\end{align*} | \begin{align*}SS\end{align*} | \begin{align*}MS\end{align*} | \begin{align*}F\end{align*} |
---|---|---|---|---|
Between groups | 5 | 40 | 8 | |
Error | xxxxxxxxx | |||
Total | 15 | 100 | xxxxxxxxx | xxxxxxxxx |
- Give a value for each of the missing elements in the following ANOVA table:
Source | \begin{align*}Df\end{align*} | \begin{align*}SS\end{align*} | \begin{align*}MS\end{align*} | \begin{align*}F\end{align*} |
---|---|---|---|---|
Between groups | 2 | 10 | ||
Error | 28 | 290 | xxxxxxxxx | |
Total | xxxxxxxxx | xxxxxxxxx |
- What is the purpose of the analysis of variance test?
- Given 5 samples with sample variances: 2.0, 2.2, 2.35, 2.41 and 2.45. Calculate the within–sample variance. There are ten observations in each sample.
- Given the means of 5 samples: 210, 212, 213, 214 and 214. Calculate the sum of squares between groups? There are ten observations in each sample.
- Use the information from problems 10 and 11 to calculate the F-statistic.
- If there are k number of populations and n number of data values, then what is the degree of freedom of the within-sample variance? What is the degree of freedom of the between-sample variance?
- List three properties of the F probability distribution.
- Workers on a tree farm are interested in testing three fertilizer mixtures on the growth of maple seedlings. They randomly select seedling for each type of fertilizer and measured the height of the seedlings, in feet. The summary data are below:
Fertilizer | N | \begin{align*}\bar{x}\end{align*} | s |
---|---|---|---|
A | 9 | 3.01 | 0.72 |
B | 19 | 2.15 | .61 |
C | 11 | 2.21 | .60 |
Conduct a complete ANOVA test at the 5% level of significance to determine if there are differences among the population mean seedling height for the different types of fertilizer.
- A study was conducted to examine the clinical usefulness of a new antidepressant. Depressed people were randomly assigned to one of three treatment groups: a moderate dose, a low dose and no dose (placebo). After a month the subjects completed a depression inventory (the higher the score, the more depressed). Below is the data.
Placebo | Low Dose | Moderate Dose |
---|---|---|
38 | 22 | 14 |
47 | 19 | 26 |
39 | 8 | 11 |
25 | 23 | 18 |
41 | 31 | 5 |
a) What is the null hypothesis in this study?
b) What is the alternative hypothesis?
c) Compute the appropriate test.
d) What significance level did you choose and why?
e) What is the critical F value?
f) What is your conclusion?
- House color and people’s stay (in years)
Blue | Green | Peach |
---|---|---|
8 | 11 | 4 |
7 | 9 | 8 |
3 | 7 | 9 |
1 | 18 | 2 |
9 | 12 | 4 |
Complete the following table:
SS | df | ms | |
---|---|---|---|
Between | |||
Within | |||
Total | |||
F = |
a) What is the null hypothesis?
b) What is the alternative hypothesis?
c) What is the F critical value?
d) What is your decision?
- Conduct an ANOVA for the scores for the following 4 groups. Do these groups differ significantly from one another at alpha = 0.05?
group 1 | group 2 | group 3 | group 4 |
---|---|---|---|
2 | 5 | 5 | 7 |
4 | 6 | 5 | 9 |
3 | 7 | 11 | 11 |
Technology Notes:
One-Way ANOVA on the TI-83/84 Calculator
Enter raw data from population 1 into L1, population 2 into L2, population 3 into L3, population 4 into L4, and so on.
Now press [STAT], scroll right to TESTS, scroll down to 'ANOVA(', and press [ENTER]. Then enter the lists to produce a command such as 'ANOVA(L1, L2, L3, L4)' and press [ENTER].
Keywords
ANOVA method
\begin{align*}F\end{align*}-distribution
Grand mean
Mean squares between groups
Mean squares within groups
Pooled estimate of the population variance