Tests of Independence and Homogeneity
As mentioned in the previous lesson, the chi-square test can be used to both estimate how closely an observed distribution matches an expected distribution (the goodness-of-fit test) and to estimate whether two random variables are independent of one another (the test of independence). In this lesson, we will examine the test of independence in greater detail.
The chi-square test of independence is used to assess if two factors are related. This test is often used in social science research to determine if factors are independent of each other. For example, we would use this test to determine relationships between voting patterns and race, income and gender, and behavior and education.
In general, when running the test of independence, we ask, “Is Variable \begin{align*}X\end{align*} independent of Variable \begin{align*}Y\end{align*}?” It is important to note that this test does not test how the variables are related, just simply whether or not they are independent of one another. For example, while the test of independence can help us determine if income and gender are independent, it cannot help us assess how one category might affect the other.
Drawing Data from Contingency Tables Needed to Perform Calculations when Running a Chi-Square Test
Contingency tables can help us frame our hypotheses and solve problems. Often, we use contingency tables to list the variables and observational patterns that will help us to run a chi-square test. For example, we could use a contingency table to record the answers to phone surveys or observed behavioral patterns.
We would use a contingency table to record the data when analyzing whether women are more likely to vote for a Republican or Democratic candidate when compared to men. In this example, we want to know if voting patterns are independent of gender. Hypothetical data for 76 females and 62 males from the state of California are in the contingency table below.
Democratic | Republican | Total | |
---|---|---|---|
Female | 48 | 28 | 76 |
Male | 36 | 26 | 62 |
Total | 84 | 54 | 138 |
Similar to the chi-square goodness-of-fit test, the test of independence is a comparison of the differences between observed and expected values. However, in this test, we need to calculate the expected value using the row and column totals from the table. The expected value for each of the potential outcomes in the table can be calculated using the following formula:
\begin{align*}\text{Expected Frequency}=\frac{(\text{Row Total})(\text{Column Total})}{\text{Total Number of Observations}}\end{align*}
In the table above, we calculated the row totals to be 76 females and 62 males, while the column totals are 84 Democrats and 54 Republicans. Using the formula, we find the following expected frequencies for the potential outcomes:
The expected frequency for female Democratic outcome is \begin{align*}76 \bullet \frac{84}{138} = 46.26\end{align*}.
The expected frequency for female Republican outcome is \begin{align*}76 \bullet \frac{54}{138} = 29.74\end{align*}.
The expected frequency for male Democratic outcome is \begin{align*}62 \bullet \frac{84}{138} = 37.74\end{align*}.
The expected frequency for male Republican outcome is \begin{align*}62 \bullet \frac{54}{138} = 24.26\end{align*}.
Using these calculated expected frequencies, we can modify the table above to look something like this:
Democratic | Democratic | Republican | Republican | Total | |
---|---|---|---|---|---|
Observed | Expected | Observed | Expected | ||
Female | 48 | 46.26 | 28 | 29.74 | 76 |
Male | 36 | 37.74 | 26 | 24.26 | 62 |
Total | 84 | 54 | 138 |
With the figures above, we are able to calculate the chi-square statistic with relative ease.
The Chi-Square Test of Independence
When running the test of independence, we use similar steps as when running the goodness-of-fit test described earlier. First, we need to establish a hypothesis based on our research question.
Consider again the example of gender and voting patterns. In this case, our null hypothesis is that there is not a significant difference in the frequencies with which females vote for a Republican or Democratic candidate when compared to males. Therefore, our hypotheses can be stated as follows:
Null Hypothesis
\begin{align*}H_0:O=E\end{align*} (There is no statistically significant difference between the observed and expected frequencies.)
Alternative Hypothesis
\begin{align*}H_a:O \neq E\end{align*} (There is a statistically significant difference between the observed and expected frequencies.)
Using the table above, we can calculate the degrees of freedom and the chi-square statistic. The formula for calculating the chi-square statistic is the same as before:
\begin{align*}\chi^2=\sum_{} \frac{(O_{}-E_{})^2}{E_{}}\end{align*}
where:
\begin{align*}\chi^2\end{align*} is the chi-square test statistic.
\begin{align*}O_{}\end{align*} is the observed frequency value for each event.
\begin{align*}E_{}\end{align*} is the expected frequency value for each event.
Using this formula and the example above, we get the following expected frequencies and chi-square statistic:
Democratic | Democratic | Democratic | Republican | Republican | Republican | |
---|---|---|---|---|---|---|
Obs. Freq. | Exp. Freq. | \begin{align*}\frac{(O-E)^2}{E}\end{align*} | Obs. Freq. | Exp. Freq. | \begin{align*}\frac{(O-E)^2}{E}\end{align*} | |
Female | 48 | 46.26 | 0.07 | 28 | 29.74 | 0.10 |
Male | 36 | 37.74 | 0.08 | 26 | 24.26 | 0.12 |
Totals | 84 | 54 |
\begin{align*}\chi^2=0.07+0.08+0.10+0.12=0.37\end{align*}
Also, the degrees of freedom can be calculated from the number of Columns ("C") and the number of Rows ("R") as follows:
\begin{align*}df &= (C-1)(R-1)\\ &= (2-1)(2-1)=1\end{align*}
With an alpha level of 0.05, we look under the column for 0.05 and the row for degrees of freedom, which, again, is 1, in the standard chi-square distribution table. According to the table, we see that the critical value for chi-square is 3.841. Therefore, we would reject the null hypothesis if the chi-square statistic is greater than 3.841.
Since our calculated chi-square value of 0.37 is less than 3.841, we fail to reject the null hypothesis. Therefore, we can conclude that females are not significantly more likely to vote for a Republican or Democratic candidate than males. In other words, these two factors appear to be independent of one another.
The Test of Homogeneity
The chi-square goodness-of-fit test and the test of independence are two ways to examine the relationships between categorical variables. To determine whether or not the assignment of categorical variables is random (that is, to examine the randomness of a sample), we perform the test of homogeneity. In other words, the test of homogeneity tests whether samples from populations have the same proportion of observations with a common characteristic. For example, we found in our last test of independence that the factors of gender and voting patterns were independent of one another. However, our original question was if females were more likely to vote for a Republican or Democratic candidate when compared to males. We would use the test of homogeneity to examine the probability that choosing a Republican or Democratic candidate was the same for females and males.
Another commonly used example of the test of homogeneity is comparing dice to see if they all work the same way.
Performing a Test of Homogeneity
The manager of a casino has two potentially loaded dice that he wants to examine. (Loaded dice are ones that are weighted on one side so that certain numbers have greater probabilities of showing up.) The manager rolls each of the dice exactly 20 times and comes up with the following results:
1 | 2 | 3 | 4 | 5 | 6 | Totals | |
---|---|---|---|---|---|---|---|
Die 1 | 6 | 1 | 2 | 2 | 3 | 6 | 20 |
Die 2 | 4 | 1 | 3 | 3 | 1 | 8 | 20 |
Totals | 10 | 2 | 5 | 5 | 4 | 14 | 40 |
Like the other chi-square tests, we first need to establish a null hypothesis based on a research question. In this case, our research question would be something like, “Is the probability of rolling a specific number the same for Die 1 and Die 2?” This would give us the following hypotheses:
Null Hypothesis
\begin{align*}H_0:O=E\end{align*} (The probabilities are the same for both dice.)
Alternative Hypothesis
\begin{align*}H_a:O \neq E\end{align*} (The probabilities differ for both dice.)
Similar to the test of independence, we need to calculate the expected frequency for each potential outcome and the total number of degrees of freedom. To get the expected frequency for each potential outcome, we use the same formula as we used for the test of independence, which is as follows:
\begin{align*}\text{Expected Frequency}=\frac{(\text{Row Total})(\text{Column Total})}{\text{Total Number of Observations}}\end{align*}
The following table includes the expected frequency (in parenthesis) for each outcome, along with the chi-square statistic, \begin{align*}\chi^2=\frac{(O-E)^2}{E}\end{align*}, in a separate column:
Number Rolled on the Potentially Loaded Dice
1 | \begin{align*}\chi^2\end{align*} | 2 | \begin{align*}\chi^2\end{align*} | 3 | \begin{align*}\chi^2\end{align*} | 4 | \begin{align*}\chi^2\end{align*} | 5 | \begin{align*}\chi^2\end{align*} | 6 | \begin{align*}\chi^2\end{align*} | \begin{align*}\chi^2\end{align*} Total | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Die 1 | 6(5) | 0.2 | 1(1) | 0 | 2(2.5) | 0.1 | 2(2.5) | 0.1 | 3(2) | 0.5 | 6(7) | 0.14 | 1.04 |
Die 2 | 4(5) | 0.2 | 1(1) | 0 | 3(2.5) | 0.1 | 3(2.5) | 0.1 | 1(2) | 0.5 | 8(7) | 0.14 | 1.04 |
Totals | 10 | 2 | 5 | 5 | 4 | 14 | 2.08 |
\begin{align*}df &= (C-1)(R-1)\\ &= (6-1)(2-1)=5\end{align*}
From the table above, we can see that the value of the test statistic is 2.08.
Using an alpha level of 0.05, we look under the column for 0.05 and the row for degrees of freedom, which, again, is 5, in the standard chi-square distribution table. According to the table, we see that the critical value for chi-square is 11.070. Therefore, we would reject the null hypothesis if the chi-square statistic is greater than 11.070.
Since our calculated chi-square value of 2.08 is less than 11.070, we fail to reject the null hypothesis. Therefore, we can conclude that each number is just as likely to be rolled on one die as on the other. This means that if the dice are loaded, they are probably loaded in the same way or were made by the same manufacturer.
Example
Example 1
A drug trial is conducted on a group of animals and the researchers hypothesize that the animals receiving the drug will survive better than those that did not receive the drug. The following data is collected:
Dead | Alive | |
---|---|---|
Treated | 36 | 14 |
Not Treated | 30 | 25 |
Test the hypothesis that survival of the animals is independent of drug treatment at the 0.05 level of significance.
Set up two matrices on the calculator: A is the matrix of observed and B is the matrix of expected under the null hypothesis. To determine the matrix of expected, for each cell multiply the row total by the column total and divide by the grand total.
Dead | Alive | |
---|---|---|
Treated | 36 | 14 |
Not Treated | 30 | 25 |
Dead | Alive | |
---|---|---|
Treated | 31.4 | 18.57 |
Not Treated | 34.57 | 25 |
On the calculator, stat – test – chi-square – enter will automatically know to read matrices A and B.
Chi-square statistic is 3.417 with one degree of freedom and the p-value is 0.065. At the 0.05 level of significance, we fail to reject the null hypothesis (since the p-value is greater than 0.05) and believe that animals receiving the drug do not necessarily survive better than those not receiving the drug.
Review
- What is the chi-square test of independence used for?
- True or False: In the test of independence, you can test if two variables are related, but you cannot test the nature of the relationship itself.
- When calculating the expected frequency for a possible outcome in a contingency table, you use the formula:
- \begin{align*}\text{Expected Frequency} = \frac{(\text{Row Total})(\text{Column Total})}{\text{Total Number of Observations}}\end{align*}
- \begin{align*}\text{Expected Frequency} = \frac{(\text{Total Observations})(\text{Column Total})}{\text{Row Total}}\end{align*}
- \begin{align*}\text{Expected Frequency} = \frac{(\text{Total Observations})(\text{Row Total})}{\text{Column Total}}\end{align*}
- Use the table below to answer the following review questions.
Studied Abroad | Did Not Study Abroad | |
---|---|---|
Females | 322 | 460 |
Males | 128 | 152 |
A. What is the total number of females in the sample?
- 450
- 280
- 612
- 782
B. What is the total number of observations in the sample?
- 782
- 533
- 1,062
- 612
C. What is the expected frequency for the number of males who did not study abroad?
- 161
- 208
- 111
- 129
D. How many degrees of freedom are in this example?
- 1
- 2
- 3
- 4
E. True or False: Our null hypothesis would be that females are as likely as males to study abroad.
F. What is the chi-square statistic for this example?
- 1.60
- 2.45
- 3.32
- 3.98
- If the chi-square critical value at 0.05 and 1 degree of freedom is 3.81, and we have a calculated chi-square statistic of 2.22, we would:
- reject the null hypothesis
- fail to reject the null hypothesis
- True or False: We use the test of homogeneity to evaluate the equality of several samples of certain variables.
- The test of homogeneity is carried out the exact same way as:
- the goodness-of-fit test
- the test of independence
- Suppose you have the following data:
Asia | Africa | South America | |
---|---|---|---|
Malaria A | 31 | 14 | 45 |
Malaria B | 2 | 5 | 53 |
Malaria C | 53 | 45 | 2 |
Test the hypothesis that the incidence of malaria is independent of region.
- Are class attendance and course performance related? Use the following data to answer this question.
Number of Days Missed | A | B | C | D | F |
---|---|---|---|---|---|
0 – 2 | 36 | 60 | 65 | 6 | 1 |
3 - 4 | 20 | 50 | 60 | 12 | 6 |
4 -5 | 10 | 30 | 30 | 18 | 7 |
6 -10 | 4 | 8 | 15 | 20 | 8 |
>11 | 0 | 2 | 10 | 14 | 8 |
- A manufacturer was interested in selling crackers that were high in a particular kind of edible fiber as a dieting aid. Twelve females were fed a controlled diet. Before each meal they ate crackers containing either bran fiber, gum fiber, a combination of the two types of fiber or no fiber. Their caloric intake was monitored. Use the data at http://lib.stat.cmu.edu/DASL/Datafiles/Fiber.html To test whether bloating is independent of cracker.
- Consider the data at http://lib.stat.cmu.edu/DASL/Datafiles/Educationbyage.html
- Construct a two-way table of educational attainment by age.
- Which age category has the highest percentage of college graduates?
- Perform an appropriate hypothesis test for determining whether age category and educational attainment are independent.
- The following two questions were posed in a study: “How do you feel about allowing legal immigrants from other countries, who are here legally, to receive welfare in the U.S.? Are you for or against this?” and “How do you feel about providing public education to the children of illegal immigrants who are in this country illegally? Are you for or against this?” The rows represent the first question and the columns represent the second question:
For | Against | |
---|---|---|
For | 215.4 | 192.1 |
Against | 109.6 | 198.3 |
a. Is there an obvious choice for which variable should be the explanatory variable in this situation?
b. State null and alterative hypotheses about the two variables that have been used to create the contingency table.
c. Do a chi-square test of the hypotheses stated in part b. State a conclusion and support your conclusion with a p-value.
d. State the null and alternative hypotheses for this contingency table.
e. Do a chi-square test of the hypotheses stated in part a. State a conclusion and support this with a p-value.
- Consider the following two-way table of grade vs. number of lessons completed:
<8 | 9 – 15 | 16 | |
---|---|---|---|
A | 3 | 9 | 25 |
B | 6 | 11 | 18 |
C | 12 | 14 | 5 |
D or F | 14 | 10 | 2 |
Does the data suggest that course grade and number of modules completed are independent?
- Is the level of skier independent of the best ski area? Use the following results of a survey to answer this question.
Ski Area | Beginner | Intermediate | Advanced |
---|---|---|---|
A | 25 | 35 | 45 |
B | 15 | 35 | 65 |
C | 15 | 45 | 55 |
- Is there a relationship between the size of car an individual owns and the number of people in the driver’s family? Suppose that 800 car owners were surveyed with the following results:
Family Size | Compact | Mid-size | Full-size | Van |
---|---|---|---|---|
1 | 20 | 35 | 40 | 35 |
2 | 20 | 50 | 70 | 80 |
3-4 | 20 | 50 | 100 | 90 |
5+ | 20 | 30 | 70 | 70 |
Conduct a test for independence.
- Suppose 300 college students are surveyed as to their college major and their starting salaries. Below are the data. Conduct a test for independence.
Major | <30,000 | 30,000-39,999 | 40,000+ |
---|---|---|---|
English | 5 | 20 | 5 |
Engineering | 10 | 40 | 50 |
Nursing | 10 | 15 | 15 |
Business | 10 | 25 | 25 |
Psychology | 15 | 40 | 15 |
- True or False: (if false rewrite so it is true). The degrees of freedom for a test of independence are equal to the sample size minus 1.
- True of False (if false rewrite so it is true). The test for Independence uses tables of observed and expected data values.
- True or False: (if false rewrite so it is true). In a Test of Independence, the expected number is equal to the row total multiplied by the column total divided by the total surveyed.
Review (Answers)
To view the Review answers, open this PDF file and look for section 10.2.