License: CC BY-NC 3.0
The eighth graders are conducting a survey. They want to know how many people enjoy pizza versus chicken. Karen is writing the survey. She only handed it out to all of the girls in the class. Her best friend Kevin said that she can’t do this because the results will be skewed. Karen doesn’t understand why this would be the case. Can you help?
In this concept, you will learn to understand bias in surveys and samples.
A key concern in sampling methods is getting a representative group of the entire population. This gives the study validity; it helps people to believe what the results of the study say. However, many sampling methods are biased. They give an unfair preference to a certain group or exclude a certain segment of the population which gives us less confidence in the results.
Any sampling method that favors one group, or gives a group a smaller likelihood of participating, is biased.
Let’s look at an example.
A biologist measures the growth of plants but only samples plants near the entrance because she cannot reach plants in the middle of the greenhouse.
The plant growth may not have been the same near the door as in other parts of the greenhouse. She used convenience sampling which is not always the best choice for a sample.
Bias is when one group of people or subjects is targeted more than another group. This provides only a specific view of the situation. Survey questions can reveal bias in the survey itself. Sometimes the people who create surveys hope for certain results and create questions to steer the answers. At other times, there are inadvertent cultural biases based on religion, language, age, economic level, etc.
You can learn to spot potential bias in survey questions by looking for questions that exclude a particular group or only include specific groups.
Can you find bias in the following questions?
1. When you visited the restroom, was the cleanliness a) bad, b) okay, or c) good?
Bias: There is the assumption that the person visited the bathroom.
2. At what time of the day do you usually use your swimming pool?
Bias: There is the assumption that the person has a swimming pool.
3. Which do you think is the most powerful book in the Bible?
Bias: There is the assumption that people belong to a certain religion and are familiar with the Bible.
4. What was more important to the history of America, the Emancipation Proclamation or Women’s Suffrage?
Bias: There is the assumption that people are familiar with these issues from U.S. history and that they understand these concepts.
5. Do you think you should go to church every Sunday?
Bias: There is the assumption that people believe in these ideas.
If a person taking a survey does not feel like the options available for a question do not accurately represent his or her true response, a bias in the survey has occurred.
As well, some people may not be willing to tell the truth, for one reason or another. If a person is asked to identify themselves and then reveal confidential or personal information, they may not answer truthfully.
Earlier, you were given a problem about the survey on pizza vs. chicken. Karen is surveying eighth graders to see how many enjoy pizza versus chicken. She handed out the survey to all of the girls in the class and Kevin said her survey results would be skewed.
Kevin is right. He told Karen that the survey results would be skewed because she only asked the girls their opinion about pizza vs. chicken. Karen needs to ask the boys too since the survey are about the eighth grade not the “girls” in the eighth grade. Asking both boys and girls is the only way to ensure successful results.
A school polls parents about traffic congestion in the morning. They ask the parents of every 3rdcar in the school drop-off area before the school bell rings to rate the traffic. Is this sampling method the best one? Is it biased?
The answer is yes, the sampling method is biased. If there is traffic congestion, some people may be arriving late. This sampling method is biased because it only includes people who arrive on time. Their opinions may be different from those that arrive late. Therefore, this is not the best way to gather a sampling.
Is a survey on dog food preference bias where only people who own big dogs are surveyed?
The answer is yes, this survey will be biased. The survey should include people who own big and small dogs.
Is a survey about the number of children who like chocolate ice cream bias if it is conducted by asking all children who come to the ice cream stand? The survey is conducted for three days.
The answer is no. This will be a fair survey.
Is a survey about the number of tenth graders who babysit bias if it is conducted by randomly selecting boys and girls in the tenth grade?
The answer is no. This will be a fair survey.
Are the following survey questions biased?
1. How old is your spouse?
2. How many times do you go to the park each month? a) 1-2, b) 3-5, c) 6-10, or d) more than 10
3. Which is more important, the First or Second Amendment of the Constitution?
4. Don’t you agree that equality for all Americans is important?
5. Don’t you think all people should drive a car?
6. How do you feel about people who are vegetarians?
7. How do you feel about your boss at work?
Look at the following survey and then answer each question.
A survey was conducted at a veterinarian’s office. For one week, all of the clients were surveyed. They were asked if they enjoy the management of the office. The week that the survey was conducted, 12 out of 80 clients had appointments. The overall result is that the clients love the way the office is managed.
8. Is this a biased survey?
9. Why or why not?
10. Is there a statistic that supports this conclusion?
11. Do you believe that the overall result of the survey is accurate?
12. Why or why not?
13. How could this survey have been conducted to reach more of the clients?
14. What kind of sampling was conducted?
15. Would a random sample work better for this survey?
To see the Review answers, open this PDF file and look for section 10.11.