- Know the terminology of data collection, variables, and measurement
- Understand how measurements are used in statistics
- Distinguish between the various methods for data collection
Data and Variables
When a topic needs to be studied or a question needs to be answered, researchers often collect data in an effort to find the answer. Data is a collection of facts, measurements, or observations about a set of individuals (data is plural, the word datum refers to a single observation). There are a variety of ways to collect data in order to study topics of interest. Researchers can analyze and compare test scores for various Minnesota High Schools. Scientists can conduct an experiment to determine the effectiveness of a new medication. Union leaders can conduct a census of every union member before deciding to strike. Or market researchers can survey a randomly selected sample of teenage girls to determine what qualities they look for when purchasing a new cell phone.
When a topic is being studied, there are often several variables, or characteristics about the individuals, that the researchers are interested in. Each person, animal, or object being studies is one individual (or subject). The variables can be either categorical or numerical. A categorical variable (or qualitative variable) can be put into categories, like favorite colors, type of car, etc. Whereas a numerical variable (or quantitative variable) can be assigned a numerical value, such as heights, distances, temperatures, etc.
Suppose 1845 teenage girls are to be surveyed by a cell phone company that wants to design a new cell phone that they can successfully market to females under 20 years old. The questionnaire will likely include questions related to: age, birth date, race, area code where they live, number of texts sent per month, amount of money willing to spend per month, services they want offered, features they want included, length of time they have had a cell phone, favorite colors, etc. All of these are variables, because they will vary from individual to individual. However, only some of these variables are numerical. Identify the individuals and the numerical variables.
Individuals: each girl who completes a questionnaire
Numerical Variables are: age, number of texts per month, amount of money willing to spend per month, length of time they have had a cell phone.
When determining which variables are numerical, it might help to decide whether or not it would be appropriate to calculate an average, or the range for the reported data. Age is numerical, because we can certainly report an average age of those surveyed. Even though birth date and area code may be reported as numbers, it would make no sense to report an 'average birth date' or 'mean area code'. Numbers such as these, social security numbers, or student ID numbers divide the data into a bunch of categories of one item each. They are simply used for identification and are considered categorical variables.
Measurement in Statistics
When a topic is to be studied the researchers decide what it is they want to know about each individual. These variables of interest can be measured using different instruments and need to be reported in specific units. The instrument is the tool used to make the measurement. This instrument could be something obvious like a scale, tape measure, thermometer, or speedometer. But, it could also be a something like a questionnaire, a rubric, or an exam. The units explain what the numbers represent, and might be feet, points, pounds, degrees Celsius, miles per hour, etc. Keep in mind that data is useless unless it is in context. For example, the number 12 could mean anything. Is it $12, or 12 inches, or 12 (in $1,000), or 12 apple pies? Without knowing the units, all you have is a meaningless list of numbers.
Validity, Reliability and Bias
The way in which any given variable is to be measured needs to be valid and reliable. Validity refers to the appropriateness of the instrument and units used. Reliability means that the instrument can be depended upon to consistently give the same measurement (or nearly the same). If an instrument gives different results when measuring the same thing, it is not reliable, and it has a lot of variability (because the results vary a lot). Another potential problem with measurements is bias. When a measurement is repeatedly too high or too low, it is said to be biased. In other words, a biased measurement is 'consistently wrong in the same direction'.
Researchers would like to limit bias in measurements as much as possible. Ideally, we hope for measurements that are valid, low in bias, and highly reliable. No measurement is perfect or necessarily accurate. Averaging repeated measurements can be a way to limit variability. Be aware though, averaging will only reduce variability (or increase reliability). Averages will not make an in invalid measurement suddenly valid. And, the average of biased measurements will still be biased.
For example, if the variable being studied is the weight of all of the members of the school wrestling team, then using a scale as the instrument and pounds as the units will be valid. And, as long as the scale being used is in working order, then the measurements reported should be reliable.
However, what if someone had set the scale being used to weigh the wrestlers to start at 10 pounds rather than zero? Each person who stepped on the scale would think that they were 10 pounds heavier than they actually were, resulting in biased measurements. If that were the case, using the scale as the instrument and pounds as the units would be valid (makes sense as a way to measure weight); reliable (if the same person steps on the scale again and again, they will have nearly the same result); and biased (each measurement is 10 pounds too heavy). So, even though something is wrong with this measurement, it doesn't mean that everything is wrong with it. We want valid, reliable, and unbiased measurements.
Suppose that a teacher intends to base grades in a math class on the students' heights. She plans to use a tape measure as her instrument and inches as her units. Her grading system will be as follows: the shortest student will receive the lowest grades and the tallest will receive the highest grades. Comment on the validity, reliability, and potential bias of this.
Validity? This clearly is not a valid way to measure a student's success and assign grades, because height has absolutely nothing to do with someone's understanding of math, or grade in a math course.
Reliability? The tape measure should be reliable. If used properly, each time a particular student's height is measured we will expect to get the same answer.
Bias? This should not be biased. Some tall people will deserve higher grades, while some will deserve lower grades. The same will be true for students of all heights.
Therefore, this teacher's method for assigning grades would be unbiased and it would be reliable (both good things), but it would also not be valid (a bad thing). She should come up with a better way to measure students' grades. Perhaps a combination of test scores and homework completion.
So, keep in mind that just because a statistical measurement is bad, does not mean that everything will be wrong with it. It is important to think through each question separately: Is it valid?; Is it reliable?; Is it unbiased?
Rates versus Counts
Something to watch out for is whether numbers should be changed to rates or percentages in order to make appropriate comparisons. For example, it would not make any sense to compare 'the number of people living in poverty' for each of the fifty states in the United States because of the variety in population sizes. Think of the number of people who live in the state of Rhode Island versus the number who live in California. It would be much more appropriate to compare 'the percentage of people living in poverty' for each state instead.
Luigi got a pair of jeans that are normally $64.95, for $52.50. Javier paid $48.75 for a pair of jeans that normally cost $58.25. Which jeans had a higher rate of discount?
Luigi's jeans were marked down $12.45 (64.95 - 52.50). Divide the amount of discount by the original cost (12.45/64.95) and get 0.1917. So, Luigi's jeans were marked down 19.17%.
Javier's jeans were marked down $9.50 (58.25 - 48.75). Divide the amount of discount by the original cost (9.50/58.25) and get 0.1631. So, Javier's jeans were marked down 16.31%.
Therefore, Luigi's jeans had a higher rate of discount.
Methods for Collecting Data
Once a question of interest is posed, there are different ways of collecting data. This is a quick overview of the methods for collecting data that will be studied in this chapter: sample surveys, census, observational studies, and experiments. Each will be covered in more detail in the following sections. As of now, we just want to be able to recognize which method was used or described.
Sample surveys are often used as a way to collect data from just some of the people or objects being studied. Some examples of sample surveys are: mailed out questionnaires, online surveys, phone interviews, or quality control checks. Another way to collect data is through a census. This means that every single person or item in the population is checked, tested, or asked. When trying to determine whether something was a sample or a census, ask yourself if the researchers asked everyone (or tested everything). If yes, then it was a census.
Sometimes it will be most appropriate to conduct an experiment - when the researchers actually 'do something' to the subjects. Observational studies are another common way to collect data. In observational studies, the researchers do not 'do anything' to the subjects, they simply collect data that has already happened or happens naturally. All of these methods of data collection can yield interesting results and often answer questions. However, the only method that can actually prove that one variable causes another is an experiment. When trying to determine whether a research method was an experiment, ask yourself if the researchers did anything to the people or objects that were being studied? If yes, then it was an experiment.
For each of the following scenarios, determine whether the situation described is an experiment, observational study, census, or a sample survey. Explain how you know.
a) Researchers suspected that aspirin could help reduce the risk of having a heart attack. Seven hundred people, aged 40 or older, were willing to participate in a study. Half of these participants were randomly selected to take an aspirin each day. The remaining participants were given a pill that looked like the aspirin, but contained no actual medicine. The study went on for five years and the participant's health was monitored.
b) In an effort to study how the high schools in Minnesota have been preparing students for college, an extensive questionnaire was developed. Ten percent of the high school juniors at every high school in the state were selected randomly to complete this questionnaire.
c) Researchers suspected that tanning beds caused skin cancer. Each time a person was diagnosed with skin cancer, they were asked a serious of questions including whether or not they had used a tanning bed. If they had, further questions were asked regarding how often, what type, and at what age, etc.
d) In an effort to determine how many fish were in Lake George, the lake was drained and the fish were counted.
a) This is an experiment because the researchers changed something. They had the people take aspirin (or fake aspirin).
b) This is a sample survey because only a part of all of the high school students were questioned.
c) This is an observational study because no change was made. The researchers simply asked about past behavior.
d) This is a census because every fish was counted. However, this is ridiculous!! So, let's hope they can find a better way to determine how many fish are in a lake next time!
Problem Set 4.1
Section 4.1 Exercises
1) Lucas is writing an article about the baseball teams for the school paper. He collects data about each player's position, batting average, number of at-bats, hits, stolen bases and whether each player is on the junior varsity or varsity team. Who are the individuals? Which variables are categorical? Which are numerical?
2) Malia has been put in charge of analyzing the employees at her company. She collects information regarding annual salary, years with the company, highest degree earned, job title, yearly contribution toward 401K, number of children, home address and phone number. Who are the individuals? Which variables are categorical? Which are numerical?
3) Determine whether each of the following variables is categorical or numerical.
a) The heights of all of the volleyball players.
b) The position played by all of the football players.
c) The brand of mascara preferred by those surveyed.
d) The numbers of texts sent per month.
e) Each person's social security number.
f) Each person's cell phone provider.
4) The fourth graders at Sand Creek Elementary are doing a unit on weather. There is a thermometer on the building just outside the classroom window. The students will record and analyze the temperature at 8:00 a.m. and 2:00 p.m. every school day for 5 weeks, and then create a graph and write a report based on their findings.
a) Identify the variable of interest, the instrument used, and the units.
b) Comment on the validity, reliability and potential bias for this study.
5) The first graders at Sand Creek Elementary are doing a unit on measurement. Each student has traced her or his own foot and cut it out. Each student will use his or her 'foot' to measure various objects around the room and school. Some of the measurements they will make are height of self and at least two other friends, width of the classroom door, length of a lunch table, etc.
a) The variables of interest are the lengths, widths and heights of various objects. Identify the instrument used, and the units.
b) Comment on the validity, reliability, and potential bias for this study.
6) Determine whether each of the following measurements would have a problem with any of the following: VALIDITY (problem would be a lack of), RELIABILITY (problem would be a lack of), BIAS, (problem would be the presence of). A measurement may have any combination of the factors. For each one with a problem, suggest a better way to make the measurement. (hint: answer similar to example #2)
a) A speedometer is totally unpredictable.
b) Cholesterol levels are determined by patients filling out a survey regarding their diet.
c) Time is measured by using the clock on a cell phone.
d) Grades in a Physics class are determined by students assessing themselves on a scale of 1 to 10.
e) Grades in a statistics class are determined by students' scores on one cumulative test.
f) Sobriety is determined by a breathalyzer that is calibrated to be too sensitive.
7) Super Duper High School has a total of 143 teachers. Suppose that you are a researcher who is interested in studying Teacher Effectiveness at SDHS. You intend to evaluate the effectiveness of all of the teachers for your report.
a) What type of data collection method is this?
b) Suggest at least two valid variables that you might study. Include an instrument that can be used to measure your variables and the units.
c) Suggest at least two invalid variables that you might study. Include an instrument that can be used to measure your variables and the units.
8) For each of the following scenarios, determine whether the situation described is an experiment, observational study, census, or a sample survey. Explain how you know.
a) The Super Spaz Energy drink company randomly selects 2% of the cans filled each day, and tests them for volume, ingredient content, and taste.
b) A government lobbyist analyzes the crime reports for the 4 counties in her community.
c) New advertisements are generally tried out on focus groups before investing a lot of money to pay for airtime on national TV.
d) Each student in Probability and Statistics will take the District Common Assessment as a final exam.
e) A teenager decides to evaluate how serious her parents are about her curfew by coming home 15 minutes late just to see what happens.
9) Pasquale's Big and Tall Shop sold 127 suits during the first quarter of this year, and 17 were returned. Marco's XXL Shop sold 268 suits during the same time period, and 27 were returned.
a) What were the number of returns for each shop? Which shop had a higher number of returns?
b) What were the rates of returns for each shop? Which shop had a higher rate of returns?
c) Which of these statistics gives a more clear representation of customer satisfaction? Explain.
10) Jolene makes $12.45 per hour at her job. Last year she made $10.85. What percent of a raise did Jolene receive?
11) Michaela’s favorite shoes are normally $42.99. Today she found a sale in which they were marked down to $27.99. What percent of a discount is this?
12) The number of incidents of hazing reported at Some Random High School was 84 during the 2010-2011 school year. The following year there were 37 incidents of hazing reported at SRHS. What is the rate of change in reported hazing incidents between these two school years? Is it an increase or a decrease?
13) SRHS has had a huge problem getting students to class on time, so the administrators have implemented a new tardy policy. In an effort to determine whether or not it is working to deter students from being tardy to class, data has been collected and analyzed. The following table shows some of the data:
a) Calculate the percent of change for each category and complete the table (round to the nearest tenth of a percent).
b) Which category saw the most significant change?
c) Based on these calculations, do you feel that the tardy policy is working? Explain your reasoning.