Skip Navigation

Chapter 6: Planning and Conducting an Experiment or Study

Difficulty Level: At Grade Created by: CK-12
Turn In

Chapter Outline

Chapter Summary

Part One: Multiple Choice

  1. A researcher performs an experiment to see if mice can learn their way through a maze better when given a high-protein diet and vitamin supplements. She carefully designs and implements a study with the random assignment of the mice into treatment groups and observes that the mice on the special diet and supplements have significantly lower maze times than those on normal diets. She obtains a second group of mice and performs the experiment again. This is most appropriately called:
    1. Matched pairs design
    2. Repeated measures
    3. Replication
    4. Randomized block design
    5. Double blind experiment
  2. Which of the following terms does not apply to experimental design?
    1. Randomization
    2. Stratification
    3. Blocking
    4. Cause and effect relationships
    5. Placebo
  3. An exit pollster is given training on how to spot the different types of voters who would typically represent a good cross-section of opinions and political preferences for the population of all voters. This type of sampling is called:
    1. Cluster sampling
    2. Stratified sampling
    3. Judgment sampling
    4. Systematic sampling
    5. Quota sampling

Use the following scenario to answer questions 4 and 5. A school performs the following procedure to gain information about the effectiveness of an agenda book in improving student performance. In September, 100 students are selected at random from the school’s roster. The interviewer then asks the selected students if they intend to use their agenda books regularly to keep track of their assignments. Once the interviewer has 10 students who will use their book and 10 students who will not, the rest of the students are dismissed. Next, the selected students’ current averages are recorded. At the end of the year, the grades for each group are compared, and overall, the agenda-book group has higher grades than the non-agenda group. The school concludes that using an agenda book increases student performance.

  1. Which of the following is true about this situation?
    1. The response variable is using an agenda book.
    2. The explanatory variable is grades.
    3. This is an experiment, because the participants were chosen randomly.
    4. The school should have stratified by gender.
    5. This is an observational study, because no treatment is imposed.
  2. Which of the following is not true about this situation?
    1. The school cannot conclude a cause-and-effect relationship, because there is most likely a lurking variable that is responsible for the differences in grades.
    2. This is not an example of a matched pairs design.
    3. The school can safely conclude that the grade improvement is due to the use of an agenda book.
    4. Blocking on previous grade performance would help isolate the effects of potential confounding variables.
    5. Incorrect response bias could affect the selection of the sample.

Part Two: Open-Ended Questions

  1. During the 2004 presidential election, early exit polling indicated that Democratic candidate John Kerry was doing better than expected in some eastern states against incumbent George W. Bush, causing some to even predict that he might win the overall election. These results proved to be incorrect. Again, in the 2008 New Hampshire Democratic primary, pre-election polling showed Senator Barack Obama winning the primary. It was, in fact, Senator Hillary Clinton who comfortably won the contest. These problems with exit polling lead to many reactions, ranging from misunderstanding the science of polling, to mistrust of all statistical data, to vast conspiracy theories. The Daily Show from Comedy Central did a parody of problems with polling. Watch the clip online at the following link. Please note that while “bleeped out,” there is language in this clip that some may consider inappropriate or offensive. http://www.thedailyshow.com/video/index.jhtml?videoId=156231&title=team-daily-polls What type of bias is the primary focus of this non-scientific, yet humorous, look at polling?
  2. Environmental Sex Determination is a scientific phenomenon observed in many reptiles in which air temperature when eggs are growing tends to affect the proportion of eggs that develop into male or female animals. This has implications for attempts to breed endangered species, as an increased number of females can lead to higher birth rates when attempting to repopulate certain areas. Researchers in the Galapagos wanted to see if the Galapagos Giant Tortoise eggs were also prone to this effect. The original study incubated eggs at three different temperatures: 25.50 C, 29.50 C, and 33.50 C. Let’s say you had 9 female tortoises, and there was no reason to believe that there was a significant difference in eggs from these tortoises.
    1. Explain how you would use a randomized design to assign the treatments and carry out the experiment.
    2. If the nine tortoises were composed of three tortoises each of three different species, how would you design the experiment differently if you thought that there might be variations in response to the treatments?
  3. A researcher who wants to test a new acne medication obtains a group of volunteers who are teenagers taking the same acne medication to participate in a study comparing the new medication with the standard prescription. There are 12 participants in the study. Data on their gender, age, and the severity of their condition are given in the following table:
Subject Number Gender Age Severity
1 M 14 Mild
2 M 18 Severe
3 M 16 Moderate
4 F 16 Severe
5 F 13 Severe
6 M 17 Moderate
7 F 15 Mild
8 M 14 Severe
9 F 13 Moderate
10 F 17 Moderate
11 F 18 Mild
12 M 15 Mild

(a) Identify the treatments, and explain how the researcher could use blinding to improve the study.

(b) Explain how you would use a completely randomized design to assign the subjects to treatment groups.

(c) The researcher believes that gender and age are not significant factors, but is concerned that the original severity of the condition may have an effect on the response to the new medication. Explain how you would assign treatment groups while blocking for severity.

(d) If the researcher chose to ignore pre-existing condition and decided that both gender and age could be important factors, he or she might use a matched pairs design. Identify which subjects you would place in each of the 6 matched pairs, and provide a justification of how you made your choice.

(e) Why would you avoid a repeated measures design for this study?


The term most frequently applied to a non-representative sample is bias.
Blind experiment
When the subjects in an experiment do not know which treatment they are receiving, it is called a blind experiment.
Blocking in an experiment serves a purpose similar to that of stratification in a survey.
If you collect information from every unit in a population, it is called a census.
Cluster sampling
Cluster sampling is when a naturally occurring group is selected at random, and then either all of that group, or randomly selected individuals from that group, are used for the sample.
Confounding variables
A slightly different type of additional variable is called a confounding variable. Confounding variables are those that affect the response variable and are also related to the explanatory variable.
Control group
The control group is typically given a placebo.
Convenience sampling
Convenience sampling is a non-probability sampling technique where subjects are selected because of their convenient accessibility and proximity to the researcher.
Double blind experiment
A double blind experiment is an experimental method used to ensure impartiality, and avoid errors arising from bias.
An experiment is a methodical procedure carried out with the goal of verifying, falsifying, or establishing the validity of a hypothesis.
Incorrect response bias
When an individual intentionally responds to a survey with an untruthful answer, this is called incorrect response bias.
Incorrect sampling frame
If the list from which you choose your sample does not accurately reflect the characteristics of the population, this is called incorrect sampling frame.
Judgement sampling
In judgement sampling, the researcher or some other "expert" uses his/her judgement in selecting the units from the population for study based on the population’s parameters.
Lurking variable
A variable that is not included in a study but that may still have an effect on the other variables involved is called a lurking variable.
Margin of error
The margin of error is a statistic expressing the amount of random sampling error in a survey's results.
Matched pairs design
A matched pairs design is a type of randomized block design in which there are two treatments to apply.
Multi-stage sampling
If we select at random from out of that group, or cluster into smaller subgroups, this is referred to as multi-stage sampling.
Non-response bias
how much these individuals' beliefs and opinions reflect those of the general population, and, therefore, almost all surveys could be prone to non-response bias.
Observational study
In an observational study such as the Royal Society’s connecting gender and a mother’s diet, it is possible that there is a third variable that was not observed that is causing a change in both the explanatory and response variables.
In order to help isolate the effects of personal expectations, the control group is typically given a placebo.
Placebo effect
Especially in medical experiments, the psychological effect of believing you are receiving a potentially effective treatment can lead to different results. This phenomenon is called the placebo effect.
Questionnaire bias
Questionnaire bias occurs when the way in which the question is asked influences the response given by the individual.
Quota sampling
In quota sampling, an individual or organization attempts to include the proper proportions of individuals of different subgroups in their sample.
Random sample
In statistical terms a random sample is a set of items that have been drawn from a population in such a way that each time an item was selected, every item in the population had an equal opportunity to appear in the sample.
The best technique for reducing bias in sampling is randomization.
Randomized block design
The Randomized Block Design is research design's equivalent to stratified random sampling.
Randomly assigned
The next element is that the treatments imposed must be randomly assigned.
Repeated measures design
the matched pair is in fact two different observations of the same subject, it is called a repeated measures design.
The other essential element of an experiment is replication. The conditions of a well-designed experiment will be able to be replicated by other researchers so that the results can be independently confirmed.
Response bias
The term response bias refers to problems that result from the ways in which the survey or poll is actually presented to the individuals in the sample.
A sample is a representative subset of a population.
Sampling error
The downside to sampling is that we can never be completely sure that we have captured the truth about the entire population, due to random variation in our sample that is called sampling error.
Sampling frame
The sampling frame is the term we use to refer to the group or listing from which the sample is to be chosen.
Seed value
the function that simulates randomness has to start at some number, called a seed value.
Simple random sample
When a simple random sample of size \begin{align*}n\end{align*} (commonly referred to as an SRS) is taken from a population, all possible samples of size \begin{align*}n\end{align*} in the population have an equal probability of being selected for the sample.
Size bias
If one particular subgroup in a population is likely to be over-represented or under-represented due to its size, this is sometimes called size bias.
Stratified sampling
In stratified sampling, the population is divided into groups, called strata (the singular term is 'stratum'), that have some meaningful relationship.
Systematic sampling
In systematic sampling, after choosing a starting point at random, subjects are selected using a jump number.
The first is that a treatment must be imposed on the subjects of the experiment.
A term often used to describe the problems when a group of the population is not represented in a survey is undercoverage.
Voluntary response bias
All of these polls usually come with a disclaimer stating that, “This is not a scientific poll.” While perhaps entertaining, these types of polls are very susceptible to voluntary response bias.

Image Attributions

Show Hide Details
Date Created:
Feb 23, 2012
Last Modified:
Jan 14, 2016
Files can only be attached to the latest version of chapter
Please wait...
Please wait...
Image Detail
Sizes: Medium | Original