Chapter 6: Planning and Conducting an Experiment or Study
Chapter Summary
Questions
Multiple Choice:
- A researcher performs an experiment to see if mice can learn their way through a maze better when given a high protein diet and vitamin supplements. She carefully designs and implements a study with random assignment of the mice into treatment groups and observes that the mice on the special diet and supplements have significantly lower maze times than those on normal diets. She obtains a second group of mice and performs the experiment again. This is most appropriately called:
- Matched pairs design
- Repeated measures
- Replication
- Randomized block design
- Double blind experiment
- Which of the following terms does not apply to experimental design?
- Randomization
- Stratification
- Blocking
- Cause and effect relationships
- Placebo
- An exit pollster is given training on how to spot the different types of voters who would typically represent a good cross-section of opinions and political preferences for the population of all voters. This type of sampling is called:
- Cluster Sampling
- Stratified Sampling
- Judgment Sampling
- Systematic Sampling
- Quota Sampling
Use the following scenario to answer questions 4 and 5. A school performs the following procedure to gain information about the effectiveness of an agenda book in improving student performance. In September, \begin{align*}100\end{align*}
100 students are selected at random from the school’s roster. The interviewer then asks the selected students if they intend to use their agenda book regularly to keep track of their assignments. Once the interviewer has \begin{align*}10\end{align*}10 students who will use their book, and \begin{align*}10\end{align*}10 students who will not, the rest of the students are dismissed. Those students current averages are recorded. At the end of the year. the grades for each group are compared and the agenda book group overall has higher grades than the non-agenda group. The school concludes that using an agenda book increases student performance. - Which of the following is true about this situation. The response variable is using an agenda book
- The explanatory variable is grades.
- This is an experiment because the participants were chosen randomly.
- The school should have stratified by gender.
- This is an observational study because no treatment is imposed.
- Which of the following is not true about this situation.
- The school cannot conclude a cause and effect relationship because there is most likely a lurking variable that is responsible for the differences in grades.
- This is not an example of a matched pairs design.
- The school can safely conclude that the grade improvement is due to the use of an agenda book.
- Blocking on previous grade performance would help isolate the effects of potential confounding variables.
- Incorrect response bias could affect the selection of the sample.
Open-Ended Questions
- During the 2004 presidential election, early exit polling indicated that Democratic candidate John Kerry was doing better than expected in some eastern states against incumbent George W. Bush, causing some to even predict that he might win the overall election. These results proved to be incorrect. Again in the 2008 New Hampshire Democratic primary, pre-election polling showed Senator Barack Obama winning the primary. It was in fact Senator Hillary Clinton who comfortably won the contest. These problems with exit polling lead to many reactions ranging from misunderstanding the science of polling, to mistrust of all statistical data, to vast conspiracy theories. The Daily Show from Comedy Central did a parody of problems with polling. Watch the clip online at the following link. Please note that while “bleeped out,” there is language in this clip that some may consider inappropriate or offensive. http://www.thedailyshow.com/video/index.jhtml?videoId=156231&title=team-daily-polls What type of bias is the primary focus of this non-scientific, yet humorous look at polling?
- Environmental Sex Determination is a scientific phenomenon observed in many reptiles in which air temperature when the eggs are growing tends to affect the proportion of eggs that develop into male or female animals. This has implications for attempts to breed endangered species as an increased number of females can lead to higher birth rates when attempting to repopulate certain areas. Researchers in the Galapagos wanted to see if the Galapagos Giant Tortoise eggs were also prone to this effect. The original study incubated eggs at three different temperatures, \begin{align*}25.50\;\mathrm{C}\end{align*}
25.50C , \begin{align*}29.50\;\mathrm{C}\end{align*}29.50C and \begin{align*}33.50\;\mathrm{C}\end{align*}33.50C . Let’s say you had \begin{align*}9\end{align*}9 female tortoises and there was no reason to believe that there was a significant difference in eggs from these tortoises.- Explain how you would use a randomized design to assign the treatments and carry out the experiment.
- If the nine tortoises were composed of three tortoises each of three different species, how would you design the experiment differently if you thought that there might be variations in response to the treatments?
- A researcher who wants to test a new acne medication obtains a group of volunteers who are teenagers taking the same acne medication to participate in a study comparing the new medication with the standard prescription. There are \begin{align*}12\end{align*}
12 participants in the study. Data on their gender, age and the severity of their condition is given in the following table:
Subject Number | Gender | Age | Severity |
---|---|---|---|
1 | M |
\begin{align*}14\end{align*} |
Mild |
2 | M |
\begin{align*}18\end{align*} |
Severe |
3 | M |
\begin{align*}16\end{align*} |
Moderate |
4 | F |
\begin{align*}16\end{align*} |
Severe |
5 | F |
\begin{align*}13\end{align*} |
Severe |
6 | M |
\begin{align*}17\end{align*} |
Moderate |
7 | F |
\begin{align*}15\end{align*} |
Mild |
8 | M |
\begin{align*}14\end{align*} |
Severe |
9 | F |
\begin{align*}13\end{align*} |
Moderate |
10 | F |
\begin{align*}17\end{align*} |
Moderate |
11 | F |
\begin{align*}18\end{align*} |
Mild |
12 | M |
\begin{align*}15\end{align*} |
Mild |
a. Identify the treatments and explain how the researcher could use blinding to improve the study.
b. Explain how you would use a completely randomized design to assign the subjects to treatment groups.
c. The researcher believes that gender and age are not significant factors, but is concerned that the original severity of the condition may have an effect on the response to the new medication. Explain how you would assign treatment groups while blocking for severity.
d. If the researcher chose to ignore pre-existing condition and decided that both gender and age could be important factors, they might use a matched pairs design. Identify which subjects you would place in each of the \begin{align*}6\end{align*}
e. Why would you avoid a repeated measures design for this study?
Answers
Multiple Choice:
- c
- b
- .
- e
- c
Open-Ended Questions
- Incorrect response bias. The main focus of the piece, and an issue in exit polling in general is that there is no guarantee that, for many possible reasons, subjects in an exit poll will answer truthfully. The pollsters also ask the questions in a variety of rude, unethical, and inappropriate ways that would manipulate the responses. Even though a real pollster would never actually engage in this type of behavior, it could be considered questionnaire bias.
- Randomly assign each tortoise a number from 1-9 using a random number generator, then incubate the eggs from tortoises 1-3 at \begin{align*}25.50\;\mathrm{C}\end{align*}
25.50C , 4-6 at \begin{align*}29.50\;\mathrm{C}\end{align*}29.50C , and 7-9 at \begin{align*}33.50\;\mathrm{C}\end{align*}33.50C . When the tortoises hatch, observe and compare the ratio of female and male tortoises (which is not easy to do) at the various temperatures. The results of this study did confirm that the ratio of females is higher found that \begin{align*}29.50\;\mathrm{C}\end{align*}29.50C is the optimum temperature for a higher female ratio and good survival rate, and \begin{align*}280\;\mathrm{C}\end{align*}280C is the best temperature to insure more males (source: Restoring the Tortoise Dynasty, Godfrey Merlin, Charles Darwin Foundation, 1999.) - This would be a blocking design. We would block on species and temperature, so there would be \begin{align*}9\end{align*}
9 blocks, \begin{align*}3\end{align*}3 of each species, and three at each incubation temperature. There really would not be any randomization in this design.
- Randomly assign each tortoise a number from 1-9 using a random number generator, then incubate the eggs from tortoises 1-3 at \begin{align*}25.50\;\mathrm{C}\end{align*}
- (a) There are two treatments, the new medication, and the existing medication. All the subjects could be told that they were receiving a new treatment, and then only some would be given the new treatment and the rest would be given their original medication. The resulting differences in skin condition between the two groups would be compared. (b) You could assign the subjects a different numbering from \begin{align*}1\end{align*}
1 to \begin{align*}12\end{align*}12 , but this time generating the assignments at random. Then subjects 1-6 would be given the new treatment, and subjects 7-12 would be given the original medication. Compare the results. (c) In blocking for condition, each block should be homogeneous for that trait. So, you would create three blocks: all \begin{align*}4\end{align*}4 mild subjects, all \begin{align*}4\end{align*}4 moderate subjects, and all \begin{align*}4\end{align*}4 severe subjects. Then, within each block, randomly assign two subjects to receive the new treatment, and two to receive the original. Compare the results.
Pair Number | Gender | Age |
---|---|---|
1 | F |
\begin{align*}13\end{align*} |
1 | F |
\begin{align*}13\end{align*} |
2 | F |
\begin{align*}15\end{align*} |
2 | F |
\begin{align*}16\end{align*} |
3 | F |
\begin{align*}17\end{align*} |
3 | F |
\begin{align*}18\end{align*} |
4 | M |
\begin{align*}14\end{align*} |
4 | M | \begin{align*}14\end{align*} |
5 | M | \begin{align*}15\end{align*} |
5 | M | \begin{align*}16\end{align*} |
6 | M | \begin{align*}17\end{align*} |
6 | M | \begin{align*}18\end{align*} |
Place the females in chronological order, then group the two youngest, the next two, and the last two. Repeat the same procedure with the males. This way we have pairs that are similar in both age and gender. One of the subjects would be chosen at random for the new treatment and the other would receive the traditional medication.
(d) Repeated measures are not a good idea with medication studies as it would be hard to distinguish if the effects from the repeated treatment are not in fact from still occurring from the presence of the first medication that was given.