Skip Navigation

4.5: Experiments and Observational Studies

Difficulty Level: At Grade Created by: Bruce DeWItt
Turn In

Learning Objectives

  • Know the terminology of basic experimental design
  • Identify the elements of an experiment
  • Distinguish between observational studies and experiments
  • Outline experiments
  • Understand the effects of lurking variables

Observational Studies and Experiments

When researchers collect data about subjects without imposing any type of treatment, they are doing an observational study. Many conclusions have been based on observational studies. The discovery that smoking causes lung cancer was initially theorized based on observational studies. Many consumers of cigarettes and tobacco companies questioned the validity of such studies, suggesting that it could have been some other variables that caused the cancers, not the cigarettes. Retrospective studies, based on past history of lung cancer patients showed that a high proportion of them were smokers. This did not convince those who either enjoyed smoking, or were making money off of tobacco. There could be some lurking variables to blame, extra variables that were not taken into account, but were actually the cause. Prospective studies, following people in the future, were undertaken in an effort to see whether or not there was a link between cigarette smoking and lung cancer. The statistics were still called into question because statisticians know that the only way to truly show causation is through a controlled experiment.

An experiment imposes some 'treatment' on the subjects. A controlled experiment involves having more than one group, where the only variable that is different between the groups is the treatment being tested. And, subjects will need to be assigned at random (left to impersonal chance) to the various treatment groups to control for lurking variables. With regard to cigarettes and lung cancer, researchers would need to find a group of non-smokers and randomly divide them into two groups. The randomization will divide up lurking variables that the researchers cannot control for. Also, there needs to be a fairly large number of subjects in each group so that the results do not appear to be some kind of a fluke. The researchers would then need to force one group to smoke cigarettes, while making sure that those in the control group did not smoke. This would go on for several years and both groups would need to be checked for lung cancer regularly. Clearly, there is no ethical way to do such an experiment. We cannot force people to do something that we suspect may cause cancer! Scientists were able to experiment on rats to see whether or not cigarettes caused cancer, and it did. Eventually, the compilation of all of these studies convinced everyone that smoking does cause cancer.

The Three Elements of Good Experimental Design are:

  1. Randomization--Subjects must be randomly assigned to treatment groups in an effort to divide up any lurking variables
  2. Control--There should be a control group-- a group that does not receive the treatment. Having more than one group, where the only difference is the treatment being tested, allows for comparisons to be made.
  3. Replication--There should be a large enough number of subjects so that the results seem believable. Also, the experiment should be able to be replicated on a different group of subjects.

Experimental Design

In an experiment, the people, animals, or objects, that are being experimented on are called the subjects. The treatment that is being tested is the explanatory variable. The result, outcome, or change that happens (or doesn't happen) is the response variable. Keep in mind that sometimes it is necessary to give a pre-test prior to imposing the treatment. For example, if we are testing a medication that claims to lower cholesterol levels, we will certainly need to know the cholesterol levels of all of our subjects prior to giving them the treatment. At the end of the experiment we will again test them and then we can compare any change in cholesterol level.

The control group may be given no treatment at all. Or, you may want to use the control group as a way to compare a new treatment to an old treatment. For example, if someone has developed a new medication that they believe will cure headaches, they will want to compare it to aspirin, acetaminophen, and ibuprofen. Such researchers will likely form four randomly assigned groups (Groups A, B, C, and D), assigning the subjects in each respective group to take a specific one of the treatments whenever they have a headache and to record whether or not it worked and how quickly. After some length of time, the researchers will collect the data from the four groups and compare the results. With the only difference being which treatment was taken, researchers can make conclusions determining which treatment (if any) worked better than the others.

There are some other potential problems here though. For instance, would you want the subjects to know which medication they are receiving? It is very possible that they may have some preconceived notions regarding the effectiveness of one or more of these medicines. Such unconscious bias can influence how they perceive the treatment to work. What researchers often do to avoid any bias that the subjects will bring with them is to not tell them what treatment they are receiving. Such an experiment is said to be blind. It is also possible that the researcher may have preconceived notions, or hopeful expectations, regarding the effectiveness of one or all of the treatments. To avoid this, a third party can package the various treatments in similar looking containers, each marked only with a code, before the researcher distributes them to the subjects. In this case neither the subjects nor the researcher distributing the treatments know who is getting what. This is a double blind experiment, and is used often in clinical trials to limit bias.

Another issue is that often a patient's symptoms may improve just at the 'idea' of getting a medication. This is called the placebo effect. Imagine a child who is crying dramatically over a scraped knee, but stops immediately once mom puts a bandaid on. The bandaid is the placebo. It is also common for a participant, who believes that she or he is receiving a potentially promising medication, to have symptoms improve simply because of her or his expectation that they will. To account for this placebo effect, researchers will often give the control group a fake treatment called a placebo. A placebo is sometimes called a "sugar pill"-- it looks like the real treatment, but has no active ingredients. Placebos make blind and double-blind experiments possible. An experiment could involve a placebo shot, or even a placebo surgery (aka sham surgery).

We will demonstrate how to outline an experiment through the following examples. See the sample outline above as a reference.

Example 1

Suppose that a group of scientists have developed a medication that they believe will cure mean-ness. They are calling it Kind At Last (KAL). There are 520 mean people who are willing to participate in this study (300 males and 220 females). This pill needs to be taken twice daily and it may take a few weeks to be fully absorbed into a person's system. Identify the following, and outline a completely randomized experiment.

a) Subjects

b) Explanatory Variable

c) Response Variable

d) Will it be blind? Double-blind? placebo controlled? is a pre-test necessary?

e) Outline a completely randomized experiment


a) Subjects: the 520 mean people (330 male & 220 female)

b) Explanatory Variable: the KAL pills

c) Response Variable: any change in mean-ness

d) will it be blind? double-blind? placebo controlled? is a pre-test necessary? this could definitely be placebo controlled and double-blind. Neither the patients, nor the person distributing the medicine will know which people are receiving which medication. The KAL pills and the placebos will look identical and be in similar packages.

e) Outline a completely randomized experiment:

The previous example is the a completely randomized experiment because all of the subjects started in one group. All subjects were then randomly assigned to treatment groups, with any combination of genders being possible. What if it was theorized that this medication actually has different effects on males than on females? With a completely randomized design it is very possible that we would not end up with an equal number of males and females in each treatment group. If that were to happen, we would not be able to tell whether the treatment affected different genders in the same way or not

Randomized Block Designs

In such a case, it is a good idea to involve blocking in your experimental design. When it is suspected that different subgroups may respond differently to the treatment, the statisticians separate them at the beginning into intentional subgroups called blocks. The subjects in an experiment may be blocked by age, gender, race, previous medical history, etc. Be sure that you do not say that you will randomly assign to the blocks. You cannot randomly choose who is male or female, and you cannot randomly choose who is which race, etc. Each block is then randomly divided among the various treatment groups. This assures a more equal distribution of the subjects among the treatments. It also directly addresses the effects of this suspected lurking variable. Experimental designs in which blocking is used are called randomized block designs.

Example 2 -

Outline a randomized block design to test the KAL pills that blocks by gender. (Continued from example 1)


*Once you have done the comparisons within blocks, you will also want to compare across blocks to see if there are differences. For example, perhaps this KAL medicine works really well on males, but doesn't do a thing for females. Or, maybe one gender experiences negative side effects from the medication.

Problem Set 4.5

Section 4.5 Exercises

1. Researchers want to determine how effective a new allergy drug called Scratch-Be-Gone is at reducing pet allergies. One pill should be taken daily with a meal. 450 pets suffering from allergies will participate in a clinical study comparing this new drug with an existing market drug and a placebo. Identify each of the following:

a) Subjects

b) Explanatory Variable

c) Response Variable

d) Is it possible for this experiment to be double-blind? Explain.

e) Outline a completely randomized experiment

2. Ms. Rokinroll has a theory that listening to music while working on probability problems will help students retain knowledge. She has a set of earphones for each student and intends to compare the effects of classical music, country music, and heavy metal music. Her first period probability class has 36 students and her last period probability class has 34 students. Identify each of the following:

a) Subjects

b) Explanatory Variable

c) Response Variable

d) Do you feel that a control group of no music is necessary? Why or why not?

e) Do you feel that any of the following should be a part of this experiment: blind, double-blind, pre-test? placebo controlled?

f) Outline a randomized block design experiment

3. Researchers want to test a new eye drop against Blink Brand Eye Drops to see if it is better at reducing dry eye symptoms for contact wearers. The researchers are also interested in whether males and females will respond differently. The subjects available are 480 male and 502 female contact wearers who suffer from frequent dry eyes. Identify the following:

a) Subjects

b) Explanatory Variable

c) Response Variable

d) Outline an appropriate experiment: (will it be blind? double-blind? blocked? placebo controlled?)

e) Clearly explain how a table of random digits can be used to do the randomization. Using line #129 select the first five males who will be in the first treatment group. (hint: this will be done just like section 4.3)

4. A new type of cell phone is being developed by The Millionaire Phone Makers Corporation. This phone, called Make-Us-More-Money (MUMM), has a target audience of college students and young professionals (18-35 year olds). The company has developed three different ad campaigns -- a commercial for each has been made and will be tried on the test subjects. The company wants to determine which ad campaign will be most effective prior to flooding the market, so they will test the various commercials on 744 University of Wisconsin students, and 3,057 people who attend this year's Young Professionals Conference in Los Angeles. After viewing a commercial, each subject will fill out a questionnaire that test how likely they would be to purchase the MUMM phone. Identify each of the following:

a) Subjects

b) Explanatory Variable

c) Response Variable

d) Outline an appropriate experiment (it will need to be blocked by the two different locations)

e) This scenario is different than the previous examples, because in the other examples we were able to do the randomization in advance. That would not be possible for something like this in either of the locations, because the subjects will walk up to the researcher and need to be assigned to a 'treatment group'. Explain how the randomization can be done in a case like this.

Review Exercises

5) Study your new vocabulary!

a) Make flashcards for the terms from this chapter. Write the term on one side of the card. On the other side, write a brief definition and include an example. Terms that appear in bold through section 4.1 through 4.5 are the new terms.

b) Study your flashcards.

Notes/Highlights Having trouble? Report an issue.

Color Highlighted Text Notes
Show More

Image Attributions

Show Hide Details
Date Created:
Jun 14, 2011
Last Modified:
Oct 01, 2014
Files can only be attached to the latest version of section
Please wait...
Please wait...
Image Detail
Sizes: Medium | Original