# 8.1: Hypothesis Testing and the P-Value

## Learning Objectives

- Develop null and alternative hypotheses to test for a given situation.
- Understand the critical regions of a graph for single- and two-tailed hypothesis tests.
- Calculate a test statistic to evaluate a hypothesis.
- Test the probability of an event using the -value.
- Understand Type I and Type II errors.
- Calculate the power of a test.

## Introduction

In this chapter we will explore **hypothesis testing,** which involves making educated guesses about a population based on a sample drawn from the population. Most times, hypothesis testing involves making guesses about the difference between the hypothesized value of the mean of an overall population and that of the sample. This is often used in statistics to analyze the likelihood that a population has certain characteristics. For example, we can use hypothesis testing to analyze if a senior class has a particular average SAT score or if a prescription drug has a certain proportion of the active ingredient.

A hypothesis is simply an educated guess about a characteristic or set of facts. When performing statistical analyses, our hypotheses provide the general framework of what we are testing and how to perform the test. These tests are never certain and we can never *prove* or *disprove* hypotheses with statistics, but the outcomes of these tests provide information that either helps support or refute the hypothesis itself.

In this section we will learn about the different types of hypothesis testing, how to develop hypotheses, how to calculate statistics to help support or refute the hypotheses and understand the errors associated with hypothesis testing.

## Developing Null and Alternative Hypotheses

As mentioned in the introduction, hypothesis testing involves testing the difference between a hypothesized value of the mean of an overall population and the mean calculated from a sample. In hypothesis testing, we are essentially determining the magnitude of the difference between the mean of the sample and they hypothesized mean of the population. If the difference is very large, we reject our hypothesis about the population. If the difference is very small, we do not. Below is an overview of this process.

In statistics, the hypothesis to be tested is called the **null hypothesis** and given the symbol . The null hypothesis states that there is no relationship or no difference between an accepted population mean and a sample mean. So finding a significant result means refuting the null hypothesis, showing that the true population mean is likely to be closer to the sample mean. We would calculate the mean of the sample and generalize these findings to the overall population. For example, if we were to test the hypothesis that the seniors had a mean SAT score of our null hypothesis would be that the SAT score would be equal to or:

where:

symbol for null hypothesis

population mean

value to be tested

We test the null hypothesis against an **alternative hypothesis,** which is given the symbol and includes the outcomes not covered by the null hypothesis. Basically, the alternative hypothesis states that there is a difference between the hypothesized population mean and the sample mean. The alternative hypothesis can be supported only by rejecting the null hypothesis. In our example above about the SAT scores of graduating seniors, our alternative hypothesis would state that there is a difference between the null and alternative hypotheses or:

Let’s take a look at a couple of examples and develop a few null and alternative hypotheses.

**Example:**

We have a medicine that is being manufactured and each pill is supposed to have of the active ingredient. What are our null and alternative hypotheses?

**Solution:**

Our null hypothesis states that the population has a mean equal to Our alternative hypothesis states that the population has a mean that is different than

**Example:**

The school principal wants to test if it is true what teachers say -- that high school juniors use the computer an average a day. What are our null and alternative hypotheses?

**Solution:**

Our null hypothesis states that the population has a mean equal to Our alternative hypothesis states that the population has a mean that differs from

## Deciding Whether to Reject the Null Hypothesis: Single and Two-Tailed Hypothesis Tests

When a hypothesis is tested, a statistician must decide on how much evidence is necessary in order to reject the null hypothesis. For example, if the null hypothesis is that the average height of a population is a statistician wouldn't measure one person who is and reject the hypothesis based on that one trial. It is too likely that the discrepancy was merely due to chance. Statisticians first choose a **level of significance** or **alpha level**, which is an event probability below which discrepancies from the null hypothesis are deemed significant. The most frequently used levels of significance are and In other words, these levels mean that when we make the decision to reject the null hypothesis, we are correct or of the time. The areas outside of these levels of significance are called the **critical regions**. When choosing the level of significance, we need to consider the consequences of rejecting or failing to reject the null hypothesis. If there is the potential for health consequences (as in the case of active ingredients in prescription medications) or great cost (as in the case of manufacturing machine parts), we should use a more ‘conservative’ critical region with levels of significance such as or

When determining the critical regions for a **two-tailed** hypothesis test, the level of significance represents the extreme areas under the normal density curve. We call this a two-tailed hypothesis test because the critical region is located in both ends of the distribution. For example, if there was a significance level of the critical region would be the most extreme under the curve with on each tail of the distribution.

Therefore, if the mean from sample taken from the population falls within these critical regions, we would conclude that there was too much of a difference and we would reject the null hypothesis. However, if the mean from the sample falls in the middle of the distribution (in between the critical regions) we would fail to reject the null hypothesis.

We calculate the critical region for the single-tail hypothesis test a bit differently. We would use a single-tail hypothesis test when the direction of the results is anticipated or we are only interested in one direction of the results. For example, a single-tail hypothesis test may be used when evaluating whether or not to adopt a new textbook. We would only decide to adopt the textbook if it improved student achievement relative to the old textbook. A single-tail hypothesis simply states that the mean is greater or less than the hypothesized value.

When performing a single-tail hypothesis test, our alternative hypothesis looks a bit different. When developing the alternative hypothesis in a single-tail hypothesis test we would use the symbols of greater than or less than. Using our example about SAT scores of graduating seniors, our null and alternative hypothesis could look something like:

In this scenario, our null hypothesis states that the mean SAT scores would be equal to while the alternate hypothesis states that the SAT scores would be greater than A single-tail hypothesis test also means that we have only one critical region because we put the entire region of rejection into just one side of the distribution. When the alternative hypothesis is that the sample mean is greater, the critical region is on the right side of the distribution. When the alternative hypothesis is that the sample is smaller, the critical region is on the left side of the distribution (see below).

To calculate the critical regions, we must first find the **critical values** or the cut-offs where the critical regions start. To find these values, we use the critical values found specified by the **-distribution**. These values can be found in a table that lists the areas of each of the tails under a normal distribution. Using this table, we find that for a significance level, our critical values would fall at standard errors above and below the mean. For a significance level, our critical values would fall at standard errors above and below the mean. Using the -distribution we can find critical values (as specified by standard scores) for any level of significance for either single- or two-tailed hypothesis tests.

**Example:**

Use the -distribution table to determine the critical value for a single-tailed hypothesis test with a significance level.

**Solution:**

Using the -distribution table, we find that a significance level of corresponds with a critical value of

## Calculating the Test Statistic

Before evaluating our hypotheses by determining the critical region and calculating the test statistic, we need to first:

- Confirm that the distribution is normal.
- Determine the hypothesized mean of the distribution.
- If we don’t have the population variance, we will need to calculate the standard deviation of the sample so that we can calculate the
**standard error of the mean**.

Remember that since we have a random sample from the population, we do not expect the sample mean to be *exactly* equal to the hypothesized value of the population mean. Therefore, the question really is: “How different can the observed sample mean be from the hypothesized mean before rejecting the null hypothesis?” Or, in other words, “If the null hypothesis is true, is it likely that we will obtain such an observed sample mean?” We use our critical values taken from the -distribution to determine those cutoffs.

To evaluate the sample mean against the hypothesized population mean, we use the concept of -scores to determine how different the two means are from each other. As we learned in previous lessons, the -score is calculated by using the formula:

where:

standardized score

sample mean

hypothesized population mean

standard error?. If we do not have the population variance, we can estimate the deviation of the samples from the true population mean by dividing the standard deviation by the square root of the number of observations .

Once we calculate the -score, we can make a decision about whether to reject or to fail to reject the null hypothesis based on the critical values.

Let’s calculate the test statistic for several different scenarios.

**Example:**

College A has an average SAT score of From a random sample of freshman psychology students we find the average SAT score to be with a standard deviation of We want to know if these freshman psychology students are representative of the overall population. What are our hypotheses and the test statistic?

**Solution:**

Let’s first develop our null and alternative hypotheses:

Our standard -score for the sample of freshman psychology students would be:

**Example:**

A farmer is trying out a planting technique that he hopes will increase the yield on his pea plants. Over the last the average number of pods on one of his pea plants was with a standard deviation of This year, after trying his new planting technique, he takes a random sample of his plants and finds the average number of pods to be He wonders whether or not this is a statistically significant increase. What is his hypotheses and the test statistic?

**Solution:**

First, we develop our null and alternative hypotheses:

Let’s calculate the test statistic for several different scenarios.

**Example:**

This alternative hypothesis is since we are only concerned with the pod *gain* which translates to above the mean.

Next, we calculate the standard -score for the sample of pea plants.

In the following lessons, we will use these standard -scores and the critical regions to evaluate the null and the alternative hypotheses.

## Testing the P-Value of an Event

We can also evaluate a hypothesis by testing the probability, or the P-value, of an event occurring. When we assume that we have normal distributions, we can determine approximately where on the normal distribution that the sample mean will fall. When we know where it falls, we can determine the **probability** of obtaining a sample value either greater or smaller than the mean by using the -score.

Let’s use the example about the pea farmer. As we mentioned, the farmer is wondering if the number of pea pods per plant has gone up with his new planting technique and finds that out of a sample of there is an average number of per plant (compared to a previous average of ). But the farmer is really hoping that some plants have a more dramatic yield increase. What is the probability of a plant having a much higher yield of over ?

To find this probability, first find the -score for the hypothesized sample mean using the formula that we learned in the section above. Therefore, a -score for a sample of plants with would be:

Using the -score distribution, we find that the area beyond a -score of is equal to This means that there is or chance that a pea plant will produce over

## Type I and Type II Errors

When we decide to reject or not reject the null hypothesis, we have four possible scenarios:

- A true hypothesis is rejected.
- A true hypothesis is not rejected.
- A false hypothesis is not rejected.
- A false hypothesis is rejected.

If a hypothesis is true and we do not reject it (Option 2) or if a false hypothesis is rejected (Option 4), we have made the correct decision. But if we reject a true hypothesis (Option 1) or a false hypothesis is not rejected (Option 3) we have made an error. Overall, one type of error is not necessarily more serious than the other. Which type is more serious depends on the specific research situation, but ideally both types of errors should be minimized during the analysis.

Decision Made | Null Hypothesis is True | Null Hypothesis is False |
---|---|---|

Reject Null Hypothesis | Type I Error | Correct Decision |

Do not Reject Null Hypothesis | Correct Decision | Type II Error |

The general approach to hypothesis testing focuses on the **Type I** error: rejecting the null hypothesis when it may be true. The level of significance, also known as the alpha level, is defined as the probability of making a Type I error when testing a null hypothesis. For example, at the level, we know that the decision to reject the hypothesis may be incorrect of the time.

Calculating the probability of making a **Type II** error (?) is not as straightforward as calculating a Type I error. The probability of making a Type II error can only be determined when values have been specified for both the alternative hypothesis and the null hypothesis. Once the value for the alternative hypothesis has been specified, it is possible to determine the probability of making a correct decision (1- ?). This quantity, 1- ?, is called the **power of the test** and is discussed in the next section.

As mentioned, our goal is to minimize the potential of both Type I and Type II errors. However, there is a relationship between these two types of errors. As the level of significance or alpha level (?) increases, the probability of making a Type II error (?) decreases and vice versa. While ? is under our direct control, ? is not. We will look at this relationship a bit more in depth in the next section.

Often we establish the alpha level based on the severity of the consequences of making a Type I error. If the consequences are not that serious, we could set an alpha level at or However, in a field like medical research we would set the alpha level very low (at for example) if there was potential bodily harm to patients. We can also attempt minimize the Type II errors by setting higher alpha levels in situations that do not have grave or costly consequences.

## Calculating the Power of a Test

The **power of a test** is defined as the probability of rejecting the null hypothesis when it is false (making the correct decision). Obviously, we want to maximize this power if we are concerned about making Type II errors. To determine the power of the test, there must be a specified value for the alternative hypothesis which is specified much in the same way as we specify the value in the null hypothesis. For example, suppose that a doctor is concerned about making a Type II error only if the active ingredient in the new medication is less than higher than what was specified in the null hypothesis (say, with a sample of and a standard deviation of ). Now we have values for both the null and the alternative hypotheses.

By specifying a value for the alternative hypothesis, we have selected one of the many values for . In determining the power of the test, we must assume that is true and determine whether we would correctly reject the null hypothesis. In other words, we want to determine the power of our test for detecting this difference. In this example, we may choose a certain dosage if there were medical repercussions above that level.

We want to find the area under the curve that is associated with making a Type II error. In the example above, this means that we need to find the power that the test has for detecting this difference. Calculating the exact value for the power of the test requires determining the area above the critical value set up to test the null hypothesis when it is re-centered around the alternative hypothesis. Say that we have an alpha level of – we would then have a critical value of for the single-tailed test which would have a value of:

Now, with a new mean set at the alternative hypothesis we want to find the value of the critical score when centered around this score. Therefore, we can figure that:

Using the standard distribution we find that the area to the right of a -score of is This means that since we assumed the alternative hypothesis to be true, there is only a chance of rejecting the null hypothesis. The power of this test is about In other words, this test of the null hypothesis is not very powerful and has only a probability of detecting the real difference between the means.

There are several things that affect the power of a test including:

- Whether the alternative hypothesis is a single-tailed or two-tailed test.
- The level of significance .
- The sample size.

## Lesson Summary

- Hypothesis testing involves making educated guesses about a population based on a sample drawn from the population. We generate null and alternative hypotheses based on the mean of the population to test these guesses.
- We establish critical regions based on level of significance or alpha levels. If the value of the test statistic falls in these critical regions, we are able to reject it.
- To evaluate the sample mean against the hypothesized population mean, we use the concept of -scores to determine how different the two means are.
- When we make a decision about a hypothesis, there are four different outcome and possibilities and two different types of errors. A Type I error is when we reject the null hypothesis when it is true and a Type II error is when we do not reject the null hypothesis, even when it is false.
- The power of a test is defined as the probability of rejecting the null hypothesis when it is false (in other words, making the correct decision). We determine the power of a test by assigning a value to the alternative hypothesis and using the -score to calculate the probability of making a Type II error.

## Review Questions

- If the difference between the hypothesized population mean and the mean of the sample is large, we ___ the null hypothesis. If the difference between the hypothesized population mean and the mean of the sample is small, we ___ the null hypothesis.
- At the Chrysler manufacturing plant, there is a part that is supposed to weigh precisely The engineers take a sample of parts and want to know if they meet the weight specifications. What are our null and alternative hypotheses?
- In a hypothesis test, if difference between the sample mean and the hypothesized mean divided by the standard error falls in the middle of the distribution and in between the critical values, we ___ the null hypothesis. If this number falls in the critical regions and beyond the critical values, we ___ the null hypothesis.
- Use the -distribution table to determine the critical value for a single-tailed hypothesis test with a significance level.
- Sacramento County high school seniors have an average SAT score of From a random sample of Sacramento High School students we find the average SAT score to be with a standard deviation of We want to know if these high school students are representative of the overall population. What are our hypotheses and the test statistic?
- During hypothesis testing, we use the -value to predict the ___ of an event occurring.
- A survey shows that California teenagers have an average of in savings . What is the probability that a randomly selected teenager will have savings greater than ?
- Please fill in the types of errors missing from the table below:

Decision Made | Null Hypothesis is True | Null Hypothesis is False |
---|---|---|

Reject Null Hypothesis | (1) ___ | Correct Decision |

Do not Reject Null Hypothesis | Correct Decision | (2) ___ |

- The __ is defined as the probability of rejecting the null hypothesis when it is false (making the correct decision). We want to maximize__if we are concerned about making Type II errors.
- The Governor’s economic committee is investigating average salaries of recent college graduates in California. They decide to test the null hypothesis that the average salary is (standard deviation is ) and is concerned with making a Type II error only if the average salary is
*less*than . For an and a sample of determine the power of a one-tailed test.

## Review Answers

- Reject, Fail to Reject
- ,
- Fail to Reject, Reject
- Probability
- Area beyond a -score of Therefore, there is a probability of that a teenager will have savings greater than
- Type I error, Type II error
- Power of the Test