These slides will provide a brief lesson on an incredibly useful statistical tool: Hypothesis Testing.

Definition: A hypothesis test is an important form of statistical analysis used to test whether an assumption or result accurately represents the population.

What makes up a hypothesis test? There are several important components:

• Null Hypothesis (\(H_0\))

• Alternative Hypothesis (\(H_a\))

• Significance Level

• Test Statistic Calculation

• P-Value

• Final Decision

What is the difference between a null and alternative hypothesis?

• Null Hypothesis (\(H_0\)): The assumption that the event in question will not occur.
• Alternative Hypothesis (\(H_a\)): The assumption that the event will occur.

For example, if we are conducting a hypothesis test to judge the effectiveness of a new pharmaceutical drug, what would our null and alternative hypotheses be?

• (\(H_0\)): The pharmaceutical drug has no effect.

• (\(H_a\)): The pharmaceutical drug has an effect (reduces, or increases some variable, depending on the hypothesis)

The significance level (\(\alpha\)) of the hypothesis test is the probability of rejecting the null hypothesis when it is true.

A higher significance level means it is more likely to falsely reject the null hypothesis, otherwise known as a Type I Error, which we will speak more about later.

Some common significant levels include: (\(\alpha\)) = 0.1, (\(\alpha\)) = 0.05, (\(\alpha\)) = 0.01

Calculating our test statistic depends on our data set. There are multiple different tests we can use to perform our analysis, but for simplicity’s sake, we will just focus on two: t-test and p-test.

What is a test statistic? The test statistic is the value we compare to our significance level to make conclusions about our hypotheses.

T-test: We use the t-test when we don’t know the population standard deviation.

Our formula is: \(t = \frac{\bar{x} - \mu_0}{s / \sqrt{n}}\)

Where \(\bar{x}\) = sample mean, \(\mu_0\) = population mean, s = sample standard deviation, n = sample size

Next is a visualization of a t-test for a t-distribution with 10 degrees of freedom and \(\alpha\) = 0.05.

Z-test: We use the z-test when we do know the population standard deviation.

Our formula is: \(z = \frac{\bar{x} - \mu_0}{\sigma / \sqrt{n}}\)

Where \(\bar{x}\) = sample mean, \(\mu_0\) = population mean, \(\sigma\) = population standard deviation, n = sample size

Next is a visualization of a z-test for a z-distribution \(\alpha\) = 0.05.

The distribution curve for t- and z- tests is very similar, with the main difference being the area under the tail of the curves.

The last step before analysis is calculating our p-value. This process differs depending on whether we are conducting a single or two-tailed test, and whether it is a z- or t-test.

Selecting a p-value for z-tests: We must utilize a probability table for different z-scores. Selecting a p-value for t-tests: We must utilize CDF, using a calculator, or we can use code in R to calculate our p-value. Let’s look at an example using a random t statistic with degree of freedom 20.

t_statistic <- 2.45
df <- 20
p_value_two_tailed <- 2 * (1 - pt(abs(t_statistic), df))
p_value_two_tailed

## [1] 0.02361728

How do variables like sample size and significance level contribute to our P-value?

Here we can see how P-values fluctuate depending on our sample size, x, and our significance level, y.

Now that we have our p-value, we must compare with our significance level.

If \(\alpha <\) p: We reject the null hypothesis.

If \(\alpha >\) p: We fail to reject the null hypothesis.

It is important to note that we can’t say we “accept” the null hypothesis. The results of our analysis simply signify that we do not have the evidence to support the alternative hypothesis over the null. We have not proved that the null hypothesis is true.

Common Errors

Type I: We reject the null hypothesis when the alternative is not true. Type II: We fail to reject the null hypothesis when the alternative is true.