Hypothesis testing is a statistical method used to compare and make decisions about a group of data points from a sample.

Hypothesis tests help us answer questions such as:

• Is this new application more effective than the previous one?
• Do men and women earn the same average income?
• Is the average quiz score of students in class really an 8/10?

There are two assumptions made in every hypothesis test.

1. Null Hypothesis (\(H_0\)): This is the default assumption. It assumes that there is no difference between the two groups.
   1. The new application is as effective as the old one.
   2. This is written as \(H_0\): \(\mu_1 = \mu_2\)
2. Alternative Hypothesis (\(H_1\)): This is the opposite assumption. It assumes that there is a difference between the two groups.
   1. The new application is more effective than the old one.
   2. This is written as \(H_1\): \(\mu_1 > \mu_2\)

Significance Level (𝞪) - This metric is used to determine how sure we want to be before confirming that the claim is false.

This value can be 0.01, 0.05, or 0.1 but most often is chosen at 0.05 (5%).

P-values - This metric is the chance of observing the occurrence of our data if \(H_0\) were to be true.

If this is more than 𝞪 we believe the claim to be true. We “fail to reject \(H_0\).”

If this is less than 𝞪 we believe the claim to be false. We “reject \(H_0\).”

Not all conclusions drawn are correct. There are two kinds of errors that can occur in these instances.

Type I Error (False Positive) - We reject the null hypothesis even though the hypothesis is true. This type of error is denoted by alpha.

Type II Error (False Negative) - We accept the null hypothesis even though the null hypothesis is false. This type of error is denoted by beta.

library(ggplot2)
x <- seq(-4, 4, 0.01)
y <- dnorm(x)
alpha <- 0.05
crit <- qnorm(1 - alpha/2)

ggplot(data.frame(x, y), aes(x, y)) +
geom_line(linewidth = 1) +
geom_vline(xintercept = c(-crit, crit), linetype = "dashed", color = "black") +
annotate("text", x = -3, y = 0.1, label = "Reject H0", color = "red") +
annotate("text", x = 0, y = 0.35, label = "Fail to Reject H0", color = "darkgreen") +
annotate("text", x = 3, y = 0.1, label = "Reject H0", color = "red") +
labs(title = "Two-Tailed Hypothesis Test (α = 0.05)",
subtitle = "Rejection regions in both tails",
x = "Test Statistic", y = "Density")