2026-11-01

Hypothesis Testing

  • Hypothesis testing is a statistical procedure that is used to accept or reject a statistical hypothesis based on sample data
  • It helps someone make decisions under uncertainty

Formulating Hypotheses

In hypothesis testing, we test two competing hypotheses:

  • Null hypothesis (\(H_0\)): this is a statement of no effect or no difference (results were due to random chance because there’s no actual relationship)

    \[H_0: \mu = \mu_0\]

  • Alternative hypothesis (\(H_a\)): represents what we want to test for; there is an actual effect or difference

    \[H_a: \mu \neq \mu_0 \quad \text{or} \quad \mu > \mu_0 \quad \text{or} \quad \mu < \mu_0\]

Choosing Hypotheses, Test Statistics, & Significance Levels

  • How to choose which hypothesis?
    • \(H_0\) is chosen to represent the default assumption (the “no change” hypothesis)
    • \(H_a\) is chosen based on the research question or claim you are trying to find evidence to support
  • Test Statistic & Significance Level (\(\alpha\)):
    • The test statistic (e.g., \(Z\) or \(T\)) standardizes our sample data to measure how far our observation is from \(H_0\)
    • The significance level (\(\alpha\)) (commonly \(0.05\)) is our risk tolerance for rejecting a true \(H_0\). If the \(p\)-value \(<\alpha\), we reject \(H_0\) in favor of \(H_a\)

Standard Normal Rejection Region

  • The shaded parts represent values that are unlikely to occur if the null hypothesis is true
  • For \(\alpha = 0.05\), a Z-score beyond \(\pm 1.96\) falls in the rejection region, providing evidence to reject \(H_0\)

Distribution Comparison

  • The null distribution represents the results we expect if \(H_0\) were true, while the alternative distribution represents results when there is an actual effect

Statistical Power

  • Statistical power: probability of correctly rejecting a false null hypothesis (\(1-\beta\))
  • Power increases as sample size and effect size increase, making a true effect easier to detect (graph shows how this changes)

R Code Example: Proportion Test

## 
##  1-sample proportions test with continuity correction
## 
## data:  55 out of 100, null probability 0.5
## X-squared = 0.81, df = 1, p-value = 0.1841
## alternative hypothesis: true p is greater than 0.5
## 95 percent confidence interval:
##  0.462983 1.000000
## sample estimates:
##    p 
## 0.55
## [1] "P-value: 0.18406012534676"

Types of Errors in Hypothesis Testing

\(H_0\) is True \(H_0\) is False
Reject \(H_0\) Type I Error (\(\alpha\)) Correct Decision (\(1 - \beta\))
Fail to Reject \(H_0\) Correct Decision (\(1 - \alpha\)) Type II Error (\(\beta\))
  • Type I Error (\(\alpha\)): Rejecting a true null hypothesis (False Positive)
  • Type II Error (\(\beta\)): Failing to reject a false null hypothesis (False Negative)

Conclusion

  • Hypothesis testing provides a mathematical foundation for decision making
  • Key steps involve setting \(H_0\) and \(H_a\), choosing a significance level \(\alpha\), calculating the test statistic, and finding the \(p\)-value