A p-value is the probability of obtaining test results at least as extreme as the observed results, assuming that the null hypothesis is true.

In simpler terms:

• It measures the strength of evidence against the null hypothesis
• Small p-value -> strong evidence against \(H_0\)
• Large p-value -> weak evidence against \(H_0\)

Key Point: The p-value is NOT the probability that the null hypothesis is true!

The hypothesis testing process:

1. State the null hypothesis (\(H_0\)) and alternative hypothesis (\(H_a\))
2. Choose a significance level (\(\alpha\)), (typically 0.05)
3. Calculate the test statistic from your data
4. Determine the p-value
5. Make a decision:
   • If p-value \(\leq \alpha\): Reject \(H_0\)
   • If p-value \(> \alpha\): Fail to reject \(H_0\)

For a test statistic \(Z\) following a standard normal distribution:

\[Z = \frac{\bar{X} - \mu_0}{\sigma/\sqrt{n}}\]

Where:

• \(\bar{X}\) = sample mean
• \(\mu_0\) = hypothesized population mean
• \(\sigma\) = population standard deviation
• \(n\) = sample size

The p-value for a two-tailed test is:

\[\text{p-value} = 2 \times P(Z > |z_{obs}|)\]

Scenario: A coffee shop claims their average cup contains 12 oz, but you suspect it’s less.

Hypotheses:

• \(H_0: \mu = 12\) oz
• \(H_a: \mu < 12\) oz

Data: Sample of \(n = 25\) cups, \(\bar{x} = 11.7\) oz, \(\sigma = 0.8\) oz

Test Statistic: \[Z = \frac{11.7 - 12}{0.8/\sqrt{25}} = \frac{-0.3}{0.16} = -1.875\]

# Sample data
n <- 25
x_bar <- 11.7
mu_0 <- 12
sigma <- 0.8

# Calculate test statistic
z_stat <- (x_bar - mu_0) / (sigma / sqrt(n))
cat("Z-statistic:", round(z_stat, 3), "\n")

## Z-statistic: -1.875

# Calculate p-value (one-tailed test)
p_value <- pnorm(z_stat)
cat("P-value:", round(p_value, 4), "\n")

## P-value: 0.0304

# Decision at alpha = 0.05
if (p_value < 0.05) {
  cat("Decision: Reject H₀ at α = 0.05")
} else {
  cat("Decision: Fail to reject H₀ at α = 0.05")
}

## Decision: Reject H₀ at α = 0.05

Conclusion: p-value = 0.0304 < 0.05, so we reject \(H_0\). Evidence suggests cups contain less than 12 oz.

WRONG Interpretations:

• ❌ “P-value is the probability that \(H_0\) is true”
• ❌ “1 - p-value is the probability that \(H_a\) is true”
• ❌ “A large p-value proves \(H_0\) is true”

CORRECT Interpretation:

• ✅ “P-value is the probability of observing data this extreme (or more extreme) if \(H_0\) were true”
• ✅ Small p-value = data is inconsistent with \(H_0\)
• ✅ Large p-value = data is consistent with \(H_0\) (but doesn’t prove it!)

1. P-values measure evidence: They quantify how compatible your data is with the null hypothesis

2. Smaller is stronger: Lower p-values indicate stronger evidence against \(H_0\)

3. Context matters: Always report the p-value along with the test statistic and effect size

4. Significance ≠ Importance: Statistical significance does not always mean practical importance

5. Use wisely: P-values are one tool among many in statistical inference

Remember: “Absence of evidence is not evidence of absence!”