The p-value is the probability of getting results as extreme as what we observed, if the null hypothesis is actually true.

\[P(X \geq x_{observed} | H_0 \text{ is true})\]

Key Points:

• Shows evidence against the null hypothesis (\(H_0\))
• Goes from 0 to 1
• Smaller p-values = stronger evidence against \(H_0\)
• We usually use \(\alpha = 0.05\) as the cutoff

Decision Rule:

\[\text{If } p\text{-value} < \alpha \text{, reject } H_0\] \[\text{If } p\text{-value} \geq \alpha \text{, fail to reject } H_0\]

What it means:

• Small p-value (≤ 0.05): Good evidence against \(H_0\)
• Large p-value (> 0.05): Not much evidence against \(H_0\)

Important: P-value is NOT the probability that \(H_0\) is true!

Scenario: You flip a coin 100 times and get 60 heads. Is the coin fair?

• \(H_0\): \(p = 0.5\) (coin is fair)
• \(H_a\): \(p \neq 0.5\) (coin is not fair)

# test the coin
n <- 100
heads <- 60
result <- binom.test(heads, n, p = 0.5)
result$p.value

## [1] 0.05688793

Result: p-value = 0.0569

Conclusion: Since p-value > 0.05, we fail to reject \(H_0\). Not enough evidence to say the coin is unfair.

Key point: When \(H_0\) is true, p-values are uniform between 0 and 1.

# Example: One-sample t-test
my_data <- c(23.5, 25.1, 24.8, 22.9, 26.2, 24.5, 23.8, 25.5)
null_mean <- 23

# do the t-test
my_result <- t.test(my_data, mu = null_mean)

# get the p-value
pvalue <- my_result$p.value
print(paste("P-value:", round(pvalue, 4)))

# calculate it manually
xbar <- mean(my_data)
s <- sd(my_data)
n <- length(my_data)
t_stat <- (xbar - null_mean) / (s / sqrt(n))
p_manual <- 2 * pt(-abs(t_stat), df = n - 1)
print(paste("Manual p-value:", round(p_manual, 4)))

What p-value is NOT:

1. The probability that \(H_0\) is true
2. The probability that the results happened by chance
3. The probability of making a mistake

What p-value IS:

The probability of seeing data this extreme (or more), if \(H_0\) is true

\[p = P(\text{data as extreme} | H_0)\]

Remember: If the p-value is big, that doesn’t prove \(H_0\) is true!

Guidelines:

• \(p < 0.01\): Very strong evidence against \(H_0\)
• \(0.01 \leq p < 0.05\): Strong evidence against \(H_0\)
• \(0.05 \leq p < 0.10\): Weak evidence against \(H_0\)
• \(p \geq 0.10\): Not much evidence against \(H_0\)

Things to remember:

• Sample size affects p-values
• Statistically significant doesn’t always mean practically significant
• Look at effect size too, not just p-value
• Avoid p-hacking (repeatedly testing until you get p < 0.05)

Main points:

1. P-value shows evidence against the null hypothesis
2. Smaller p-values = more evidence
3. We usually use \(\alpha = 0.05\) as the cutoff
4. When \(H_0\) is true, p-values are uniformly distributed
5. Statistical significance is not the same as practical importance

Formula:

\[\text{Reject } H_0 \text{ if } p < \alpha\]