The p-value is one of the most widely used and misunderstood concepts in statistics.

Formal Definition:

The p-value is the probability of observing a test statistic as extreme or more extreme than the one actually observed, assuming the null hypothesis is true.

In plain terms: How surprising is our data if nothing interesting is happening?

• A small p-value → data is unlikely under \(H_0\) → evidence against \(H_0\)
• A large p-value → data is consistent with \(H_0\) → no strong evidence against \(H_0\)

Every p-value lives inside a hypothesis test:

• \(H_0\): Null hypothesis — the default assumption (e.g., “no effect”, “no difference”)
• \(H_a\): Alternative hypothesis — what we want to detect

The logic:

1. Assume \(H_0\) is true
2. Compute a test statistic from the data
3. Ask: how often would we see a result this extreme by chance?
4. That probability is the p-value

Decision rule: Reject \(H_0\) if \(p\text{-value} < \alpha\), where \(\alpha\) is the chosen significance level (commonly \(\alpha = 0.05\)).

For a one-sample z-test comparing a sample mean \(\bar{x}\) to a known \(\mu_0\):

\[Z = \frac{\bar{X} - \mu_0}{\sigma / \sqrt{n}}\]

Under \(H_0\), the test statistic follows a standard normal distribution: \(Z \sim \mathcal{N}(0,1)\)

The p-values for each alternative are:

\[p = P(Z \geq z_{\text{obs}}) \quad \text{(right-tailed)}\] \[p = P(Z \leq z_{\text{obs}}) \quad \text{(left-tailed)}\] \[p = 2 \cdot P(Z \geq |z_{\text{obs}}|) \quad \text{(two-tailed)}\]

For a t-test (unknown \(\sigma\)), replace \(Z\) with \(T \sim t_{n-1}\).

The shaded regions represent the p-value, the combined tail probability.

Question: A coin is flipped 40 times and lands heads 28 times. Is the coin fair?

• \(H_0: p = 0.5\) (fair coin) \(\quad\) \(H_a: p \neq 0.5\) (two-tailed)

\[\hat{p} = \frac{28}{40} = 0.70, \quad z = \frac{0.70 - 0.50}{\sqrt{0.5 \cdot 0.5 / 40}} \approx 2.53\]

\[p\text{-value} = 2 \cdot P(Z \geq 2.53) \approx 0.0114\]

Since \(p \approx 0.011 < 0.05 = \alpha\), we reject \(H_0\). There is statistically significant evidence the coin is not fair.

For any \(\alpha\) to the right of the dashed line, we reject \(H_0\).

# One-proportion z-test: 28 heads in 40 flips
result <- prop.test(x = 28, n = 40, p = 0.5,
                    alternative = "two.sided",
                    correct = FALSE)
result

## 
##  1-sample proportions test without continuity correction
## 
## data:  28 out of 40, null probability 0.5
## X-squared = 6.4, df = 1, p-value = 0.01141
## alternative hypothesis: true p is not equal to 0.5
## 95 percent confidence interval:
##  0.5456998 0.8192515
## sample estimates:
##   p 
## 0.7

Always report effect sizes and confidence intervals alongside p-values.

Remember: The p-value is a tool not a verdict. Use it alongside effect sizes, confidence intervals, and domain knowledge for sound statistical inference.