In hypothesis testing, the p-value is the probability of observing results at least as extreme as the result actually observed, assuming the null hypothesis is true. In practice, a small p-value (typically less than 0.05) is taken as evidence to reject the null hypothesis.

Consider testing a hypothesis about a population mean:

\[ H_0: \mu = \mu_0 \quad \text{vs.} \quad H_a: \mu \neq \mu_0. \]

For a test statistic \(T\) computed from your data, the p-value for a two-tailed test is given by

\[ p\text{-value} = 2 \, P\Big( T \geq |t_{\text{obs}}| \,\Big|\, H_0 \Big). \]

Below we illustrate the p-value for a one-tailed test on a standard normal distribution. The shaded area represents the probability of obtaining a value greater than the critical value (here, \(z = 1.96\)).

For a two-tailed test, both tails are considered. Here, we shade the regions where \(|z| > 1.96\).

ggplot(df, aes(x, y)) +
  geom_line(color = "black") +
  geom_area(data = subset(df, x > 1.96), aes(x, y),
            fill = "blue", alpha = 0.3) +
  geom_area(data = subset(df, x < -1.96), aes(x, y),
            fill = "blue", alpha = 0.3) +
  labs(title = "Two-Tailed Test: Shaded p-value Regions",
       x = "Test Statistic (z)", y = "Density")

The p-value depends on the observed test statistic, effect size, and sample size. In the following interactive Plotly plot, we visualize a surface where: - x-axis: Sample size (\(n\)) - y-axis: Effect size (\(d\)) - z-axis: Two-tailed p-value, computed as \[ p\text{-value} = 2\Big[1 - \Phi\Big(d\sqrt{n}\Big)\Big], \] assuming a standard normal distribution under \(H_0\).

When interpreting p-values, keep in mind:

• Small p-value (e.g., < 0.05): Strong evidence against \(H_0\); reject the null hypothesis.
• Large p-value: Insufficient evidence to reject \(H_0\).

Mathematically, the decision rule is often written as:

\[ \text{Reject } H_0 \quad \text{if} \quad p\text{-value} < \alpha, \]

where \(\alpha\) is the chosen significance level.