2025-06-07

What is a P-Value?

  • The p-value is the probability of obtaining a result at least as extreme as the observed one, assuming the null hypothesis \(H_0\) is true.

  • It helps us decide whether the evidence is strong enough to reject \(H_0\).

  • A small p-value (typically < 0.05) suggests that the observed data is unlikely under \(H_0\), and we may reject it.

\[ \text{p-value} = P(\text{Test Statistic} \geq \text{observed value} \mid H_0) \] —

Example – Is the Coin Fair?

We toss a coin 100 times and get 60 heads.

We want to test:

\[ H_0: p = 0.5 \quad \text{(the coin is fair)} \\ H_a: p \neq 0.5 \quad \text{(the coin is biased)} \] We’ll use a simulation approach to understand how likely it is to get 60 or more heads if the coin were actually fair. —

Simulating Under \(H_0\)

We simulated 10,000 coin tosses (100 flips each) assuming a fair coin. We calculated how often we observed 60 or more heads or 40 or fewer heads — results as extreme or more extreme than our observed value.

## [1] 0.0567

How is the P-Value Computed?

For a two-tailed binomial test, we calculate:

\[ \text{p-value} = P(X \leq x_{\text{obs}}) + P(X \geq n - x_{\text{obs}}) \]

Where:

  • \(X \sim \text{Binomial}(n, p)\)
  • \(x_{\text{obs}}\) is the observed number of heads
  • \(p = 0.5\), the hypothesized proportion under \(H_0\)

This formula reflects the probability of getting a result as extreme or more extreme than the observed data.

Visualizing the Distribution of Heads

How Often Are Results Extreme?

A 3D View of P-Values

Summary and Takeaways

  • The p-value helps us decide whether to reject the null hypothesis \(H_0\).
  • A small p-value (e.g., < 0.05) suggests the observed data is unlikely under \(H_0\).
  • In our example, we simulated coin tosses and found a p-value ≈ 0.0567 — not small enough to reject \(H_0\).
  • Visualization helps make statistical reasoning more intuitive.

Remember: p-value is not the probability that the null hypothesis is true. It is the probability of observing such data, if the null hypothesis is true.