Statistics helps us decide whether a pattern in data is probably real or could reasonably happen by chance.

Question for this example:

Does caffeine change average reaction time?

We will use a one-sample t-test on the caffeine group and compare the sample mean to a baseline of 300 milliseconds.

A hypothesis test starts with two competing statements.

\[H_0: \mu = 300\]

\[H_A: \mu \ne 300\]

• \(H_0\) is the null hypothesis: caffeine users have an average reaction time of 300 ms.
• \(H_A\) is the alternative hypothesis: the average reaction time is different from 300 ms.

The t-statistic measures how many standard errors the sample mean is away from the null value.

\[t = \frac{\bar{x} - \mu_0}{s / \sqrt{n}}\]

For this example:

\[\bar{x} = 283.84, \quad s = 12.57, \quad n = 20\]

\[t = -5.745\]

The p-value answers this question:

If the null hypothesis were true, how surprising would our sample result be?

For a two-sided t-test:

\[p = 2P(T_{n-1} \ge |t|)\]

In this example:

\[p = 0\]

A smaller p-value means the observed sample mean is harder to explain under the null hypothesis.

The t-statistic depends on the sample mean and sample standard deviation.

Using a common significance level of \(\alpha = 0.05\):

• If \(p < \alpha\), reject \(H_0\).
• If \(p \ge \alpha\), do not reject \(H_0\).

Here, the p-value is 0.

Because this p-value is below 0.05, the sample provides evidence that the average reaction time for the caffeine group is different from 300 ms.

library(ggplot2)

set.seed(301)
reaction <- data.frame(
  group = rep(c("No caffeine", "Caffeine"), each = 20),
  reaction_ms = c(rnorm(20, mean = 310, sd = 18),
                  rnorm(20, mean = 285, sd = 16))
)

ggplot(reaction, aes(x = group, y = reaction_ms, fill = group)) +
  geom_boxplot(alpha = 0.75, width = 0.45) +
  geom_jitter(width = 0.08, alpha = 0.75) +
  theme_minimal()

A hypothesis test does not prove a claim with certainty.

It gives a structured way to decide whether observed data is surprising under a null hypothesis.

In this example, the sample mean reaction time was far enough from 300 ms that the result would be unusual if \(H_0\) were true.