A p-value is the probability of getting a test statistic at least as extreme as what we observed if the null hypothesis was true.
- The p-value measures how likelihood of the data are if \(H_0\) were true.
- It is not the probability that \(H_0\) is true.
- A small p-value would mean the data is unlikely under \(H_0\). A large p-value would mean the data looks typical under \(H_0\).
- You compare \(p\) to a chosen threshold \(\alpha\) (usually 0.05) to make a decision with the data givens.

For a test statistic \(T\) and observed value \(t_{obs}\): \[ p = \begin{cases} P(T \ge t_{obs}\mid H_0), & \text{right-tailed}\\[4pt] P(T \le t_{obs}\mid H_0), & \text{left-tailed}\\[4pt] 2\min\{P(T \ge |t_{obs}|\mid H_0),\,P(T \le -|t_{obs}|\mid H_0)\}, & \text{two-tailed} \end{cases} \]

where \(T\) is the test statistic under \(H_0\) and \(t_{obs}\) is the observed value.

Let’s run a one-sample t-test.
1. Choose a question: Is the mean height \(\mu\) equal to 170 cm?
2. Set hypotheses: \(H_0:\mu=170\) Vs. \(H_1:\mu\ne170\).
3. Pick \(\alpha\) = 0.05.
4. Compute \(t\) and find the p-value from the t distribution.
5. Decision: if \(p<\alpha\), reject \(H_0\); otherwise, do not reject.

We will test whether the mean height of students is 170 cm.

Null: \(H_0:\mu=170\).

Alt: \(H_1:\mu\ne170\).

t_test <- t.test(heights, mu = 170)

Key results

\(t=\) 3.452, \(df=\) 29

\(p=\) 0.001728.

• \(p<\alpha\): reject \(H_0\). The data is inconsistent with \(H_0\) at level \(\alpha\).
  
• \(p\ge\alpha\): do not reject \(H_0\). There’s not enough evidence to say \(\mu\ne170\).

Common mistakes

1. The p-value is the probability \(H_0\) is true. This is false.
   
2. Using \(p\) as the size of the effect. We would use confidence intervals and estimates for the size.

\[ \text{Reject } H_0 \text{ if } p < \alpha \quad\text{(e.g., } \alpha=0.05\text{).} \]

Confidence intervals relate to the same idea:

if \(170\) is outside the \(95\%\) CI, then \(p<0.05\);

if \(170\) is inside the CI, then \(p\ge0.05\).

The p-value shows that the observed data are under \(H_0\) is extremely small.

Here we found \(p=\) 0.001728.

Smaller p-values mean stronger evidence against \(H_0\).

Always interpret p-values with effect size and confidence intervals:
- Mean = 172.3 cm
- 95% CI = 170.9 to 173.7 cm

Therefore, we would reject \(H_0\) since the p-value is less than our alpha.

I hope this shows what the p-value represents and how we can use it for statistical analysis!