• A p-value (short for probability value) shows us how likely it is some event would have happened under the “null hypothesis” (the case where what’s being tested does not exist. In other words, it helps answer the question: “If there really is no effect, how surprising is this data?”

• p-values range from 0 to 1:

  • A value close to 1 means the observed result is very likely under the null hypothesis.
  • A value closer to 0 means the result is very unlikely without some change.

\[ p = P(\text{data} \mid H_0 \text{ is true}) \]

• As stated in the previous slide, p-values help us understand the likelihood the results are under the null hypothesis.

• Used in hypothesis testing, and help decide whether to reject or fail to reject the null hypothesis.

• EXAMPLE:

  • \(H_0\): A new drug will have no effect on blood pressure.
  • \(H_1\): The new drug will lower blood pressure.

• After conducting an experiment (give the drug to one group, and a placebo to another, and measure their blood pressure over time), we can calculate the p-value of the difference in change in blood pressure of the two groups

• From there we can decide whether the results give enough evidence to support \(H_1\).

When a p-value is very small (typically < .05) it gives us strong evidence to reject the null hypothesis - it’s considered Statistically significant

From the previous slide: if the group who received the drug had a major reduction in blood pressure while the placebo group did not, that would likely indicate Statistical Significance.

Researchers often use a significance level (denoted \(\alpha\)) to define a threshold for rejecting \(H_0\).

The most common threshold is \(\alpha = 0.05\). If \(p < \alpha\), we reject the null hypothesis.

While this is an arbitrary value, it is widely accepted.

Once the experiment is complete, a test statistic is computed (t-score or z-score) that tells us how far our sample result is from what we’d expect under \(H_0\).

Then we calculate the probability of a result at least that extreme in the distribution under \(H_0\). That probability is the p-value

Here’s what a z-test looks like: \[ Z = \frac{\bar{x} - \mu_0}{\sigma / \sqrt{n}} \]

• \(\bar{x}\): Sample Mean
• \(\mu_0\): Hypothesized Population Mean
• \(\sigma\): Population Standard Deviation
• n: sample size

normal_curve <- function(x, mean = 0, sd = 1) {
  1 / (sd * sqrt(2 * pi)) * exp(-0.5 * ((x - mean) / sd)^2)}

x_vals <- seq(-4, 4, by = 0.01)
z_val <- 1.4 #Example observed z-score
y_vals <- normal_curve(x_vals)
df <- data.frame(x = x_vals, y = y_vals)
shade_df <- subset(df, x >= z_val)

ggplot(df, aes(x, y)) + geom_line(color = "black") + 
  geom_area(data = shade_df, aes(x, y), fill = "cyan", alpha = 0.4) +
  geom_vline(xintercept = z_val, linetype = "dashed", color = "red") 
  + labs(title = "P-Value Visualization (Z = 1.4)", subtitle = 
  "Shaded region shows area beyond observed test statistic",
    x = "Z-score", y = "Density") + theme_minimal()

This Graph will be displayed on the next slide!

• We ran an experiment with two groups: one received a new drug, the other a placebo.
• The plot below shows the resulting blood pressure changes from both groups after two weeks.
• A t-test gives us a p-value indicating whether the difference is statistically significant.

• This 3D plot shows how p-values change based on sample size and observed Z-score. As Z-score or sample size (or both) increase, p-value tends to decrease. This helps us visualize why a large sample size can help improve a test.