2025-03-15

What is a P-Value?

A p-value measures the strength of evidence against the null hypothesis.

How It Works:
- The p-value represents the probability of observing a test statistic as extreme (or more extreme) than the one calculated from the sample, assuming the null hypothesis is true.
- If the p-value is small → The observed data is unlikely under the null hypothesis → More evidence to reject the null hypothesis.
- If the p-value is large → The observed data is likely under the null hypothesis → Insufficient evidence to reject the null hypothesis.
Why It Matters:
- P-values help determine if a result is statistically significant.
- They provide a basis for deciding whether to reject or fail to reject a hypothesis.

Mathematical Definition

The p-value is:

p = Probability of getting a result as extreme as the observed result, assuming the null hypothesis is true.
The p-value is calculated as:

\[ p = P(T \geq t \mid H_0) \]

Where:
- \(T\) = test statistic
- \(H_0\) = null hypothesis
- \(t\) = observed test statistic

Example of Hypothesis Test

We want to test whether the average height of students is 170 cm.
1. Set up hypotheses:
-Null Hypothesis (\(H_0\)): The average height of students = 170 cm

-Alternative Hypothesis (\(H_1\)): The average height of students ≠ 170 cm


2. Decision:
- If \(p < 0.05\) → Reject the null hypothesis
- If \(p \geq 0.05\) → Fail to reject the null hypothesis

Visualizing the P-Value

x <- seq(-3, 3, length = 100)
y <- dnorm(x)

plot(x, y, type = "l", col = "blue", main = "P-Value Region")
polygon(x[c(51:100)], y[c(51:100)], col = "red", border = NA)

Effect of Sample Size on P-Value

sample_sizes <- c(10, 20, 30, 50, 100)
p_values <- c(0.2, 0.1, 0.05, 0.01, 0.001)

par(mar = c(4, 4, 2, 1))  
par(cex.axis = 0.8)       
par(cex.lab = 0.9)        

Sample Size vs P-Value Plotly

R Code Example

set.seed(123)
data <- rnorm(20, mean = 75, sd = 5)
t.test(data, mu = 75)
## 
##  One Sample t-test
## 
## data:  data
## t = 0.65116, df = 19, p-value = 0.5227
## alternative hypothesis: true mean is not equal to 75
## 95 percent confidence interval:
##  73.43201 77.98423
## sample estimates:
## mean of x 
##  75.70812

Conclusion

A p-value estimates the evidence for the null hypothesis, and that indicates whether or not the result seen is statistically significant. It is the chance of observing data (or more extreme) when the null hypothesis is true.

If the p-value is less than 0.05, the result is statistically significant and suggests that the null hypothesis will likely not be true. If the p-value is greater than or equal to 0.05, there is not enough evidence to reject the null hypothesis.

While p-values are helpful in identifying statistical significance, they are not able to measure the magnitude or degree of an effect. They have to therefore be used together with other factors such as study design and confidence intervals to make accurate conclusions.