September 17, 2023

Definition

In statistical hypothesis testing, the p-value is the probability of obtaining a result that is at least as “extreme” as the result actually observed, given that the stated null hypothesis is true. A very small p-value would indicate that the observed outcome is very unlikely with the stated null hypothesis. In this case, th p-value might indicate that the null hypothesis can be rejected.

It is important to note that a very low p-value does not prove that the null hypothesis is false. It would only indicate that there is a very small probability that, given the observed results, the null hypothesis fits the total population.

Mathmatical definition

Given an observed test statistic \(t\) from an unknown population \(T\), the p-value \(p\) in relation to the null hypothesis \(H_0\) can be described as:

  • \(p = \text{Pr}(T \geq t ~\vert ~H_0 )\) for a one-sided right tail test,
  • \(p = \text{Pr}(T \leq t ~\vert ~H_0 )\) for a one-side left tail test,
  • \(p = \text{2 min}\{\text{Pr}(T \geq t ~\vert ~H_0 ),\text{Pr}(T \leq t ~\vert ~H_0 )\}\) for a two-sided test.

Generic example of normal distribution with a two-tailed test

Steps to find p-value

  • Define the null hypothesis, \(H_0\), and confidence level, \(\alpha\)
  • Determine the appropriate test (i.e. one-tail or two-tail)
  • Calculate sample mean, \(\bar{x}\), and sample standard deviation, \(s\)
  • Calculate t statistic, \(\displaystyle t = {\bar{x} - \mu_0 \over (s/\sqrt{n})}\)
  • Determine degrees of freedom, \(df = n - 1\)
  • Use calculator to calculate p-value. Alternatively, you can use a t distribution table to approximate the p-value

Calculating p-value on height-age of loblolly pine trees, from Loblolly data set

df = Loblolly %>% mutate(haratio = height / age)
head(df)
   height age Seed  haratio
1    4.51   3  301 1.503333
15  10.89   5  301 2.178000
29  28.72  10  301 2.872000
43  41.74  15  301 2.782667
57  52.70  20  301 2.635000
71  60.92  25  301 2.436800

ggplot(data = df, aes(x = haratio)) + 
  geom_histogram(aes(y=..density..), bins = 12, 
                 col = "black", fill = "blue") + 
  labs(x = "height-age ratio")

Determine null hypothesis, confidence level and type of test

  • \(H_0: \mu_0 = 95\)
  • \(\alpha = 0.95\)
  • Two-tailed test - Generally is used as the default

Calculate sample mean and sample standard deviation

sample_mean = mean(df$haratio)
sample_sd = sd(df$haratio)
print(sample_mean)
[1] 2.31429
print(sample_sd)
[1] 0.4856617

Calculate t statistic and degrees of freedom

t = (sample_mean - mu_0)/(sample_sd/sqrt(length(df$haratio)))
print(t)
[1] -1749.116
deg_free = length(df$haratio) - 1
print(deg_free)
[1] 83

Calculate p-value

2*pt(-abs(t), df = deg_free)
[1] 2.681712e-191

As the calculated p-value is less than \(1 - \alpha = 0.05\), we can state that that it is statistically significant and that the null hypothesis can be rejected.

Alternate way to calculate p-value in r

t.test(x = df$haratio, mu = 95, conf = 0.95)
    One Sample t-test

data:  df$haratio
t = -1749.1, df = 83, p-value < 2.2e-16
alternative hypothesis: true mean is not equal to 95
95 percent confidence interval:
 2.208895 2.419685
sample estimates:
mean of x 
  2.31429 

This function within R not only calculates p-value, it also gives the interval of values that would satisfy the null hypothesis.

Density function of height-age ratio in Loblolly data set