• A statistical measurement to assess the significance of observed data against a hypothesis.
• The probability of obtaining results at least as extreme as the one observed when assuming that the null hypothesis is correct.
• A smaller p-value suggests that the observed is unlikely to occur by chance and provides stronger evidence for rejecting the null hypothesis.

• Assume the null and alternative hypothesis

• Determine a test statistic \(\mathbf{T}\) and its probability distribution

• Calculate the value of the test statistic \(\mathbf{T_{obs}}\) according to observed data

• Derive the p-value, often using integral calculus from the area under the probability distribution curve for all values of \(\mathbf{T}\) that are with greater deviation between the observed value and reference value.

A medical researcher is investigating the efficacy of a new drug for increasing life expectancy. After conducting a randomized controlled trial, the control group (receiving a placebo) show an average life expectancy of +3 years, while the treatment group (receiving the new drug) have a life expectancy of +5 years. Detailed data are as follow.

Control group:

age: 53, 49, 40, 66, 78, 72, 55, 83, 74, 67

life expectancy: 1, 1, 2, 2, 3, 3, 4, 4, 5, 5

Treatment group:

age: 45, 73, 80, 52, 56, 64, 50, 48, 71, 69

life expectancy: 2, 3, 3, 3, 5, 5, 6, 7, 8, 8

T test

Null hypothesis: the new drug has no effect on improving life expectancy.

Alternative hypothesis is that the new drug increases life expectancy.

Test Statistic: \(t = \frac{\bar{X}_1-\bar{X}_2}{\sqrt{\frac{(n_1-1)S_1^2+(n_2-1)S_2^2}{n_1+n_2-2}(\frac{1}{n_1}+\frac{1}{n_2})}}\)

where \(S_1^2\), \(S_2^2\) are sample variance and \(n_1\), \(n_2\) are sample size for two groups, respectively.

Data

Data size

Codes of the precious plot

library(plotly)
library(ggplot2)
age<-c(53,49,40,66,78,72,55,83,74,67,45,73,80,52,56,64,50,48,71,69)
year<-c(1, 1, 2, 2, 3, 3, 4, 4, 5, 5,2, 3, 3, 3, 5, 5, 6, 7, 8, 8)
group<-c(rep('Control',10),rep('Treat',10))
life<-data.frame(cbind(age,year,group))
life$life_expectancy<-as.numeric(year)
life$age<-as.numeric(age)
life$all_life<-year+age
p1<-ggplot(data = life,aes(x=age,y=life_expectancy,color=group)) +
    geom_point(size=4)
p2<-ggplot(data=life,aes(group,fill=group))+geom_bar()

Hypothesis testing

Hypothesis testing

## 
##  Welch Two Sample t-test
## 
## data:  treat and control
## t = 2.3717, df = 15.781, p-value = 0.01539
## alternative hypothesis: true difference in means is greater than 0
## 95 percent confidence interval:
##  0.5264865       Inf
## sample estimates:
## mean of x mean of y 
##         5         3

Plot of P-value

P values are most often used to detect whether a certain pattern which was measured is statistically significant. The smaller p-value, the stronger evidence to reject the null hypothesis.

However, how small is small enough? The most common threshold is 0.05. Under the circumstance, the p-value in the above example \(p=0.01539 < \alpha = 0.05\), leading to conclude that the new drug has an effect on life expectancy. If the threshold was set to be 0.01, the researchers do not have valid evidence to reject the null hypothesis.

Noticeably, p value can only show the extent to which the null hypothesis is supported and the observational result could arise as a result of chance.