Introduction to the P-Value

  • Definition: a probability that measures how likely the results of an observation are to occur by chance, if the null hypothesis is assumed to be true
  • Purpose: Measures the strength of evidence against the null hypothesis
  • in p-value, the “p” stands for “probability”

Numerical Representation of P-Values

  • A p-value is a probability
  • Ranges between 0 and 1
  • Thresholds of Significance:
    • Statistically Significant: a p-value less than or equal to 0.05
    • Not Significant: a p-value greater than 0.05

Contextual Representation of P-Values

  • The p-value determines how strongly observed results can happen by chance
  • To utilize a p-value, it is required to assume the null hypothesis is true
  • A null hypothesis can be rejected with more confidence when the statistical significance is stronger
  • Contextual Explanation Format: We can/cannot reject the null hypothesis, assuming that it is true, because there is a (p-value * 100)% chance of obtaining data at least as extreme as our observed results, purely by chance.

Example Problem: Simple Linear Regression Data

##              Estimate Std. Error   t value     Pr(>|t|)
## (Intercept) 37.285126   1.877627 19.857575 8.241799e-19
## wt          -5.344472   0.559101 -9.559044 1.293959e-10
  • the p-value for the vehicle weight coefficient (wt) is \(1.294 \times 10^{-10}\).
  • Because this value is much lower than 0.05, we can be very confident in rejecting the null hypothesis

Example Problem: 3D Plotting Code

library(plotly)

plot_ly(mtcars,
      x = ~hp, 
      y = ~wt,
      z = ~mpg,
      type = 'scatter3d',
      mode = 'markers',
      marker = list(size = 5,
                    color = ~mpg,
                    colorscale = 'Viridis',
                    showscale = TRUE)) %>% 
  layout(title = "Car Performance: MPG by Weight and Horsepower",
         scene = list(xaxis = list(title = 'Horsepower (hp)'),
               yaxis = list(title = 'Weight (1000 lbs)'),
               zaxis = list(title = 'Gas Mileage (mpg)'))
  )

Example Problem: Simple Linear Regression (3D Plot)

Example Problem: Linear Trend (Regression P-Value)

Example Problem: Model Residuals (Histogram)

Example Problem: Simple Linear Regression Model Equation

\[Y_i = \beta_0 + \beta_1 X_i + \epsilon_i\]

Where:

  • \(Y_i\) is the dependent variable (Miles Per Gallon)
  • \(X_i\) is the independent variable (Car Weight)
  • \(\beta_0\) is the y-intercept coefficient
  • \(\beta_1\) is the slope coefficient
  • \(\epsilon_i\) is the random error term for observation \(i\)

Example Problem: Hypothesis Testing and t-Statistic (P-Value)

\[H_0: \beta_1 = 0 \quad \text{(Weight has no effect on MPG)}\] \[H_a: \beta_1 \neq 0 \quad \text{(Weight has a significant effect on MPG)}\]

To find the p-value, R computes a \(t\)-test statistic using this formula:

\[t = \frac{\hat{\beta}_1 - 0}{SE(\hat{\beta}_1)}\]

If the resulting \(t\)-statistic is large, the corresponding p-value drops close to zero, leading us to reject the null hypothesis.