2025-10-19

The Model

We model a continuous response variable \(Y\) as a linear function of one predictor \(X\):

\[ Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i, \quad i = 1, 2, \ldots, n \]

where:

  • \(\beta_0\) = intercept
  • \(\beta_1\) = slope coefficient
  • \(\varepsilon_i \sim \mathcal{N}(0, \sigma^2)\) are independent random errors

The key idea is to find the line that best fits the observed data by minimizing the sum of squared residuals.

Estimating the Coefficients

Ordinary Least Squares (OLS) finds estimates \(\hat{\beta_0}, \hat{\beta_1}\) that minimize:

\[ \text{SSE}(\beta_0, \beta_1) = \sum_{i=1}^{n} (Y_i - \beta_0 - \beta_1 X_i)^2 \]

The closed-form solutions are:

\[ \hat{\beta_1} = \frac{\sum_i (X_i - \bar{X})(Y_i - \bar{Y})}{\sum_i (X_i - \bar{X})^2}, \quad \hat{\beta_0} = \bar{Y} - \hat{\beta_1}\bar{X} \]

These formulas define the best-fitting regression line \(\hat{Y} = \hat{\beta_0} + \hat{\beta_1}X\).

Data Overview

##                    mpg    wt  hp
## Mazda RX4         21.0 2.620 110
## Mazda RX4 Wag     21.0 2.875 110
## Datsun 710        22.8 2.320  93
## Hornet 4 Drive    21.4 3.215 110
## Hornet Sportabout 18.7 3.440 175
## Valiant           18.1 3.460 105
##       mpg              wt              hp       
##  Min.   :10.40   Min.   :1.513   Min.   : 52.0  
##  1st Qu.:15.43   1st Qu.:2.581   1st Qu.: 96.5  
##  Median :19.20   Median :3.325   Median :123.0  
##  Mean   :20.09   Mean   :3.217   Mean   :146.7  
##  3rd Qu.:22.80   3rd Qu.:3.610   3rd Qu.:180.0  
##  Max.   :33.90   Max.   :5.424   Max.   :335.0

ggplot 1: Scatter with Fitted line

##Fit the Model

## # A tibble: 2 × 5
##   term        estimate std.error statistic  p.value
##   <chr>          <dbl>     <dbl>     <dbl>    <dbl>
## 1 (Intercept)    37.3      1.88      19.9  8.24e-19
## 2 wt             -5.34     0.559     -9.56 1.29e-10
## # A tibble: 1 × 12
##   r.squared adj.r.squared sigma statistic  p.value    df logLik   AIC   BIC
##       <dbl>         <dbl> <dbl>     <dbl>    <dbl> <dbl>  <dbl> <dbl> <dbl>
## 1     0.753         0.745  3.05      91.4 1.29e-10     1  -80.0  166.  170.
## # ℹ 3 more variables: deviance <dbl>, df.residual <int>, nobs <int>

Plotly 3D: MPG vs Weight & Horsepower

ggplot 2: Residuals vs Fitted

ggplot 3: Residual normal Q-Q

Quick Interpretation and Uncertainty

##              Estimate Std. Error   t value     Pr(>|t|)
## (Intercept) 37.285126   1.877627 19.857575 8.241799e-19
## wt          -5.344472   0.559101 -9.559044 1.293959e-10
##                 2.5 %    97.5 %
## (Intercept) 33.450500 41.119753
## wt          -6.486308 -4.202635

R code Slide

## 
## Call:
## lm(formula = mpg ~ wt, data = mtcars)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -4.5432 -2.3647 -0.1252  1.4096  6.8727 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  37.2851     1.8776  19.858  < 2e-16 ***
## wt           -5.3445     0.5591  -9.559 1.29e-10 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 3.046 on 30 degrees of freedom
## Multiple R-squared:  0.7528, Adjusted R-squared:  0.7446 
## F-statistic: 91.38 on 1 and 30 DF,  p-value: 1.294e-10