Linear Regression: Theory & Visualization

A presentation using mtcars.

• Data & motivation
  
• Exploratory ggplots (2)
  
• 3D interactive plotly demo
  
• Math: OLS derivation (LaTeX)
  
• Model fit & code (shows R code)
  
• Diagnostics (plots)
  
• Conclusions

data(mtcars) head(mtcars) summary(mtcars$mpg)

We will model mpg using wt and hp.

The linear model:

\[ y = X\beta + \varepsilon,\quad \varepsilon \sim N(0, \sigma^2 I) \]

The OLS estimator:

\[ \hat\beta = (X^\top X)^{-1} X^\top y \]

Variance of the estimator:

\[ \operatorname{Var}(\hat\beta)=\sigma^2 (X^\top X)^{-1} \]

t-test for coefficient \(\beta_j\):

\[ t = \frac{\hat\beta_j}{\widehat{\operatorname{se}}(\hat\beta_j)} \sim t_{n-p} \]

Fit multiple regression and show summary (this chunk is shown)

## 
## Call:
## lm(formula = mpg ~ wt + hp, data = mtcars)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -3.941 -1.600 -0.182  1.050  5.854 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 37.22727    1.59879  23.285  < 2e-16 ***
## wt          -3.87783    0.63273  -6.129 1.12e-06 ***
## hp          -0.03177    0.00903  -3.519  0.00145 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.593 on 29 degrees of freedom
## Multiple R-squared:  0.8268, Adjusted R-squared:  0.8148 
## F-statistic: 69.21 on 2 and 29 DF,  p-value: 9.109e-12

Interpret coefficients: negative wt coefficient means heavier reduces mpg, also the hp effect is also show