• What is simple linear regression?
• OLS estimators
• Example with mtcars
• Model diagnostics
• Prediction & intervals
• 3D Plotly view
• Assumptions & takeaways

We model a quantitative response \(Y\) with one predictor \(X\):

\[ \( Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i, \quad \varepsilon_i \sim N(0, \sigma^2) \) \]

OLS finds the line that minimizes the total squared distance between observed and predicted values.

The Ordinary Least Squares (OLS) method minimizes: \[ SSE = \sum_{i=1}^{n} (Y_i - \beta_0 - \beta_1 X_i)^2 \]

Solving gives: \[ \hat{\beta}_1 = \frac{\sum (X_i - \bar X)(Y_i - \bar Y)}{\sum (X_i - \bar X)^2}, \quad \hat{\beta}_0 = \bar Y - \hat{\beta}_1 \bar X \]

In matrix form: \[ \hat{\boldsymbol{\beta}} = (X^T X)^{-1} X^T Y \]

We analyze 32 cars from the mtcars dataset.
Mean MPG = 20.09, mean weight = 3.22 (×1000 lbs), mean horsepower = 147.

df <- mtcars %>%
  rownames_to_column("car") %>%
  mutate(cyl = factor(cyl))
summary(df[, c("mpg", "wt", "hp")])

##       mpg              wt              hp       
##  Min.   :10.40   Min.   :1.513   Min.   : 52.0  
##  1st Qu.:15.43   1st Qu.:2.581   1st Qu.: 96.5  
##  Median :19.20   Median :3.325   Median :123.0  
##  Mean   :20.09   Mean   :3.217   Mean   :146.7  
##  3rd Qu.:22.80   3rd Qu.:3.610   3rd Qu.:180.0  
##  Max.   :33.90   Max.   :5.424   Max.   :335.0

As vehicle weight increases, fuel efficiency decreases — suggesting a strong negative correlation.

mod <- lm(mpg ~ wt, data = df)
summary(mod)

## 
## Call:
## lm(formula = mpg ~ wt, data = df)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -4.5432 -2.3647 -0.1252  1.4096  6.8727 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  37.2851     1.8776  19.858  < 2e-16 ***
## wt           -5.3445     0.5591  -9.559 1.29e-10 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 3.046 on 30 degrees of freedom
## Multiple R-squared:  0.7528, Adjusted R-squared:  0.7446 
## F-statistic: 91.38 on 1 and 30 DF,  p-value: 1.294e-10

coef(mod)

## (Intercept)          wt 
##   37.285126   -5.344472

The negative coefficient confirms that heavier cars have lower MPG. The model explains ~75% of the variation in MPG.

In \(\hat{Y} = \hat{\beta}_0 + \hat{\beta}_1 X\):

• \(\hat{\beta}_1\) shows the change in \(Y\) per unit increase in \(X\).
• If \(\hat{\beta}_1 < 0\), heavier cars have lower MPG.
• The \(p\)-value tests if the relationship is statistically significant.

Fitted model:
MPG = β₀ + β₁ × Weight

Coefficients:
β₀ = 37.29
β₁ = -5.34
R² = 0.753
p-value = 1.29^{-10}

For a new \(x_0\): \(\hat{y}_0 = \hat{\beta}_0 + \hat{\beta}_1 x_0\)

Confidence interval: \[ \hat{y}_0 \pm t_{\alpha/2, n-2}\,s \sqrt{\frac{1}{n}+\frac{(x_0-\bar x)^2}{\sum (x_i-\bar x)^2}} \]

Prediction interval: \[ \hat{y}_0 \pm t_{\alpha/2, n-2}\,s \sqrt{1+\frac{1}{n}+\frac{(x_0-\bar x)^2}{\sum (x_i-\bar x)^2}} \]

Prediction intervals are always wider than confidence intervals since they include individual variability.

The fitted plane illustrates how both weight and horsepower jointly affect fuel efficiency.

For the model
\[ Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i \] assumptions:

1. Linearity: \(E(Y|X) = \beta_0 + \beta_1 X\)
2. Independence: \(\text{Cov}(\varepsilon_i,\varepsilon_j)=0\)
3. Homoscedasticity: \(\text{Var}(\varepsilon_i)=\sigma^2\)
4. Normality: \(\varepsilon_i \sim N(0, \sigma^2)\)

• Negative slope ⇒ heavier cars → lower MPG.
  
• Model explains about 75.3% of MPG variation.
  
• Diagnostics look good.
  
• Plotly 3D shows how multiple predictors extend the model.
  
• Solid base for multiple linear regression.

if (!dir.exists("models")) dir.create("models", recursive = TRUE)
saveRDS(mod, "models/slr_mpg_wt.rds")
m2 <- readRDS("models/slr_mpg_wt.rds")
predict(m2, newdata = data.frame(wt = 3), interval = "prediction")