Linear regression is a statistical method used to model the relationship between a dependent variable and one or more independent variables by fitting a linear equation to the data.

We model how \(Y\) changes with \(X\): \[ Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i, \] where \(\varepsilon_i\) are random errors with mean 0.

Estimation

We find the best-fit line \(\hat{Y} = \hat{\beta}_0 + \hat{\beta}_1 X\) that makes the squared errors smallest.

Formulas: \[ \hat{\beta}_1 = \frac{\sum (x_i-\bar{x})(y_i-\bar{y})}{\sum (x_i-\bar{x})^2}, \quad \hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x}. \]

Inference

To measure uncertainty in the slope (\(\beta_1\)):

\[ \text{CI: } \hat{\beta}_1 \pm t_{n-2,\,1-\alpha/2}\times \text{SE}(\hat{\beta}_1). \]

Prediction

For a new \(x^*\): \[ \hat{y}(x^*) = \hat{\beta}_0 + \hat{\beta}_1 x^*. \]

That’s the predicted mean response at \(x^*\).

ggplot(mt, aes(wt, mpg, color = cyl)) +
  geom_point(size = 2) +
  geom_smooth(method = "lm", se = TRUE) +
  labs(title = "Fuel Efficiency vs Vehicle Weight",
       x = "Weight (1000 lbs)",
       y = "Miles per Gallon",
       color = "Cylinders")

MPG drops as weight increases (with 95% CI band).

ggplot(mt, aes(fitted(mod1), resid(mod1))) +
  geom_hline(yintercept = 0, linetype = "dashed") +
  geom_point() +
  labs(title = "Residuals vs Fitted (SLR mpg ~ wt)",
       x = "Fitted MPG",
       y = "Residuals")

plot_ly(mt, x = ~wt, y = ~hp, z = ~mpg, color = ~cyl) |>
  add_markers() |>
  layout(
    title = "3D View: mpg by weight and horsepower",
    scene = list(
      xaxis = list(title = "wt (1000 lbs)"),
      yaxis = list(title = "hp"),
      zaxis = list(title = "mpg")
    )
  )

summary(mod1)

## 
## Call:
## lm(formula = mpg ~ wt, data = mt)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -4.5432 -2.3647 -0.1252  1.4096  6.8727 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  37.2851     1.8776  19.858  < 2e-16 ***
## wt           -5.3445     0.5591  -9.559 1.29e-10 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 3.046 on 30 degrees of freedom
## Multiple R-squared:  0.7528, Adjusted R-squared:  0.7446 
## F-statistic: 91.38 on 1 and 30 DF,  p-value: 1.294e-10

confint(mod1)

##                 2.5 %    97.5 %
## (Intercept) 33.450500 41.119753
## wt          -6.486308 -4.202635

• Weight is a strong negative predictor of MPG in mtcars
• The slope \(\hat\beta_1\) is interpretable as MPG change per 1000 lbs