The built-in mtcars data: 32 cars from a 1974 Motor Trend road test.

• Response \(y\): miles per gallon (mpg)
• Predictor \(x\): weight in 1000 lb (wt)

For cars \(i = 1, \dots, n\):

\[Y_i = \beta_0 + \beta_1 x_i + \varepsilon_i, \qquad \varepsilon_i \stackrel{iid}{\sim} N(0, \sigma^2)\]

• \(\beta_0\) is the intercept and \(\beta_1\) the slope (change in mean mpg per extra 1000 lb)
• \(\varepsilon_i\) is random error with variance \(\sigma^2\)

Choose \(\beta_0, \beta_1\) to minimize the sum of squared errors

\[SSE(\beta_0, \beta_1) = \sum_{i=1}^{n} \left(y_i - \beta_0 - \beta_1 x_i\right)^2\]

The minimizers are

\[\hat\beta_1 = \frac{\sum_{i=1}^{n}(x_i - \bar x)(y_i - \bar y)}{\sum_{i=1}^{n}(x_i - \bar x)^2}, \qquad \hat\beta_0 = \bar y - \hat\beta_1 \bar x\]

Each point \((\beta_0, \beta_1)\) is a candidate line and its height is the SSE. Drag to rotate: the red dot is the least-squares solution.

fit <- lm(mpg ~ wt, data = mtcars)
knitr::kable(summary(fit)$coefficients, digits = 4)

Fitted line: \(\hat y = 37.29 + (-5.34)\,x\)

ggplot(mtcars, aes(wt, mpg)) +
  geom_point(size = 3, color = "#1F4E79") +
  geom_smooth(method = "lm", formula = y ~ x, color = "#E07B00",
              fill = "#E07B00", alpha = 0.2) +
  labs(x = "Weight (1000 lb)", y = "Miles per gallon")

Each extra 1000 lb is associated with a drop of about 5.34 mpg on average.

\[R^2 = 1 - \frac{SSE}{SST} = 1 - \frac{\sum (y_i - \hat y_i)^2}{\sum (y_i - \bar y)^2}\]

To test \(H_0: \beta_1 = 0\) we use

\[t = \frac{\hat\beta_1}{SE(\hat\beta_1)} \sim t_{n-2}, \qquad SE(\hat\beta_1) = \frac{s}{\sqrt{\sum (x_i - \bar x)^2}}, \quad s^2 = \frac{SSE}{n-2}\]

Here \(R^2 = 0.753\) and the slope’s p-value is 1.3e-10.

For weight \(x_0\), a 95% prediction interval for one new car is

\[\hat y_0 \pm t_{0.025,\, n-2}\; s \sqrt{1 + \frac{1}{n} + \frac{(x_0 - \bar x)^2}{\sum (x_i - \bar x)^2}}\]

Dropping the \(1\) under the root gives the interval for the mean response. For \(x_0 = 3\) (3000 lb):

• Simple linear regression models the mean of \(Y\) as a straight line in \(x\).
• Least squares picks the line with the smallest SSE.
• In mtcars, each extra 1000 lb lowers expected mpg by about 5.3, and weight explains 75% of the variation in mpg.
• Always check the residuals before trusting the inference.