• Regression summarizes how a numeric outcome changes with a predictor.
• It helps us describe patterns and make predictions.
• Here’s a question we can pose:
• Question: Do heavier cars tend to have lower fuel efficiency?
• We can examine the built-in mtcars dataset to try to figure out an answer to the question.

• Response: mpg = miles per gallon (higher means more efficient).
• Explanatory Variable/Predictor: wt = vehicle weight, in thousands of pounds.
• Each row represents one car model from the dataset.

A simple linear regression model is

\[Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i\]

• \(Y_i\): fuel efficiency (MPG) for car \(i\).
• \(X_i\): weight (in 1,000 lb).
• \(\beta_0\): intercept; \(\beta_1\): change in mean MPG per 1,000 lb.
• \(\varepsilon_i\): difference between observed and modeled MPG.

In an ideal world, this would be the equation that would accurately predict the fuel efficiency of a car in relation to its weight.

However, we must use estimated coefficients:

\[\widehat{Y} = \widehat{\beta}_0 + \widehat{\beta}_1 X\]

The method of ordinary least squares chooses the line minimizing squared residuals:

\[\operatorname*{min}_{b_0,b_1}\; \sum_{i=1}^{n}(y_i-b_0-b_1x_i)^2\]

For a fitted observation, its residual is:

\[e_i = y_i - \widehat{y}_i\] In other words, the residual \(e_i\) is the difference between the observed data point \(y_i\) and the predicted data point \(\widehat{y}_i\).

For the mtcars dataset, the fitted model is:

\[\widehat{\mathrm{MPG}} = 37.29 - 5.34 \times \mathrm{weight}\]

The slope suggests that an extra 1,000 lbs is associated with approximately 5.34 fewer MPG in this sample.

Say we want to predict the MPG of a car weighing 3,000 lb. Since \(x\) is measured in thousands of pounds, \(x = 3\).

\[\widehat{\mathrm{MPG}} = \widehat{\beta}_0 + \widehat{\beta}_1(3)\]

Substituting our estimated values:

\[\widehat{\mathrm{MPG}} = 37.29 - 5.34(3.0) \approx 21.25\]

Prediction: The model predicts 21.3 MPG for a 3,000-lb car.

Hover over points to inspect car names, weight, and MPG. Zoom and pan to explore the scatterplot.

Residual plots help us look for curvature, uneven variability, and unusual observations. Ideally, a residual plot should be symmetrically distributed.

library(ggplot2)
library(plotly)
data(mtcars)

model <- lm(mpg ~ wt, data = mtcars)
summary(model)
predict(model, newdata = data.frame(wt = 3.0))

ggplot(mtcars, aes(wt, mpg)) +
  geom_point() +
  geom_smooth(method = "lm")

• Estimated slope: -5.34 MPG per 1,000 lbs.
• \(R = 0.868\): \(R\) is the correlation coefficent (the strength of the relationship between predictor and response).
• \(R^2 = 0.753\): \(R^2\) numerically describes how much the predictor (weight) explains the variation in the response variable (MPG). Within in this dataset, weight explains about 75.3% of the variation in MPG in this model.
• The relationship is correlational, not proof that changing weight alone causes this exact MPG change.
• Other factors, such as engine design, may matter (more).

• A regression line turns a scatterplot pattern into a numerical prediction.
• The slope describes the direction and size of a correlation.
• \(R\) describes the strength of a correlation; \(R^2\) describes the variability.
• Residual plots are an important check before relying on predictions. They can determine whether the current model captures the true relationship of data.