Multiple linear regression models one numeric response using two or more predictors.

• Response: the variable we want to predict.
• Predictors: the variables used to explain the response.
• Each coefficient describes the change in the response when one predictor increases by one unit, while the other predictors stay fixed.

Example: predict a car’s miles per gallon from its weight and horsepower.

For observations \(i = 1, \dots, n\):

\[ y_i = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + \varepsilon_i \]

For our example:

\[ \text{mpg}_i = \beta_0 + \beta_1 \text{wt}_i + \beta_2 \text{hp}_i + \varepsilon_i \]

• \(\beta_0\) is the intercept.
• \(\beta_1\) and \(\beta_2\) are the slopes for the two predictors.
• \(\varepsilon_i\) is the error term.

The coefficients are chosen to minimize the sum of squared residuals:

\[ \text{SSE}(\boldsymbol{\beta}) = \sum_{i=1}^{n} \left( y_i - \beta_0 - \beta_1 x_{i1} - \beta_2 x_{i2} \right)^2 \]

The solution has a closed form:

\[ \hat{\boldsymbol{\beta}} = (\mathbf{X}^\top \mathbf{X})^{-1} \mathbf{X}^\top \mathbf{y} \]

In R, this is computed by lm().

##                    mpg    wt  hp
## Mazda RX4         21.0 2.620 110
## Mazda RX4 Wag     21.0 2.875 110
## Datsun 710        22.8 2.320  93
## Hornet 4 Drive    21.4 3.215 110
## Hornet Sportabout 18.7 3.440 175
## Valiant           18.1 3.460 105

• mpg: miles per gallon.
• wt: weight in 1000 lbs.
• hp: horsepower.

The dataset has 32 cars and no missing values.

Heavier cars tend to have lower MPG. At the same weight, higher horsepower appears in darker colors.

The residuals show no strong pattern, so the linear model looks reasonable.

The surface is the fitted model. Vertical distances from points to the surface are the residuals.

model <- lm(mpg ~ wt + hp, data = mtcars)

b <- coef(model)
wt_seq <- seq(min(mtcars$wt), max(mtcars$wt), length.out = 25)
hp_seq <- seq(min(mtcars$hp), max(mtcars$hp), length.out = 25)
grid <- expand.grid(wt = wt_seq, hp = hp_seq)
grid$mpg <- b[1] + b[2] * grid$wt + b[3] * grid$hp
z <- t(matrix(grid$mpg, nrow = length(wt_seq), ncol = length(hp_seq)))

plot_ly() %>%
  add_markers(data = mtcars, x = ~wt, y = ~hp, z = ~mpg, name = "Data") %>%
  add_surface(x = wt_seq, y = hp_seq, z = z, opacity = 0.6, name = "Fit")

The matrix and transpose are needed so the surface matches the order of the weight and horsepower values.

##                Estimate Std. Error   t value     Pr(>|t|)
## (Intercept) 37.22727012 1.59878754 23.284689 2.565459e-20
## wt          -3.87783074 0.63273349 -6.128695 1.119647e-06
## hp          -0.03177295 0.00902971 -3.518712 1.451229e-03

Each row gives an estimated coefficient, its standard error, and a p value for the test that the coefficient equals zero. Both predictors have small p values, so both are useful in the model.

• Multiple linear regression extends simple regression to more than one predictor.
• lm() fits the model by least squares.
• Residual plots check whether the linear form is reasonable.
• A 3D plot shows the data and the fitted plane together.