Multiple Linear Regression is used to model the relationship between one dependent variable and two or more independent variables.
Goal: To see how the independent variables predict the dependent variable.
Example: How does a car’s weight (wt) and horsepower (hp) predict its gas mileage (mpg)?
We will use the built-in mtcars data set for our example.
The model is described by a simple equation. \[ Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + ... + \beta_p X_p + \epsilon \]
\(Y\) is the variable we want to predict (e.g., mpg). \(X_1, X_2, ...\) are the predictor variables (e.g., wt, hp). \(\beta_0, \beta_1, ...\) are the coefficients the model calculates. \(\epsilon\) is the error term.
A 3D scatter plot helps us see the relationship between our three variables: mpg, wt, and hp.
This slide shows a block of R code. It’s the exact code used to create the plot on the previous slide. The .smaller tag makes the text fit nicely.
fig <- plot_ly( data = mtcars, x = ~wt, y = ~hp, z = ~mpg, type = "scatter3d", mode = "markers", color = ~as.factor(cyl) ) config(fig, displaylogo = FALSE)
Before we trust our model, we should check if it’s valid. We do this by looking at the residuals (the prediction errors). We can visualize them with ggplot2.
First, we create a simple linear model. Let’s just predict mpg from wt.
Here are two plots to check our model.
Residuals vs. Fitted: Checks for patterns in the errors. We want to see random scatter.
Normal Q-Q: Checks if the errors are normally distributed. Points should be on the line.
The summary() function in R gives us the coefficients for our equation.
If we fit the model mpg ~ wt + hp, we get: \[ \text{mpg} \approx 37.23 - 3.88 \times (\text{wt}) - 0.03 \times (\text{hp}) \]
This formula tells us:
For every 1000 lbs increase in weight (wt), mpg decreases by about 3.88, holding horsepower constant.
For every 1 unit increase in horsepower (hp), mpg decreases by about 0.03, holding weight constant.
After interpreting the model, we need to know how well it explains the data. The most common measure for this is the Coefficient of Determination, or R-squared (\(R^2\)).
\(R^2\) is a value between 0 and 1 that represents the proportion of the variance in the dependent variable that is predictable from the independent variables.
\[ R^2 = 1 - \frac{SS_{res}}{SS_{tot}} \]\(SS_{res} = \sum_{i=1}^{n} (y_i - \hat{y}_i)^2\) is the Sum of Squared Residuals (prediction errors).
\(SS_{tot} = \sum_{i=1}^{n} (y_i - \bar{y})^2\) is the Total Sum of Squares (total variance in Y).
A higher \(R^2\) value indicates a better fit for the model.
We used Multiple Linear Regression to understand how wt and hp affect a car’s mpg.
plotly helped us visualize the data in 3D.
ggplot2 helped us check the model’s assumptions.
This is a simple but powerful tool for statistical analysis.