Simple linear regression describes how one quantitative predictor is associated with one quantitative response.
The variable used to explain or predict another variable.
The outcome whose value we want to understand or predict.
Example question: Can vehicle weight help predict fuel efficiency?
\[ Y_i = \beta_0 + \beta_1 X_i + \epsilon_i \]
The fitted regression line is
\[
\hat{Y} = \hat{\beta}_0 + \hat{\beta}_1 X
\]
The least-squares method chooses the line that minimizes:
\[ SSE = \sum_{i=1}^{n}(y_i-\hat{y}_i)^2 \]
\[ \hat{\beta}_1 = \frac{\sum (x_i-\bar{x})(y_i-\bar{y})} {\sum (x_i-\bar{x})^2} \]
\[ \hat{\beta}_0 = \bar{y}-\hat{\beta}_1\bar{x} \]
Least squares finds the line with the smallest overall squared prediction error.
We use the built-in R dataset mtcars.
wtmpgwt is measured in 1000 lbs and mpg is miles per gallon.
| Variable | Min | Mean | Max |
|---|---|---|---|
| Weight (1000 lbs) | 1.51 | 3.22 | 5.42 |
| Miles per gallon | 10.40 | 20.09 | 33.90 |
The downward pattern suggests a negative linear association.
The model can be fitted with just two lines of R code.
model <- lm(mpg ~ wt, data = mtcars) summary(model)
The fitted equation is:
\[ \widehat{MPG} = 37.29 + (-5.34) \times Weight \]
A residual measures the prediction error: \[ e_i = y_i - \hat{y}_i \]
Residuals should ideally be scattered around zero without a strong systematic pattern.
Hover over a point to see the car name, weight, and MPG.
MPG change associated with a 1000-lb increase in weight.
of the variation in MPG is explained by vehicle weight.
Because the slope is negative, heavier vehicles are predicted to have lower fuel efficiency.
In this example, vehicle weight is strongly associated with MPG.
Association does not automatically imply causation.