Simple linear regression is a way to understand how one variable affects another.
For example: does a heavier car get worse gas mileage?
To describe this relationship mathematically, we use the following model:
\[Y = \beta_0 + \beta_1 X + \varepsilon\]
Where:
The fitted model uses estimated coefficients:
\[\hat{Y} = \hat{\beta}_0 + \hat{\beta}_1 X\]
The coefficients \(\hat{\beta}_0\) and \(\hat{\beta}_1\) are chosen to minimize the Residual Sum of Squares (RSS):
\[RSS = \sum_{i=1}^{n}(y_i - \hat{y}_i)^2 = \sum_{i=1}^{n}(y_i - \hat{\beta}_0 - \hat{\beta}_1 x_i)^2\]
The closed-form solutions are:
\[\hat{\beta}_1 = \frac{\sum(x_i - \bar{x})(y_i - \bar{y})}{\sum(x_i - \bar{x})^2}, \quad \hat{\beta}_0 = \bar{y} - \hat{\beta}_1\bar{x}\]
The mtcars dataset in R records various stats for 32 cars from a 1974 Motor Trend magazine.
## wt mpg ## Mazda RX4 2.620 21.0 ## Mazda RX4 Wag 2.875 21.0 ## Datsun 710 2.320 22.8 ## Hornet 4 Drive 3.215 21.4 ## Hornet Sportabout 3.440 18.7
wt: weight of the car (1000 lbs)mpg: fuel efficiency (miles per gallon)Heavier cars likely burn more fuel - let’s see if the data supports that.
We can already see a negative trend - as weight increases, fuel efficiency tends to drop, suggesting an inverse relationship.
model <- lm(mpg ~ wt, data = mtcars) coef(summary(model))
## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 37.285126 1.877627 19.857575 8.241799e-19 ## wt -5.344472 0.559101 -9.559044 1.293959e-10
Residuals appear randomly scattered around zero, supporting the assumptions of the linear model.
The model estimates that for every extra 1000 lbs of weight, fuel efficiency drops by about 5.34 mpg - which makes intuitive sense.