Linear regression is a technique in statistics that is used to predict values of a dependent variable (y) with an independent variable (x) with given data.
We can study the weight of a vehicle and it’s relationship to how many miles per gallon (mpg) it gets and use that to predict MPG values of varying weights.
Scatterplot of Weight vs. MPG
mtcars## mpg cyl disp hp drat wt qsec vs am gear carb ## Mazda RX4 21.0 6 160.0 110 3.90 2.620 16.46 0 1 4 4 ## Mazda RX4 Wag 21.0 6 160.0 110 3.90 2.875 17.02 0 1 4 4 ## Datsun 710 22.8 4 108.0 93 3.85 2.320 18.61 1 1 4 1 ## Hornet 4 Drive 21.4 6 258.0 110 3.08 3.215 19.44 1 0 3 1 ## Hornet Sportabout 18.7 8 360.0 175 3.15 3.440 17.02 0 0 3 2 ## Valiant 18.1 6 225.0 105 2.76 3.460 20.22 1 0 3 1 ## Duster 360 14.3 8 360.0 245 3.21 3.570 15.84 0 0 3 4 ## Merc 240D 24.4 4 146.7 62 3.69 3.190 20.00 1 0 4 2 ## Merc 230 22.8 4 140.8 95 3.92 3.150 22.90 1 0 4 2 ## Merc 280 19.2 6 167.6 123 3.92 3.440 18.30 1 0 4 4 ## Merc 280C 17.8 6 167.6 123 3.92 3.440 18.90 1 0 4 4 ## Merc 450SE 16.4 8 275.8 180 3.07 4.070 17.40 0 0 3 3 ## Merc 450SL 17.3 8 275.8 180 3.07 3.730 17.60 0 0 3 3 ## Merc 450SLC 15.2 8 275.8 180 3.07 3.780 18.00 0 0 3 3 ## Cadillac Fleetwood 10.4 8 472.0 205 2.93 5.250 17.98 0 0 3 4 ## Lincoln Continental 10.4 8 460.0 215 3.00 5.424 17.82 0 0 3 4 ## Chrysler Imperial 14.7 8 440.0 230 3.23 5.345 17.42 0 0 3 4 ## Fiat 128 32.4 4 78.7 66 4.08 2.200 19.47 1 1 4 1 ## Honda Civic 30.4 4 75.7 52 4.93 1.615 18.52 1 1 4 2 ## Toyota Corolla 33.9 4 71.1 65 4.22 1.835 19.90 1 1 4 1 ## Toyota Corona 21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1 ## Dodge Challenger 15.5 8 318.0 150 2.76 3.520 16.87 0 0 3 2 ## AMC Javelin 15.2 8 304.0 150 3.15 3.435 17.30 0 0 3 2 ## Camaro Z28 13.3 8 350.0 245 3.73 3.840 15.41 0 0 3 4 ## Pontiac Firebird 19.2 8 400.0 175 3.08 3.845 17.05 0 0 3 2
We can use linear regression to predict MPG (y) from weight (x) in the form of the “best fit line”.
The equation for linear regression is \[\hat{y}_i = \beta_{0} + \beta_{1} x_{i}\]
Where
- \(\hat{y}_i\) is the predicted value.
- \(\beta_{0}\) is the intercept, the value when the independent variable (x) is 0.
- \(\beta_{1} x_{i}\) is the change of the predicted value (slope).
This is essentially an equation for a line.
\[y = mx + b\]
\(\beta_{o}\) and \(\beta_{1} x_{i}\) must be chosen to produce a \(\hat{y}_i\) point in such a way that the distance (residuals) between the line and the actual values is as minimal as possible.
\(\beta_{1} = \frac{\sum (x_{i} - \overline{x})(y_{i} - \overline{y})} {\sum (x_{i} - \overline{x})^2}\)
\(\beta_{o} = \overline{y} - \beta_{1}\overline{x}\)
Where
\(\overline{x}\) is the average of x
\(\overline{y}\) is the average of y
Using Microsoft Excel to manually calculate \(\beta_{1}\) and \(\beta_{o}\).
\(\beta_{1}\) is 37.28513 \(\beta_{o}\) is -5.34447
summary(lm(mpg~wt, mtcars))
## ## Call: ## lm(formula = mpg ~ wt, data = mtcars) ## ## Residuals: ## Min 1Q Median 3Q Max ## -4.5432 -2.3647 -0.1252 1.4096 6.8727 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 37.2851 1.8776 19.858 < 2e-16 *** ## wt -5.3445 0.5591 -9.559 1.29e-10 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 3.046 on 30 degrees of freedom ## Multiple R-squared: 0.7528, Adjusted R-squared: 0.7446 ## F-statistic: 91.38 on 1 and 30 DF, p-value: 1.294e-10
\(\beta_{o}\) is the (Intercept) value under the Estimate column (37.2851) \(\beta_{1}\) is the wt (x value name) value under the Estimate column (-5.3445)
Formula: \(\hat{y}_i = -5.3445x_{i} + 37.2851\)
Using geom_abline(), we can add this info to your previous plot. geom_abline(intercept = 37.2851, slope = -5.3445,)
ggplot2 was used for the plot on the previous slide
ggplot(data=mtcars, aes(x=wt, y=mpg,)) +
geom_point() + ggtitle("Weight vs. MPG") +
xlab("Weight (lb/1000)") +
ylab("MPG") +
geom_abline(aes(intercept = 37.2851, slope = -5.3445, colour = "Best Fit")) +
scale_colour_manual(values="red")
mpgR = mtcars[,'mpg'] wtR = mtcars[,'wt'] resMod = lm(mpgR~wtR) res = resid(resMod) plot(fitted(resMod), res, ylab = "distance from best fit (mpg)", xlab = "mpg", main="Residuals of MPG vs best fit line") abline(0,0)
We can also leverage built in functions to get the line of best fit.
geom_smooth() with the method “lm” (linear models) will give the line of best fit.
geom_smooth(method = "lm", formula = y ~ x, aes(colour="Best Fit"))
ggplot2 was used for the plot on the previous slide
ggplot(data=mtcars, aes(x=wt, y=mpg,)) +
geom_point() + ggtitle("Weight vs. MPG") +
xlab("Weight (lb/1000)") +
ylab("MPG") +
geom_smooth(method = "lm", formula = y ~ x, aes(colour="Best Fit")) +
scale_colour_manual(name="", values="red")
plotly was used for the plot on the previous slide
plotlyLR = plot_ly(data=mtcars, x=~wt, y=~mpg, type="scatter", mode="markers",
name = "MPG data") %>%
layout(title = "Weight vs. MPG",
xaxis = list(title = "Weight (lb/1000)"),
yaxis = list (title = "Miles Per Gallon"))
add_lines(p=plotlyLR, x=~wt, y=fitted(lm(mpg~wt, mtcars)),
name = "Best Fit")
\[\hat{y}_i = -5.3445x_{i} + 37.2851\] Using the simple linear regression formula, we can predict the MPG of a vehicle weighting 5000 pounds.
Plugging values in
(note, units are (lb/1000) so x = 5)
\(\hat{y}_i = -5.3445(5) + 37.2851\)
Using R code
prediction = function(x) {
yhat = (-5.3445*x) + 37.2851
return(yhat)
}
prediction(5)
## [1] 10.5626
The predicted MPG for a vehicle weighing 5000 pounds is….
\(\hat{y}_i= 10.5626\) \(mpg\)
predict() functionWe can also use the predict() function in R to predict this value.
tempDF = data.frame(wt = 5) tempLM = lm(formula=mpg~wt, data=mtcars) predict(tempLM, tempDF)
## 1 ## 10.56277
The answer using predict is very close to the answer manually calculated from the previous slide.