Linear Regression Assignment
Hani Mebar
22 November 2016
Assignment Instructions
For this assignment we have been given the following scenario:
You work for Motor Trend, a magazine about the automobile industry. Looking at a data set of a collection of cars, they are interested in exploring the relationship between a set of variables and miles per gallon (MPG) (outcome). They are particularly interested in the following two questions:
- “Is an automatic or manual transmission better for MPG”"
- “Quantify the MPG difference between automatic and manual transmissions”
Part 1: Do we travel farther with Manual or Automatic - based on this data set?
As I’ve done in my previous assignments, good practice is to always take a look at the data to understand what the parameters and data are; ie do some Exploratory Data Analysis.
Let’s call up the data and get a look.
data(mtcars)
head(mtcars)## mpg cyl disp hp drat wt qsec vs am gear carb
## Mazda RX4 21.0 6 160 110 3.90 2.620 16.46 0 1 4 4
## Mazda RX4 Wag 21.0 6 160 110 3.90 2.875 17.02 0 1 4 4
## Datsun 710 22.8 4 108 93 3.85 2.320 18.61 1 1 4 1
## Hornet 4 Drive 21.4 6 258 110 3.08 3.215 19.44 1 0 3 1
## Hornet Sportabout 18.7 8 360 175 3.15 3.440 17.02 0 0 3 2
## Valiant 18.1 6 225 105 2.76 3.460 20.22 1 0 3 1Note: I added this next part after running into problems while creating plots and realized that some variables had to be converted to factors as they represented categories.
To see the what type of data we’re looking at, I ran the str() function and found all to be ‘numerical’.
str(mtcars)## 'data.frame': 32 obs. of 11 variables:
## $ mpg : num 21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
## $ cyl : num 6 6 4 6 8 6 8 4 4 6 ...
## $ disp: num 160 160 108 258 360 ...
## $ hp : num 110 110 93 110 175 105 245 62 95 123 ...
## $ drat: num 3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
## $ wt : num 2.62 2.88 2.32 3.21 3.44 ...
## $ qsec: num 16.5 17 18.6 19.4 17 ...
## $ vs : num 0 0 1 1 0 1 0 1 1 1 ...
## $ am : num 1 1 1 0 0 0 0 0 0 0 ...
## $ gear: num 4 4 4 3 3 3 3 4 4 4 ...
## $ carb: num 4 4 1 1 2 1 4 2 2 4 ...To see what some of the column names represented I checked ?mtcars. Based on the above questions, my focus is going to first be the relationship of mpg and am (automatic vs manual).
Before I can plot them, let’s convert the required categories to factors and update the label.
mtcars$am[mtcars$am==1] <- "Manual"
mtcars$am[mtcars$am==0] <- "Automatic"
mtcars$am <- as.factor(mtcars$am)
Trans <- mtcars$amI will use a boxplot to get a visual representation of these two comparisons
boxplot(mtcars$mpg ~ Trans, ylab= "mpg", xlab = "Transmission", main = "MPG vs Transmission")
So as per many an article I’ve read, manual transmission consumes less petrol per mile (MPG). For this assignment though, that is not enough: we need to prove it by checking that the null hypothesis (that there is no difference) is less than 0.05. We can do a t.test!
t.test(mpg~am, data = mtcars)##
## Welch Two Sample t-test
##
## data: mpg by am
## t = -3.7671, df = 18.332, p-value = 0.001374
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -11.280194 -3.209684
## sample estimates:
## mean in group Automatic mean in group Manual
## 17.14737 24.39231Conclusion for part 1 :
* The boxplot (visual comparison representation) showed the Manual transmission has a large MPG ratio.
* The t.test showed a p-value of 0.001374 < 0.05 which rejects the null hypothesis.Part 2: Show us the difference!
In the second part we have to quantify the MPG difference between the two transmission types.
So first we need to create the model. I will use the lm() function to get a regression model. Next I will use stepwise regression to compare all the (predictive) variabless t-tests. I will utilize the backward elimination method so we will be left with the variables that have the most impact statistically on the model fit.
Note: I needed to convert some more of the variables from Numeric to Factors as the data wasn’t all being read properly.
mtcars$gear <- as.factor(mtcars$gear)
mtcars$cyl <- as.factor(mtcars$cyl)
mtcars$vs <- as.factor(mtcars$vs)
mtcars$carb <- as.factor(mtcars$carb)
mtcarsmod <- lm(data=mtcars, mpg~.)
stepmtcarsmod <- step(mtcarsmod, direction = "backward")## Start: AIC=76.4
## mpg ~ cyl + disp + hp + drat + wt + qsec + vs + am + gear + carb
##
## Df Sum of Sq RSS AIC
## - carb 5 13.5989 134.00 69.828
## - gear 2 3.9729 124.38 73.442
## - am 1 1.1420 121.55 74.705
## - qsec 1 1.2413 121.64 74.732
## - drat 1 1.8208 122.22 74.884
## - cyl 2 10.9314 131.33 75.184
## - vs 1 3.6299 124.03 75.354
## <none> 120.40 76.403
## - disp 1 9.9672 130.37 76.948
## - wt 1 25.5541 145.96 80.562
## - hp 1 25.6715 146.07 80.588
##
## Step: AIC=69.83
## mpg ~ cyl + disp + hp + drat + wt + qsec + vs + am + gear
##
## Df Sum of Sq RSS AIC
## - gear 2 5.0215 139.02 67.005
## - disp 1 0.9934 135.00 68.064
## - drat 1 1.1854 135.19 68.110
## - vs 1 3.6763 137.68 68.694
## - cyl 2 12.5642 146.57 68.696
## - qsec 1 5.2634 139.26 69.061
## <none> 134.00 69.828
## - am 1 11.9255 145.93 70.556
## - wt 1 19.7963 153.80 72.237
## - hp 1 22.7935 156.79 72.855
##
## Step: AIC=67
## mpg ~ cyl + disp + hp + drat + wt + qsec + vs + am
##
## Df Sum of Sq RSS AIC
## - drat 1 0.9672 139.99 65.227
## - cyl 2 10.4247 149.45 65.319
## - disp 1 1.5483 140.57 65.359
## - vs 1 2.1829 141.21 65.503
## - qsec 1 3.6324 142.66 65.830
## <none> 139.02 67.005
## - am 1 16.5665 155.59 68.608
## - hp 1 18.1768 157.20 68.937
## - wt 1 31.1896 170.21 71.482
##
## Step: AIC=65.23
## mpg ~ cyl + disp + hp + wt + qsec + vs + am
##
## Df Sum of Sq RSS AIC
## - disp 1 1.2474 141.24 63.511
## - vs 1 2.3403 142.33 63.757
## - cyl 2 12.3267 152.32 63.927
## - qsec 1 3.1000 143.09 63.928
## <none> 139.99 65.227
## - hp 1 17.7382 157.73 67.044
## - am 1 19.4660 159.46 67.393
## - wt 1 30.7151 170.71 69.574
##
## Step: AIC=63.51
## mpg ~ cyl + hp + wt + qsec + vs + am
##
## Df Sum of Sq RSS AIC
## - qsec 1 2.442 143.68 62.059
## - vs 1 2.744 143.98 62.126
## - cyl 2 18.580 159.82 63.466
## <none> 141.24 63.511
## - hp 1 18.184 159.42 65.386
## - am 1 18.885 160.12 65.527
## - wt 1 39.645 180.88 69.428
##
## Step: AIC=62.06
## mpg ~ cyl + hp + wt + vs + am
##
## Df Sum of Sq RSS AIC
## - vs 1 7.346 151.03 61.655
## <none> 143.68 62.059
## - cyl 2 25.284 168.96 63.246
## - am 1 16.443 160.12 63.527
## - hp 1 36.344 180.02 67.275
## - wt 1 41.088 184.77 68.108
##
## Step: AIC=61.65
## mpg ~ cyl + hp + wt + am
##
## Df Sum of Sq RSS AIC
## <none> 151.03 61.655
## - am 1 9.752 160.78 61.657
## - cyl 2 29.265 180.29 63.323
## - hp 1 31.943 182.97 65.794
## - wt 1 46.173 197.20 68.191Now we can see the summary to make conclusions
summary(stepmtcarsmod)##
## Call:
## lm(formula = mpg ~ cyl + hp + wt + am, data = mtcars)
##
## Residuals:
## Min 1Q Median 3Q Max
## -3.9387 -1.2560 -0.4013 1.1253 5.0513
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 33.70832 2.60489 12.940 7.73e-13 ***
## cyl6 -3.03134 1.40728 -2.154 0.04068 *
## cyl8 -2.16368 2.28425 -0.947 0.35225
## hp -0.03211 0.01369 -2.345 0.02693 *
## wt -2.49683 0.88559 -2.819 0.00908 **
## amManual 1.80921 1.39630 1.296 0.20646
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 2.41 on 26 degrees of freedom
## Multiple R-squared: 0.8659, Adjusted R-squared: 0.8401
## F-statistic: 33.57 on 5 and 26 DF, p-value: 1.506e-10To interpret these results after the step procedure: * Variance: 86.59% based on the final model * MPG : decreases as the number of cyclinders goes up (coef -3.031 for 6cyl, and -2.16 for cyl) * Horsepower : decreases by MPG -0.032 * Weight : decreases MPG by -2.49 for ever 1000lbs added * Transmission: Manual transmission shows an increase in MPG by 1.8.
In conclusion, Manual transmission is better for traveling longer distances on the same tank of petrol. One caveat: the mtcars database has only 32 obvservations which means it’s a very small sample size to conclude decisively.
Some more plots to ponder over based on some swirl lessons :)
pairs(mpg ~ ., mtcars)
Residual plots
par(mfrow = c(2,2))
plot(stepmtcarsmod)