Introduction to Linear Regression
Brian K. Liles
November 6, 2017
The Data
download.file("http://www.openintro.org/stat/data/mlb11.RData", destfile = "mlb11.RData")
load("mlb11.RData")Exercise 1
What type of plot woud you use to display the relationship between runs and one of the other numerical variables?
You would use a scatterplot.
Plot this relationship using the variable at_bats as the predictor.
#simple scatterplot
#the basic function is plot(x,y) where x and y are numeric vectors denoting the (x,y) points to plot
plot(mlb11$at_bats, mlb11$runs, main ="Run/At Bats Scatterplot", xlab="# of At Bats", ylab="Runs Scored")
#Traditionally the dependent or response variable is plotted on the vertical or Y-axis, while the "independent"" or predictor variable is plotted on the horizontal or X-axis.Does the relationship look linear? The relationship between both variables appear somewhat linear.
If you knew a team’s at_bats, would you be comfortable using a linear model to predict the number of runs? Yes
cor(mlb11$runs, mlb11$at_bats)## [1] 0.610627Sum of Squared Residuals
Exercise 2
Looking at your plot from the previous exercise, describe the relationship between these two variables. Make sure to discuss the form, direction, and strength of the relationship as well as any unusual observations.
Linear, positive with constant variability.
#We can summarize the relationship between these two variables by finding the line that best follows their association.
plot_ss(x = mlb11$at_bats, y = mlb11$runs)
## Click two points to make a line.
## Call:
## lm(formula = y ~ x, data = pts)
##
## Coefficients:
## (Intercept) x
## -2789.2429 0.6305
##
## Sum of Squares: 123721.9#The most common way to do linear regression is to select the line that minimizes the sum of squared residuals.To visualize the squared residuals, you can return the plot command and add the argument showSquares = TRUE
plot_ss(x = mlb11$at_bats, y = mlb11$runs, showSquares = TRUE)
## Click two points to make a line.
## Call:
## lm(formula = y ~ x, data = pts)
##
## Coefficients:
## (Intercept) x
## -2789.2429 0.6305
##
## Sum of Squares: 123721.9Exercise 3
Using plot_ss, choose a line that does a good job of minimizing the sum of squares. Run the function several times. What was the smallest sum of squares that you got? How does it compare to your neighbors?
The Sum of Squares = 123721.9
The Linear Model
#use the lm function to fit the linear model aka regression line
m1 <- lm(runs ~ at_bats, data=mlb11)
#first argument y ~ x
#second argument specifies thes mlb11 dataf frame
summary(m1)##
## Call:
## lm(formula = runs ~ at_bats, data = mlb11)
##
## Residuals:
## Min 1Q Median 3Q Max
## -125.58 -47.05 -16.59 54.40 176.87
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -2789.2429 853.6957 -3.267 0.002871 **
## at_bats 0.6305 0.1545 4.080 0.000339 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 66.47 on 28 degrees of freedom
## Multiple R-squared: 0.3729, Adjusted R-squared: 0.3505
## F-statistic: 16.65 on 1 and 28 DF, p-value: 0.0003388Let’s consider this output piece by piece. First, the formula used to describe the model is shown at the top. After the formula you find the five-number summary of the residuals. The “Coefficients” table shown next is key; its first column displays the linear model’s y-intercept and the coefficient of at_bats. With this table, we can write down the least squares regression line for the linear model:
y^=???2789.2429+0.6305???atbats
One last piece of information we will discuss from the summary output is the Multiple R-squared, or more simply, R2. The R2 value represents the proportion of variability in the response variable that is explained by the explanatory variable. For this model, 37.3% of the variability in runs is explained by at-bats.
Exercise 4
Fit a new model that uses homeruns to predict runs. Using the estimates from the R ouput, write the equation of the regression line. What does the slope tell us in the context of the relationship between success of a team and its home runs?
m2 <- lm(runs ~ homeruns, data = mlb11)
summary(m2)##
## Call:
## lm(formula = runs ~ homeruns, data = mlb11)
##
## Residuals:
## Min 1Q Median 3Q Max
## -91.615 -33.410 3.231 24.292 104.631
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 415.2389 41.6779 9.963 1.04e-10 ***
## homeruns 1.8345 0.2677 6.854 1.90e-07 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 51.29 on 28 degrees of freedom
## Multiple R-squared: 0.6266, Adjusted R-squared: 0.6132
## F-statistic: 46.98 on 1 and 28 DF, p-value: 1.9e-07Prediction and Prediction Errors
plot(mlb11$runs ~ mlb11$at_bats)
abline(m1)
Exercise 5
If a team manager saw the least squares regression line and not the actual data, how many runs would he or she predict for a team with 5,578 at-bats?
He would predict between 700 and 750 runs.
Is this an overestimate or an underestimate, and by how much? In other words, what is the residual for this prediction?
Its an overestimate because the Philadelphia Phillies had 5579 at bats and had 713 runs.
Model Diagnostics
plot(m1$residuals ~ mlb11$at_bats)
abline(h = 0, lty = 3) #adds a horizontal dashed line @ y=0
Exercise 6
Is there any apparent pattern in the residuals plot? What does this indicate about the linearity of the relationship between runs and at-bats?
It is linear
hist(m1$residuals)
Exercise 7
Based on the histogram and the normal probability plot, does the nearly normal residuals condition appear to be met?
Yes
Exercise 8
Based on the plot in (1), does the constant variability condition appear to be met?
Yes, the constant variability appears to be met
On Your Own
Choose another traditional variable from mlb11 that you think might be a good predictor of runs. Produce a scatterplot of the two variables and fit a linear model. At a glance, does there seem to be a linear relationship?
plot(mlb11$bat_avg, mlb11$runs, main ="Run/Batting Avg Scatterplot", xlab="Batting Avg", ylab="Runs Scored")
How does this relationship compare to the relationship between runs and at_bats? Use the R2 values from the two model summaries to compare. Does your variable seem to predict runs better than at_bats? How can you tell?
cor(mlb11$bat_avg, mlb11$runs)## [1] 0.8099859Now that you can summarize the linear relationship between two variables, investigate the relationships between runs and each of the other five traditional variables. Which variable best predicts runs? Support your conclusion using the graphical and numerical methods we’ve discussed (for the sake of conciseness, only include output for the best variable, not all five).
Batting average predicts the runs best.
m3=lm(runs~bat_avg,data=mlb11)
plot(mlb11$bat_avg,mlb11$runs,xlab="Batting Avergae",ylab="Runs Scored")
abline(m3)
summary(m3)##
## Call:
## lm(formula = runs ~ bat_avg, data = mlb11)
##
## Residuals:
## Min 1Q Median 3Q Max
## -94.676 -26.303 -5.496 28.482 131.113
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -642.8 183.1 -3.511 0.00153 **
## bat_avg 5242.2 717.3 7.308 5.88e-08 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 49.23 on 28 degrees of freedom
## Multiple R-squared: 0.6561, Adjusted R-squared: 0.6438
## F-statistic: 53.41 on 1 and 28 DF, p-value: 5.877e-08sum(m3$residuals^2)## [1] 67849.52Now examine the three newer variables. These are the statistics used by the author of Moneyball to predict a teams success. In general, are they more or less effective at predicting runs that the old variables? Explain using appropriate graphical and numerical evidence. Of all ten variables we’ve analyzed, which seems to be the best predictor of runs? Using the limited (or not so limited) information you know about these baseball statistics, does your result make sense?
Check the model diagnostics for the regression model with the variable you decided was the best predictor for runs.