DATA606_chapter8: Multiple and Logistic Regression
Yun Mai
May 5, 2017
library(xtable)
library(knitr)
library(ggplot2)## Warning: package 'ggplot2' was built under R version 3.3.3Multiple and Logistic Regression
Graded: 8.2, 8.4, 8.8, 8.16, 8.188.5.1 Introduction to multiple regression
8.2 Baby weights, Part II. Exercise 8.1 introduces a data set on birth weight of babies. Another variable we consider is parity, which is 0 if the child is the first born, and 1 otherwise.
The summary table below shows the results of a linear regression model for predicting the average birth weight of babies, measured in ounces, from parity.
| Estimate | Std. Error | t value | Pr(>|t|) | |
|---|---|---|---|---|
| (Intercept) | 120.07 | 0.60 | 199.94 | 0.0000 |
| parity | -1.93 | 1.19 | -1.62 | 0.1052 |
- Write the equation of the regression line.
\[\hat{bodyweight}=120.07-1.93 \times parity \]
- Interpret the slope in this context, and calculate the predicted birth weight of first borns and others.
The slope means that when the baby is first born, the estimated birth weight of the baby will decrease 1.93 ounces.
\[\hat{bodyweight_{first}}=120.07-1.93 \times parity = 120.07-1.93 \times 1 = 118.14 ounces\] \[\hat{bodyweight_{first}}=120.07-1.93 \times parity = 120.07-1.93 \times 0 = 120.07 ounces\]
- Is there a statistically significant relationship between the average birth weight and parity?
P value is 0.1052 which is larger than 0.05.So there is no significan relationship between the average birth weight and parity.
8.4 Absenteeism. Researchers interested in the relationship between absenteeism from school and certain demographic characteristics of children collected data from 146 randomly sampled students in rural New South Wales, Australia, in a particular school year. Below are three observations from this data set.
| eth | sex | lrn | days | |
|---|---|---|---|---|
| 1 | 0 | 1 | 1 | 2 |
| 2 | 0 | 1 | 1 | 11 |
| . | . | . | . | . |
| . | . | . | . | . |
| . | . | . | . | . |
| . | . | . | . | . |
| . | . | . | . | . |
| 146 | 1 | 0 | 0 | 37 |
The summary table below shows the results of a linear regression model for predicting the average number of days absent based on ethnic background (eth: 0 - aboriginal, 1 - not aboriginal), sex (sex: 0 - female, 1 - male), and learner status (lrn: 0 - average learner, 1 - slow learner).18
| Estimate | Std. Error | t value | Pr(>|t|) | |
|---|---|---|---|---|
| (Intercept) | 18.93 | 2.57 | 7.37 | 0.0000 |
| eth | -9.11 | 2.60 | -3.51 | 0.0000 |
| sex | 3.10 | 2.64 | 1.18 | 0.2411 |
| lrn | 2.15 | 2.65 | 0.81 | 0.4177 |
- Write the equation of the regression line.
\[\hat{days}=18.93-9.11 \times eth + 3.10 \times sex + 2.15 \times lrn\]
- Interpret each one of the slopes in this context. given that all other variables are held constant, or controled for, then
a non-aboriginal student has 1.93 days less in average number of days absent than a aboriginal student;
a male student has 3.10 days more in average number of days absent than a female student;
a slow learner has 2.15 days more in average number of days absent than an average learner.
- Calculate the residual for the first observation in the data set: a student who is aboriginal,male, a slow learner, and missed 2 days of school.
\[\hat{e_1} = y_1 - \hat{y_1}\] \[\hat{e_1} = 2 - (18.93-9.11 \times 0 + 3.10 \times 1 + 2.15 \times 1)\] \[\hat{e_1} = -22.18\]
- The variance of the residuals is 240.57, and the variance of the number of absent days for all students in the data set is 264.17. Calculate the R2 and the adjusted R2. Note that there are 146 observations in the data set.
VAR.e <- 240.57
VAR.y <- 264.17
n <- 146
k <- 3
R2 <- 1-(VAR.e/VAR.y)
R2.adj <- 1-(VAR.e/VAR.y)*(n-1)/(n-k-1)
paste("R2 = ",round(R2,4))## [1] "R2 = 0.0893"paste("R2.adj = ",round(R2.adj,4))## [1] "R2.adj = 0.0701"8.5.2 Model selection
8.8 Absenteeism, Part II. Exercise 8.4 considers a model that predicts the number of days absent using three predictors: ethnic background (eth), gender (sex), and learner status (lrn). The table below shows the adjusted R-squared for the model as well as adjusted R-squared values for all models we evaluate in the first step of the backwards elimination process.
| Model | Adjusted R2 | |
|---|---|---|
| 1 | Fullmodel | 0.0701 |
| 2 | Noethnicity | -0.0033 |
| 3 | Nosex | 0.0676 |
| 4 | No learner status | 0.0723 |
Which, if any, variable should be removed from the model first?
When taking out learner status, adjusted \(R^2\) increased from 0.0701 to 0.0723, so learner status should be removed first.
8.5.4 Introduction to logistic regression
8.16 Challenger disaster, Part I. On January 28, 1986, a routine launch was anticipated for the Challenger space shuttle. Seventy-three seconds into the flight, disaster happened: the shuttle broke apart, killing all seven crew members on board. An investigation into the cause of the disaster focused on a critical seal called an O-ring, and it is believed that damage to these O-rings during a shuttle launch may be related to the ambient temperature during the launch. The table below summarizes observational data on O-rings for 23 shuttle missions, where the mission order is based on the temperature at the time of the launch. Temp gives the temperature in Fahrenheit, Damaged represents the number of damaged O-rings, and Undamaged represents the number of O-rings that were not damaged.
table8.161 <- data.frame(v0=c("Temperature","Damaged","Undamaged"),v1=c(53,5,1),v2=c(57,1,5),v3=c(58,1,5),v4=c(63,1,5),v5=c(66,0,6),v6=c(67,0,6),v7=c(67,0,6),v8=c(67,0,6),v9=c(68,0,6),v10=c(69,0,6),v11=c(70,1,5),v12=c(70,0,6),v13=c(70,1,5),v14=c(70,0,6),v15=c(72,0,6),v16=c(73,0,6),v17=c(75,0,6),v18=c(75,1,5),v19=c(76,0,6),v20=c(76,0,6),v21=c(78,0,6),v22=c(79,0,6),v23=c(81,0,6))
colnames(table8.161) <- c("Shuttle Mission",as.character(seq(1:23)))
kable(table8.161,header=F)| Shuttle Mission | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Temperature | 53 | 57 | 58 | 63 | 66 | 67 | 67 | 67 | 68 | 69 | 70 | 70 | 70 | 70 | 72 | 73 | 75 | 75 | 76 | 76 | 78 | 79 | 81 |
| Damaged | 5 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 |
| Undamaged | 1 | 5 | 5 | 5 | 6 | 6 | 6 | 6 | 6 | 6 | 5 | 6 | 5 | 6 | 6 | 6 | 6 | 5 | 6 | 6 | 6 | 6 | 6 |
- Each column of the table above represents a different shuttle mission. Examine these data and describe what you observe with respect to the relationship between temperatures and damaged O-rings.
temperatures <- c(53, 57, 58, 63, 66, 67, 67, 67, 68, 69, 70, 70, 70, 70, 72, 73, 75, 75, 76, 76, 78, 79, 81)
damage_o_ring <- c( 5, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0)
plot(temperatures,jitter(damage_o_ring))
When the temperatures is higher then 66 Fahrenheit, O-rings are very often not damaged.
Failures have been coded as 1 for a damaged O-ring and 0 for an undamaged O-ring, and a logistic regression model was fit to these data. A summary of this model is given below. Describe the key components of this summary table in words.
Est imate Std . Error z v alue Pr( >|z|) (Intercept) 11.6630 3.2963 3.54 4e-04 Temperature -0.2162 0.0532 -4.07 0e+00
The purpose of this logistics regression model is to find out the probability of failures of o-ring. The reading is similar to linear regression model. The row labeled intercept represents the information for the intercept. The intercepts is the value of damage o-ring when temperature is 0. The row labeled temperature represents the information for the slope, which is the coefficient of the temperature variable.
The intercept of the model is 11.63. The transformed probability of O-ring failures (logit(\(p_i\))) will decrease 0.2162 when the temperature increase an extra 1 Fahrenheit degree. The second column is a standard error for the slope: 0.0532. The third column is a Z-test statistic for the null hypothesis that \(\beta_1\) = 0: Z = -4.07. The fourth column is p-value for a two-sided alternative hypothes is: 0.0000.
- Write out the logistic model using the point estimates of the model parameters.
\[log\frac{\hat{p}}{1-\hat{p}} = 11.6630 -0.2162\times temperature \]
- Based on the model, do you think concerns regarding O-rings are justified? Explain.
The concerns regarding o-rings are justified. The P value is 0.0000 which is smaller than 0.05, suggesting that the coefficient of temperature is not 0. The higher the temperature, the smaller the logit(\(p_i\)) and \(p_i\).
8.18 Challenger disaster, Part II. Exercise 8.16 introduced us to O-rings that were identified as a plausible explanation for the breakup of the Challenger space shuttle 73 seconds into takeoff in 1986. The investigation found that the ambient temperature at the time of the shuttle launch was closely related to the damage of O-rings, which are a critical component of the shuttle. See this earlier exercise if you would like to browse the original data.
- The data provided in the previous exercise are shown in the plot. The logistic model fit to these data may be written as
\[log\frac{\hat{p}}{1-\hat{p}} = 11.6630 -0.2162\times Temperature \]
where \(\hat{p}\) is the model-estimated probability that an O-ring will become damaged. Use the model to calculate the probability that an O-ring will become damaged at each of the following ambient temperatures: 51, 53, and 55 degrees Fahrenheit. The model-estimated probabilities for several additional ambient temperatures are provided below, where subscripts indicate the temperature: \[\hat{p}_{57} = 0.341, \hat{p}_{59}= 0.251, \hat{p}_{61} = 0.179, \hat{p}_{63}= 0.124,\hat{p}_{65} = 0.084,\hat{p}_{67} = 0.056, \hat{p}_{69} = 0.037, \hat{p}_{71} = 0.024\]
temp <- 51
p_51 <- exp(11.6630 -0.2162* temp)/(1+exp(11.6630 -0.2162* temp))
paste("p_51 =",p_51) ## [1] "p_51 = 0.654029738456095"temp <- 53
p_53 <- exp(11.6630 -0.2162* temp)/(1+exp(11.6630 -0.2162* temp))
paste("p_53 =",p_53) ## [1] "p_53 = 0.550922829727926"temp <- 55
p_55 <- exp(11.6630 -0.2162* temp)/(1+exp(11.6630 -0.2162* temp))
paste("p_55 =",p_55) ## [1] "p_55 = 0.443245647107059"- Add the model-estimated probabilities from part (a) on the plot, then connect these dots using a smooth curve to represent the model-estimated probabilities.
temp <- 73
p_73 <- exp(11.6630 -0.2162* temp)/(1+exp(11.6630 -0.2162* temp))
paste("p_73 =",p_73) ## [1] "p_73 = 0.0159911412936585"temp <- 75
p_75 <- exp(11.6630 -0.2162* temp)/(1+exp(11.6630 -0.2162* temp))
paste("p_75 =",p_75) ## [1] "p_75 = 0.0104360317538603"temp <- 77
p_77 <- exp(11.6630 -0.2162* temp)/(1+exp(11.6630 -0.2162* temp))
paste("p_77 =",p_77) ## [1] "p_77 = 0.00679736276932658"temp <- 79
p_79 <- exp(11.6630 -0.2162* temp)/(1+exp(11.6630 -0.2162* temp))
paste("p_79 =",p_79) ## [1] "p_79 = 0.00442169796103641"temp <- 81
p_81 <- exp(11.6630 -0.2162* temp)/(1+exp(11.6630 -0.2162* temp))
paste("p_81 =",p_81) ## [1] "p_81 = 0.00287392077328203"temperatures <- c(51,53,55,57,59,61,63,65,67,69,71,73,75,77,79,81)
probability <- c(round(p_51,3), round(p_53,3), round(p_55,3), 0.341, 0.251, 0.179, 0.124, 0.084, 0.056, 0.037, 0.024,round(p_73,3), round(p_75,3),round(p_77,3),round(p_79,3),round(p_81,3))
df <- data.frame(temperature=temperatures, probability = probability)
head(df, 5)## temperature probability
## 1 51 0.654
## 2 53 0.551
## 3 55 0.443
## 4 57 0.341
## 5 59 0.251x <- df$temperature
y <- df$probability
ggplot(df,aes(x=temperature,y=probability)) + labs(x='Temperature (Fahrenheit)', y='Probability of damage') + geom_point() + geom_smooth(formula = y ~ splines::ns(x,2),se = TRUE,method='lm')
temperatures <- c(53, 57, 58, 63, 66, 67, 67, 67, 68, 69, 70, 70, 70, 70, 72, 73, 75, 75, 76, 76, 78, 79, 81)
damage_o_ring <- c( 5, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0)
df2 <- data.frame(temperatures=temperatures, damage_o_ring=damage_o_ring)
df2$true <- df2$damage_o_ring/6
df3 <- rbind(df2[which(df2$temperatures == 53),],df2[which(df2$temperatures == 57),], df2[which(df2$temperatures == 63),], df2[which(df2$temperatures == 67),],df2[which(df2$temperatures == 69),],df2[which(df2$temperatures == 73),],df2[which(df2$temperatures == 75),],df2[which(df2$temperatures == 79),],df2[which(df2$temperatures == 81),])df4 <- rbind(df[which(df$temperature == 53),],df[which(df$temperature == 57),], df[which(df$temperature == 63),], df[which(df$temperature == 67),],df[which(df$temperature == 67),],df[which(df$temperature == 67),],df[which(df$temperature == 69),],df[which(df$temperature == 73),],df[which(df$temperature == 75),],df[which(df$temperature == 75),],df[which(df$temperature == 79),],df[which(df$temperature == 81),])
df.ns <- data.frame(df4$probability, df3$true)
x <- df4$probability
y <- df3$true
ggplot(df.ns,aes(x=x,y=jitter(y)))+ labs(x='Predicted probability', y='Truth') + geom_point() + geom_smooth(formula = y ~ splines::ns(x, 2),se = TRUE,method='lm')+geom_abline(x=y)## Warning: Ignoring unknown parameters: x
- Describe any concerns you may have regarding applying logistic regression in this application, and note any assumptions that are required to accept the model’s validity.
The concern regarding applying logistic regression in this application is that the curve fit with natural splines and its confidence bound does not closely follow the y = x line. The deviations indicate the model relating the parameter to the predictors does not closely resemble the true relationship.
The assumptions for applying this model are:
- the predictor temperature is linearly related to logit(\(p_i\)).
- Each outcome, the probability of failure of o-rings, is independent of the other outcomes.