AS4-1 Understanding Why People Vote

1.

1.1 Exploration and Logistic Regression

What proportion of people in this dataset voted in this election?

options(digits=6, scipen=12)
Gerber = read.csv("data/gerber.csv")
table(Gerber$voting)/nrow(Gerber)

     0      1 
0.6841 0.3159 

1.2 Exploration and Logistic Regression

Which of the four “treatment groups” had the largest percentage of people who actually voted (voting = 1)?

tapply(Gerber$voting, Gerber$civicduty, mean)

       0        1 
0.316070 0.314538 

tapply(Gerber$voting, Gerber$hawthorne, mean)

       0        1 
0.315091 0.322375 

tapply(Gerber$voting, Gerber$self, mean)

       0        1 
0.312245 0.345151 

tapply(Gerber$voting, Gerber$neighbors, mean)

       0        1 
0.308151 0.377948 

• Neighbors

1.3 Exploration and Logistic Regression

Build a logistic regression model for voting using the four treatment group variables as the independent variables (civicduty, hawthorne, self, and neighbors). Use all the data to build the model (DO NOT split the data into a training set and testing set). Which of the following coefficients are significant in the logistic regression model? Select all that apply.

model = glm(voting ~ civicduty+hawthorne+self+ neighbors,Gerber,family="binomial")
summary(model)

Call:
glm(formula = voting ~ civicduty + hawthorne + self + neighbors, 
    family = "binomial", data = Gerber)

Deviance Residuals: 
   Min      1Q  Median      3Q     Max  
-0.974  -0.869  -0.839   1.459   1.559  

Coefficients:
            Estimate Std. Error z value        Pr(>|z|)    
(Intercept) -0.86336    0.00501 -172.46         < 2e-16 ***
civicduty    0.08437    0.01210    6.97 0.0000000000031 ***
hawthorne    0.12048    0.01204   10.01         < 2e-16 ***
self         0.22294    0.01187   18.79         < 2e-16 ***
neighbors    0.36509    0.01168   31.26         < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 429238  on 344083  degrees of freedom
Residual deviance: 428090  on 344079  degrees of freedom
AIC: 428100

Number of Fisher Scoring iterations: 4

• Civic Duty
• Hawthorne Effect
• Self
• Neighbors

1.4 Exploration and Logistic Regression

Using a threshold of 0.3, what is the accuracy of the logistic regression model? (When making predictions, you don’t need to use the newdata argument since we didn’t split our data.)

model = glm(voting ~ civicduty+hawthorne+self+ neighbors,Gerber,family="binomial")
Pred = predict(model,type="response")
table(Gerber$voting,Pred>0.3)

   
     FALSE   TRUE
  0 134513 100875
  1  56730  51966

(134513+51966)/nrow(Gerber)

[1] 0.541958

1.5 Exploration and Logistic Regression

Using a threshold of 0.5, what is the accuracy of the logistic regression model?

table(Gerber$voting,Pred>0.5)

   
     FALSE
  0 235388
  1 108696

235388/nrow(Gerber)

[1] 0.6841

1.6 Exploration and Logistic Regression

Compare your previous two answers to the percentage of people who did not vote (the baseline accuracy) and compute the AUC of the model. What is happening here?

table(Gerber$voting)

     0      1 
235388 108696 

235388/nrow(Gerber)

[1] 0.6841

library(ROCR)

package 愼㸱愼㸵ROCR愼㸱愼㸶 was built under R version 3.4.4Loading required package: gplots
package 愼㸱愼㸵gplots愼㸱愼㸶 was built under R version 3.4.4
Attaching package: 愼㸱愼㸵gplots愼㸱愼㸶

The following object is masked from 愼㸱愼㸵package:stats愼㸱愼㸶:

    lowess

ROCRpredTest = prediction(Pred,Gerber$voting)
auc = as.numeric(performance(ROCRpredTest, "auc")@y.values)
auc

[1] 0.530846

• Even though all of the variables are significant, this is a weak predictive model.

2

2.1 Trees

Leave all the parameters at their default values. You can use the following command in R to build the tree.Plot the tree. What happens, and if relevant, why?

library(rpart)

package 愼㸱愼㸵rpart愼㸱愼㸶 was built under R version 3.4.4

CARTmodel = rpart(voting ~ civicduty + hawthorne + self + neighbors, data=Gerber)
library(rpart.plot)

package 愼㸱愼㸵rpart.plot愼㸱愼㸶 was built under R version 3.4.4

prp(CARTmodel)

• No variables are used (the tree is only a root node) - none of the variables make a big enough effect to be split on.

2.2 Trees

….to force the complete tree to be built. Then plot the tree. What do you observe about the order of the splits?

CARTmodel2 = rpart(voting ~ civicduty + hawthorne + self + neighbors, data=Gerber, cp=0.0)
prp(CARTmodel2)

• Neighbor is the first split, civic duty is the last.

2.3 Trees

Using only the CART tree plot, determine what fraction (a number between 0 and 1) of “Civic Duty” people voted:

• 0.31

2.4 Trees

Make a new tree that includes the “sex” variable, again with cp = 0.0. Notice that sex appears as a split that is of secondary importance to the treatment group.

In the control group, which gender is more likely to vote?

CARTmodel3 = rpart(voting ~ civicduty + hawthorne + self + neighbors + sex, data=Gerber, cp=0.0)
prp(CARTmodel3)

Control = subset(Gerber,control==1)
tapply(Control$voting,Control$sex,mean)

       0        1 
0.302795 0.290456 

• Men

In the “Civic Duty” group, which gender is more likely to vote?

• Men

### 3

#### 3.1 - Interaction Terms

Let’s just focus on the “Control” treatment group. Create a regression tree using just the “control” variable, then create another tree with the “control” and “sex” variables, both with cp=0.0.

In the “control” only tree, what is the absolute value of the difference in the predicted probability of voting between being in the control group versus being in a different group? You can use the absolute value function to get answer, i.e. abs(Control Prediction - Non-Control Prediction). Add the argument “digits = 6” to the prp command to get a more accurate estimate.

Controlmodel = rpart(voting ~ control, data=Gerber, cp=0.0)
Controlmodel2 = rpart(voting ~ control+sex, data=Gerber, cp=0.0)
prp(Controlmodel,digits = 6)

abs(0.34-0.296638)

[1] 0.043362

3.2 - Interaction Terms

Now, using the second tree (with control and sex), determine who is affected more by NOT being in the control group (being in any of the four treatment groups):

prp(Controlmodel2,digits = 6)

• They are affected about the same (change in probability within 0.001 of each other).

3.3 - Interaction Terms

Going back to logistic regression now, create a model using “sex” and “control”. Interpret the coefficient for “sex”:

LogModelSex = glm(voting ~ control+sex, data=Gerber,family = "binomial")

• Coefficient is negative, reflecting that women are less likely to vote

3.4 - Interaction Terms

We can quantify this precisely. Create the following dataframe (this contains all of the possible values of sex and control), and evaluate your logistic regression using the predict function (where “LogModelSex” is the name of your logistic regression model that uses both control and sex):

Possibilities = data.frame(sex=c(0,0,1,1),control=c(0,1,0,1))
predict(LogModelSex, newdata=Possibilities, type="response")

       1        2        3        4 
0.346256 0.302446 0.333738 0.290806 

#( (Man, Not Control), (Man, Control), (Woman, Not Control), (Woman, Control) )
abs(0.290456-0.290806 )

[1] 0.00035

3.5 - Interaction Terms

So the difference is not too big for this dataset, but it is there. We’re going to add a new term to our logistic regression now, that is the combination of the “sex” and “control” variables - so if this new variable is 1, that means the person is a woman AND in the control group. We can do that with the following command

How do you interpret the coefficient for the new variable in isolation? That is, how does it relate to the dependent variable?

LogModel2 = glm(voting ~ sex + control + sex:control, data=Gerber, family="binomial")

• If a person is a woman and in the control group, the chance that she voted goes down.

3.6 - Interaction Terms

Now what is the difference between the logistic regression model and the CART model for the (Woman, Control) case? Again, give your answer with five numbers after the decimal point.

predict(LogModel2, newdata=Possibilities, type="response")

       1        2        3        4 
0.345818 0.302795 0.334176 0.290456 

abs(0.290456-0.290456)

[1] 0

3.7 - Interaction Terms

This example has shown that trees can capture nonlinear relationships that logistic regression can not, but that we can get around this sometimes by using variables that are the combination of two variables. Should we always include all possible interaction terms of the independent variables when building a logistic regression model?

• NO