Goal

It has been a while since we did logistic regression!! We are going to use our time today to review concepts for logistic regression to help prepare you for the final exam. If you have questions about anything, please let me know!

The Data

In this lab, we will return to the data set about the Titanic. Recall that the RMS Titanic was a huge, luxury passenger liner designed and built in the early 20th century. Despite the fact that the ship was believed to be unsinkable, during her maiden voyage on April 15, 1912, the Titanic collided with an iceberg and sank.

We have information on \(n = 714\) passengers from the Titanic. Unlike last time, today we are interested in modeling \(Y =\):

• Survived: Whether or not each passenger survived (0 = no, 1 = yes)

In addition to this \(Y\) variable, we have 9 possible explanatory variables:

• PassengerId: a unique number assigned to each passenger to identify them.
• Pclass: The class of the ticket held by the passenger; 1 = 1st class, 2 = 2nd class, 3 = 3rd class.
• Name: the name of each passenger.
• Sex: the sex of the passenger, limited to male and female.
• Age: The age of the passenger in years. Note: Decimals are for children under a year in age.
• SibSp: The number of spouses and/or siblings the passenger had on board.
• Parch: The number of parents and/or children the passenger had on board.
• Ticket: the number of their ticket (unique to each passenger)
• Fare: the cost of the ticket in US dollars.

To load the data, copy and paste the following into a code chunk and press play:

# Load the data 
Titanic <- read.csv("https://www.dropbox.com/scl/fi/26vhy0o0n4zykjvpjgolo/Titanic.csv?rlkey=b4a6ewvjrgvd25gr42tuayoyh&st=yppg7u63&dl=1")

# Some Data cleaning
Titanic$Sex <- as.factor(Titanic$Sex)
Titanic$Pclass <- as.factor(Titanic$Pclass)
Titanic <- na.omit(Titanic)
Titanic <- Titanic[,-c(11,12)]

Model 1: One Numeric X

We are approached by an historian, Historian 1. Historian 1 knows from their work that in the era when the Titanic sank, social conventions would have indicated that women were prioritized over men for space on the life boats. They therefore suggest you start your work in modeling \(Y\) = survival by building a model using \(X\) = whether the passenger was male or female. Historian 1 believes women were more likely to have survived than men.

To build the appropriate regression model in R, we use:

model1 <- glm(Survived ~ Sex, data = Titanic, family = "binomial")

To see the coefficients of the model and formatted them neatly, we use:

knitr::kable( summary( model1 )$coefficients )

summary(model1)$null.deviance

summary(model1)$deviance

Model 2: Adding Age

We have completed everything Historian 1 asked of us, and now we are approached by Historian 2. Historian 2 agrees with Historian 1 that social conventions in 1912 would have prioritized women over men for space in the life boats. However, Historian 2 believes the age of the passenger would also have mattered. They request that we add age into the model, too.

• Age: The age of the passenger in years. Note: Decimals are for children under a year in age.

Historian 2 is asking for a multiple logistic regression model, which just means more than one X variable is in the model. To add a new variable into the model, we just use +.

model2 <- glm(Survived ~ Sex + Age, data = Titanic, family = "binomial")

It turns out that the deviance for Model 2 is only a tiny bit smaller than the deviance of Model 1. This means that when we add age into the model, we only get a small improvement in the deviance. Is this enough improvement to suggest that adding age into the model improves the model fit?

When we compare logistic regression models with different numbers of coefficients, we use something called the AIC to help us answer this question:

\[AIC = Deviance(Model) + 2p\] where \(p\) is the number of \(\beta\) terms in the model, including the intercept.

The key here is the +2p. Because the AIC is based on the deviance, we prefer models with small AIC values. The penalty + 2 p means that in order for the AIC to decrease, adding a new \(\beta\) into the model must result in a decrease in the deviance of more than 2. In other words, in order for us to claim adding a new variable into the model improves model fit, the deviance must improve by more than 2.

To check, we can compute the AIC using the following, where we replace p with the number of \(\beta\) terms in the model:

summary(model1)$deviance + 2*p

Model 3: Using Child

Now Historian 3 enters the conversation. They argue that age is important, but specifically think that women and children would have been prioritized in the life boats. They ask us to build a model that uses (1) whether or not a passenger is male and (2) whether or not a passenger is a young child as explanatory variables. They define a young child as being under 10 years in age.

We build their requested Model 3 as follows:

model3 <- glm(Survived ~ Sex + (Age < 10), data = Titanic, family = "binomial")

Note that Age < 10 creates a categorical variable where 1 means the individual was a child less than 10 years in age and 0 means an individual was at least 10 years of age.

Model 4: All The Features

A fourth historian joins the discussion and says that since we have decided on how to handle age, we really should go ahead and put all the explanatory variables we can into the model!

model4 <- glm( Survived ~ Sex + (Age < 10) + ADD MORE HERE, data = Titanic, family = "binomial")

knitr::kable( summary(model4)$coefficients )