The Goal

We have been working on logistic regression models in class, and today we are going to practice working with them!

The Data

The RMS Titanic was a huge, luxury passenger liner designed and built in the early 20th century. Despite the fact that the ship was believed to be unsinkable, during her maiden voyage on April 15, 1912, the Titanic collided with an iceberg and sank.

The loss of life during the Titanic tragedy was enormous, but there were survivors. Was it random chance that these particular people survived? Or were there some specific characteristics of these people that led to their positions in the life boats? Today, we’re going to build a model to find out.

The data for today can be downloaded from Canvas. We have \(n = 891\) rows and 12 different variables. Some of the variables are categorical and some are numeric. The meaning of each variable is as follows:

• Passenger: A unique ID number for each passenger.
• Survived: An indicator for whether the passenger survived (1) or perished (0) during the disaster.
• Pclass: Indicator for the class of the ticket held by the passenger; 1 = 1st class, 2 = 2nd class, 3 = 3rd class.
• Name: The name of the passenger.
• Sex: Binary Indicator for the biological sex of the passenger.
• Age: Age of the passenger in years; Age is fractional if the passenger was less than 1 year old.
• SibSp: number of siblings/spouses the passenger had aboard the Titanic. Here, siblings are defined as brother, sister, stepbrother, and stepsister. Spouses are defined as husband and wife.
• Parch: number of parents/children the passenger had aboard the Titanic. Here, parent is defined as mother/father and child is defined as daughter,son, stepdaughter or stepson. NOTE: Some children traveled only with a nanny, therefore parch=0 for them. There were no parents aboard for these children.
• Ticket: The unique ticket number for each passenger.
• Fare: How much the ticket cost in US dollars.
• Cabin: The cabin number assigned to each passenger. Some cabins hold more than one passenger.
• Embarked: Port where the passenger boarded the ship; C = Cherbourg, Q = Queenstown, S = Southampton

Our goal is to build a model to help explore what characteristics were related to whether or not a passenger survived the disaster, so our \(Y\) variable is Survived. Because \(Y\) is a binary random variable, we will be building a logistic regression model.

As we discussed in class, there are three steps to building a logistic regression model.

• Step 1: Choose a reasonable distribution for \(Y_i\)
• Step 2: Build a model for the parameters of interest
• Step 3: Estimate the parameters of interest

We will work through each of these steps with these data.

EDA

Whenever we have a variable that we want to model, it is helpful to begin by visualizing the variable in question. This helps us to check for missing data and it also helps us see what values are possible for the variable. Because we are dealing with a binary random variable, we tend to use a bar plot or a table.

There are several ways to make tables in R, but we will discuss two. The first is very direct. We tell R we want to use the Titanic data set and the variable Survived, by using the code Titanic$Survived. Then, we use the table(whatWeWantToMakeATableWith) command to actually make the table.

table(Titanic$Survived)

However, this makes a table that is not particularly pretty or professional when you knit. A second option that does create professional tables is:

knitr::kable(table(Titanic$Survived), col.names=c("Survival", "Number of People") )

The code is more complex, but the heart of it is the same table. This table will not look very pretty when you press play, but go ahead and knit. See how nicely the table gets formatted?

Step 1: Distribution of the Response variable

Now that we have done our EDA, we are ready to start modeling!

The first step in building our logistic regression model is deciding on a distribution that is reasonable to use to model the response variable \(Y\). For us, this variable indicates whether or not an individual survived the disaster.

Doing things like writing down distributions often involve parameters. Parameters in statistics are generally represented as Greek letters. How can we write Greek letters and other mathematical notation in our Markdown file so they show up when we knit?

If you want to write mathematical notation, we need to tell Markdown, “Hey, we’re going to make a math symbol!” To do that, you use dollar signs. For instance, to make \(\hat{\beta}_1\), you simply put $\hat{\beta}_1$ into the white space (not a chunk) in your Markdown.

Go ahead and do that. See how the dollar signs change colors? Also note that if you hover your mouse over what you just pasted, the mathematical symbol we want will appear.

If you want the symbol to appear on its own line in your Markdown, you need to put two $ signs at the beginning and end of the line (so $$). Try that now.

The same thing works for other mathematical symbols. Let’s say I want to write out a regression line. The code is $\widehat{Y_i} = \hat{\beta}_0 + \hat{\beta}_1 X_i$.

We’ll notice that we used \hat for the \(\hat{\beta}_1\) but \widehat over \(Y_i\). Why? Because we usually replace \(Y_i\) with a longer word, so it needs a bigger (wider) hat.

A word of caution. You must make sure that both the dollar sign at the beginning and end of your mathematical expression is touching text. In other words, $\hat{y}$ will knit just fine but $\hat{y} $ will yield an error. This is important. Your document will not knit if you forget! Now, you can put spaces inside, meaning that $\hat{y} = 4$ is fine, but the beginning and end can have NO spaces.

Step 2: Construct a Model for the Parameters

Okay, we have chosen a distribution for \(Y_i\). Now, let’s move on Step 2: Constructing a model for the parameters.

When we were computing the likelihoods in the previous section, we chose a single value of \(\pi_i\) to represent the probability of survival for everyone in the data set. This means that we assume that every single person on the ship has the same chances of surviving the disaster.

Instead of assuming every person has the same probability of success, let’s choose a model for \(\pi_i\) that allows the probability to be different for different people, depending on some characteristic of that person. There are a lot of possible explanatory variables we could use, but let’s start with \(X_i = Fare_i\), the cost of the ticket for each passenger.

If we assume that the probability differs depending on fare, our next step is to see how these two things (probability of survival and fare) might be connected so we can choose a reasonable model.

get_logOddsPlot <- function(x, y, xname, formulahere){
  sort = order(x)
  x = x[sort]
  y = y[sort]
  a = seq(1, length(x), by=.05*length(y))
  b = c(a[-1] - 1, length(x))
  
  prob = xmean = ns = rep(0, length(a)) # ns is for CIs
  for (i in 1:length(a)){
    range = (a[i]):(b[i])
    prob[i] = mean(y[range])
    xmean[i] = mean(x[range])
    ns[i] = b[i] - a[i] + 1 # for CI 
  }
  
  extreme = (prob == 1 | prob == 0)
  #prob[prob == 0] = min(prob[!extreme])
  #prob[prob == 1] = max(prob[!extreme])
  prob[prob == 0] = .01
  prob[prob == 1] = .99
  
  g = log(prob/(1-prob))
  
  dataHere <- data.frame("x" = b, "LogOdds" = g)
  
  suppressMessages(library(ggplot2))
  
  ggplot(dataHere, aes(x =xmean, y = LogOdds)) + geom_point() + geom_smooth(formula = formulahere, method = "lm", se = FALSE) + labs(x = xname, y = "Log Odds")
}

get_logOddsPlot(x= , y=, xname = "Fare", formulahere = y ~ x)`

Step 3: Estimate the Parameters of Interest

Our next step is to estimate the parameters in involved in the chosen model. The parameters we need to estimate are the \(\beta\) terms.

To estimate these parameters, we will be using R. This is done using very similar code to LSLR, except we use the glm function instead of lm. Additionally, we have one extra argument (or input) for our code. We must indicate that we wish to fit a logistic model by including the argument family=binomial.

To examine the coefficient estimates from our logistic regression model, we use the summary command.

Wrapping Up

We have now seen how to build a logistic regression model using the three-step process of building a parametric model. We have also practiced using the estimates of the parameters we have obtained using R for interpretation and prediction.

Our next steps are to remind ourselves of how to perform inference using hypothesis tests and confidence intervals for these parameters. We also want to explore how R is actually computing the estimates of the parameters. That is where we are headed!