Checking Conditions

Survival is a random variable and it is binary, so the first condition is met. What about the second and third?

Visualizing the relationship

One other step we might consider before building a model is to visualize the relationship between X and Y. This is not strictly necessary, especially when we are building a model with only one X. However, when we start working with data with multiple X variables, it is very helpful to explore visually whether there seems to be a relationship between X and Y. This can help us decide which variables we want might want to include in the model.

With a categorical X and a categorical Y, we use a mosaic plot to visualize the relationship between the two variables. We talked about these in class, but here is the code:

We first need to load two libraries.

library(ggplot2)
library(ggmosaic)

Once you have the libraries loaded, there is one more step you need to take before you can create the mosaic plot. Our Y variable is recorded in our data set as 0s and 1s. This is fine, but mosaic plots require two categorical variables to work. Our brains know that survival is categorical, but the computer sees 0s and 1s and thinks that the variable is numeric. To check this, run this code:

class(Titanic$Survived)

This returns integer, which means the computer thinks the variable is a numeric variable. To convert to a categorical variable, we can use the code as.factor():

Survival <- as.factor(Titanic$Survived)

Now, we are ready to create the plot!

# Create the plot 
ggplot(data = Titanic) +
  geom_mosaic(aes(x = product(Survival, Sex), fill = Survival)) +   
  labs(x="Passenger Sex", y="Survival", title = "Figure 1",
  caption = "Mosaic plot of survival by sex") +
  theme_mosaic() +
  theme(legend.position = "none")

Creating an Empirical Log Odds Plot

One of the conditions we need when X is numeric is the shape condition. This means that we assume a certain shape of the relationship between X and the log odds of survival. To see what shape might be reasonable, we need to build an empirical log odds plot.

We are going to use a function in R to produce the empirical log odds plot. To do so, create an R chunk, copy the LONG function below, paste it into an R chunk in your Markdown, and run it. You will notice that nothing seems to happen, but if you check the top right panel of your RStudio session, you should notice that a new function called get_logOddsPlot has appeared. R allows individual users to create functions as needed, and in essence you have just “taught” R a new function.

get_logOddsPlot <- function(x, y, xname, yname, chosentitle, formulahere){
  sort = order(x)
  x = x[sort]
  y = y[sort]
  a = seq(from = 1, to = length(x), by=.05*length(y))
  a = round(a)
  b = c(a[-1] - 1, length(x))
  
  prob = xmean = rep(0, length(a))
  for (i in 1:length(a)){
    range = (a[i]):(b[i])
    prob[i] = mean(y[range])
    xmean[i] = mean(x[range])
  }
  
  prob[prob == 0] = .001
  prob[prob == 1] = .99

  g = log(prob/(1-prob))
  
  dataHere <- data.frame("x" = b, "LogOdds" = g)
  
  suppressMessages(library(ggplot2))
  
  ggplot(dataHere, aes(x =xmean, y = LogOdds)) +
    geom_point() + 
    geom_smooth(formula = formulahere, method = "lm", se = FALSE) + 
    labs(x = xname, y = yname, title = chosentitle)
}

To use the function, you will need to specify the inputs, meaning the pieces of information the computer needs in order to run the code.

• x: the column containing the x variable (Titanic$Fare)
• y: the column containing the y variable (Titanic$Survived)
• xname: The label you want on the x axis of the graph; ex: “Fare”
• yname: The label you want on the y axis of the graph; ex: “Log Odds of Survival”
• chosentitle: The title you want on the graph
• formula: The shape of the relationship you want to plot. If you want a line, this is y ~ x. If you want a 3rd order polynomial, you use y ~ poly(x,3)

Here is some of the code to create the plot. You will need to fill in the gaps.

get_logOddsPlot(x= , y=, xname = "Ticket Cost",
  yname = "Log Odds of Survival", chosentitle = " ",
  formulahere = y ~ x)

Non-linear shapes

If the empirical log odds plot indicates that the relationship is not linear, this does not mean we cannot use logistic regression. It just means we have to change the right hand side of our logistic regression model to reflect a shape other than a line.

There are two major non-linear shapes we will see in this course:

• 1. Natural Logs
• 2. Polynomials

We choose a natural log of X if the relationship between \(X\) and the log odds of survival does not change direction. This means that relationship is always increasing or always decreasing, but the steepness of that increase or decrease changes across the \(X\) axis.

We choose a polynomial for X if the relationship between \(X\) and the log odds of survival changes direction. This means that relationship may start off as increasing and then changes to decreasing at some point on the X axis, or vice versa. We need one power added to our polynomial for every time the direction changes. If it changes once, we need a 2nd order polynomial (a quadratic). If the direction changes twice, we need a 3rd order polynomial (a cubic).

In this case, the relationship does not change direction, so we want to consider using the following model:

\[Y_i \sim Bernoulli(\pi_i)\] \[log \left( \frac{{\pi_i}}{ 1- {\pi}_i} \right) = \beta_0 + \beta_1 log(Fare_i)\]

Now ideally, we want to be able to determine what is a better fit by using more than just our eyes. In other words, we want to have some kind of numeric quantity that allows us to compare models. We will get to that shortly!

Interpreting with Logs: Odds Form

With our new model, we have logs on both sides of the fitted model.

\[log \left( \frac{{\hat{\pi}_i}}{ 1- \hat{\pi}_i} \right) = \hat{\beta}_0 + \hat{\beta}_1 log(X_i)\]

When we have this, we actually get a really nice interpretation of \(\hat{\beta}_1\): If \(X_i\) increases by 1%, we predict the odds that \(Y_i = 1\) increases by \(\hat{\beta}_1\)%.

References