Machine Learning from Disaster: Titanic

This is my short summary of my investigation of the Titanic data set. It consists of three parts:

• Data loading and preprocessing
• Feature engineering
• Machine learning with Decision Tree

1 Manipulating Data

First, lets load data

train <- read.csv("files/train.csv", header = TRUE,stringsAsFactors = FALSE)
test <- read.csv("files/test.csv", header = TRUE,stringsAsFactors = FALSE)

We combine the two data sets, train and test, into one data set data_full. dplyr library provides handy command bind_rows:

data_full <- bind_rows(train,test)

Let’s check our data

str(data_full)

'data.frame':   1309 obs. of  12 variables:
 $ PassengerId: int  1 2 3 4 5 6 7 8 9 10 ...
 $ Survived   : int  0 1 1 1 0 0 0 0 1 1 ...
 $ Pclass     : int  3 1 3 1 3 3 1 3 3 2 ...
 $ Name       : chr  "Braund, Mr. Owen Harris" "Cumings, Mrs. John Bradley (Florence Briggs Thayer)" "Heikkinen, Miss. Laina" "Futrelle, Mrs. Jacques Heath (Lily May Peel)" ...
 $ Sex        : chr  "male" "female" "female" "female" ...
 $ Age        : num  22 38 26 35 35 NA 54 2 27 14 ...
 $ SibSp      : int  1 1 0 1 0 0 0 3 0 1 ...
 $ Parch      : int  0 0 0 0 0 0 0 1 2 0 ...
 $ Ticket     : chr  "A/5 21171" "PC 17599" "STON/O2. 3101282" "113803" ...
 $ Fare       : num  7.25 71.28 7.92 53.1 8.05 ...
 $ Cabin      : chr  "" "C85" "" "C123" ...
 $ Embarked   : chr  "S" "C" "S" "S" ...

survived and pclass features have 2 and 3 distinct values respectively. They better be converted into factors

data_full$Survived <- as.factor(data_full$Survived)
data_full$Pclass <- as.factor(data_full$Pclass)

pclass is an important feature. Rich passengers traveled in higher pclass survived as higher rate as can be seen from the following plot

p<-ggplot(data_full[1:891,], aes(x = Pclass, fill = Survived)) +
  geom_bar(stat = 'count',position='dodge') +
  xlab("Pclass") +
  ylab("Count") +
  labs(fill = "Survived")+theme_light()
ggplotly(p)

2 Feature engineering

One powerful desired feature is title. It can be extracted from existing feature name. Each name in the full data set has a title in it. For example, the first passenger Owen Haris was titled as ‘Mr.’:

data_full$Name[1]

[1] "Braund, Mr. Owen Harris"

Braund, Mr. Owen Harris

So, let’s extract titles from name using regular expressions:

data_full$Title <- gsub('(.*, )|(\\..*)','',data_full$Name)
table(data_full$Sex,data_full$Title)

        
         Capt Col Don Dona  Dr Jonkheer Lady Major Master Miss Mlle Mme
  female    0   0   0    1   1        0    1     0      0  260    2   1
  male      1   4   1    0   7        1    0     2     61    0    0   0
        
          Mr Mrs  Ms Rev Sir the Countess
  female   0 197   2   0   0            1
  male   757   0   0   8   1            0

Too many titles. Let’s group them

data_full[data_full$Title %in% c("Dona", "the Countess"),'Title'] <- "Lady"
data_full[data_full$Title %in% c("Ms", "Mlle"),'Title'] <- "Miss"
data_full[data_full$Title == "Mme",'Title'] <- "Mrs"
data_full[data_full$Title %in% c("Jonkheer", "Don"),'Title'] <- "Sir"
data_full[data_full$Title %in% c("Col", "Capt", "Major"),'Title'] <- "Officer"

data_full[data_full$Title == "Lady",'Title'] <- "Mrs"
data_full[data_full$Title %in% c("Rev", "Sir", "Officer"),'Title'] <- "Mr"

The title Dr is a bit tricky. There is one female with this title. Let’s fix this:

table(data_full$Sex,data_full$Title)

        
          Dr Master Miss  Mr Mrs
  female   1      0  264   0 201
  male     7     61    0 775   0

data_full[data_full$Sex=='female' & data_full$Title=='Dr','title']<-'Mrs'
data_full[data_full$Title=='Dr','Title'] <- 'Mr'
table(data_full$Sex,data_full$Title)

        
         Master Miss  Mr Mrs
  female      0  264   1 201
  male       61    0 782   0

data_full$Title <- as.factor(data_full$Title)

p<-ggplot(data_full[1:891,], aes(x = Title, fill = Survived)) +
  geom_bar(stat='count') + facet_grid(.~Pclass)+
  xlab("Title") +
  ylab("Count") +
  labs(fill = "Survived")+theme_light()
ggplotly(p)

mosaicplot(table(data_full$Title,data_full$Survived),main='title by survival')

mosaicplot(table(data_full$Pclass,data_full$Survived),main='pclass by survival')

Main conclusions so far:

• people with the title ‘Mr’ had high probability to perish.
• people from the 3d class had high probability to perish.

One more feature which appeared to be useful is family size. It can be extracted by adding sibsp + parch + 1. We further bin the family size into 3 categories:

data_full$familysize <- data_full$SibSp + data_full$Parch + 1

data_full$fsize[data_full$familysize==1] <- 'single'
data_full$fsize[data_full$familysize <= 4 & data_full$familysize > 1] <- 'small'
data_full$fsize[data_full$familysize >= 5] <- 'large'
data_full$fsize <- as.factor(data_full$fsize)
mosaicplot(table(data_full$fsize,data_full$Survived),main='famility size by survival')

There is a survival penalty for traveling with a large family.

3 Decision Tree

Random Forests are more powerful than single trees, but single trees have the advantage of being easier to understand.

X <- data_full[1:891,c('Pclass','fsize','Title')]
y <- as.factor(data_full[1:891,'Survived'])

folds <- createMultiFolds(y, k = 3, times = 10)
ctrl  <- trainControl(method = "repeatedcv", number = 3, repeats = 10,
                       index = folds)

rpart.cv <- train(x = X, y = y, method = "rpart", tuneLength = 10, 
                    trControl = ctrl)


prp(rpart.cv$finalModel,type = 0, extra = 1, under = TRUE) # display the results