10 Classifier Showdown w/ Caret
Ed Mwanza
December 1, 2016
In this blog post, the performance of ten classification models are explored on the titanic dataset. The primary objetive of this analysis is to predict passenger Survival aboard the unsinkable Titanic ship.In order to achieve a fair model performance evaluation, I will use the Caret package from R which provides a convenient way to create a unified platform or interface to fit and anlyze different models.Below is the roadmap to achieving the intended objective.
Workflow:
- Load data & required packages
- Data exploration & cleansing
- Missing data analysis & correction
- Feature engineering & visualizations
- Feature importance analysis
- Model building (wide cardinality)
- Model performance evaluation & final candidate selection
- Prediction & summary
Load data & required packages
library(readr)
library(data.table)
library(ggplot2)
library(ggthemes)
library(scales)
library(Amelia)
library(tabplot)
library(dplyr)
library(mice)
library(randomForest)
library(VIM)
library(stringr)
library(caret)Data exploration & Cleansing
I will begin by merging the train & test sets into a single set to massage the data all at once (no data leaks)!Why not kill 2 birds with a stone if you can? Then I will perform some brief but necessary data checks such as summary() function.
my_df <- bind_rows(train, test)#format and wrangle data together at once
str(my_df)## Classes 'tbl_df', 'tbl' and 'data.frame': 1309 obs. of 12 variables:
## $ PassengerId: int 1 2 3 4 5 6 7 8 9 10 ...
## $ Survived : int 0 1 1 1 0 0 0 0 1 1 ...
## $ Pclass : int 3 1 3 1 3 3 1 3 3 2 ...
## $ Name : chr "Braund, Mr. Owen Harris" "Cumings, Mrs. John Bradley (Florence Briggs Thayer)" "Heikkinen, Miss. Laina" "Futrelle, Mrs. Jacques Heath (Lily May Peel)" ...
## $ Sex : chr "male" "female" "female" "female" ...
## $ Age : num 22 38 26 35 35 NA 54 2 27 14 ...
## $ SibSp : int 1 1 0 1 0 0 0 3 0 1 ...
## $ Parch : int 0 0 0 0 0 0 0 1 2 0 ...
## $ Ticket : chr "A/5 21171" "PC 17599" "STON/O2. 3101282" "113803" ...
## $ Fare : num 7.25 71.28 7.92 53.1 8.05 ...
## $ Cabin : chr NA "C85" NA "C123" ...
## $ Embarked : chr "S" "C" "S" "S" ...summary(my_df)## PassengerId Survived Pclass Name
## Min. : 1 Min. :0.0000 Min. :1.000 Length:1309
## 1st Qu.: 328 1st Qu.:0.0000 1st Qu.:2.000 Class :character
## Median : 655 Median :0.0000 Median :3.000 Mode :character
## Mean : 655 Mean :0.3838 Mean :2.295
## 3rd Qu.: 982 3rd Qu.:1.0000 3rd Qu.:3.000
## Max. :1309 Max. :1.0000 Max. :3.000
## NA's :418
## Sex Age SibSp Parch
## Length:1309 Min. : 0.17 Min. :0.0000 Min. :0.000
## Class :character 1st Qu.:21.00 1st Qu.:0.0000 1st Qu.:0.000
## Mode :character Median :28.00 Median :0.0000 Median :0.000
## Mean :29.88 Mean :0.4989 Mean :0.385
## 3rd Qu.:39.00 3rd Qu.:1.0000 3rd Qu.:0.000
## Max. :80.00 Max. :8.0000 Max. :9.000
## NA's :263
## Ticket Fare Cabin
## Length:1309 Min. : 0.000 Length:1309
## Class :character 1st Qu.: 7.896 Class :character
## Mode :character Median : 14.454 Mode :character
## Mean : 33.295
## 3rd Qu.: 31.275
## Max. :512.329
## NA's :1
## Embarked
## Length:1309
## Class :character
## Mode :character
##
##
##
## glimpse(my_df)## Observations: 1,309
## Variables: 12
## $ PassengerId (int) 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,...
## $ Survived (int) 0, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 0,...
## $ Pclass (int) 3, 1, 3, 1, 3, 3, 1, 3, 3, 2, 3, 1, 3, 3, 3, 2, 3,...
## $ Name (chr) "Braund, Mr. Owen Harris", "Cumings, Mrs. John Bra...
## $ Sex (chr) "male", "female", "female", "female", "male", "mal...
## $ Age (dbl) 22, 38, 26, 35, 35, NA, 54, 2, 27, 14, 4, 58, 20, ...
## $ SibSp (int) 1, 1, 0, 1, 0, 0, 0, 3, 0, 1, 1, 0, 0, 1, 0, 0, 4,...
## $ Parch (int) 0, 0, 0, 0, 0, 0, 0, 1, 2, 0, 1, 0, 0, 5, 0, 0, 1,...
## $ Ticket (chr) "A/5 21171", "PC 17599", "STON/O2. 3101282", "1138...
## $ Fare (dbl) 7.2500, 71.2833, 7.9250, 53.1000, 8.0500, 8.4583, ...
## $ Cabin (chr) NA, "C85", NA, "C123", NA, NA, "E46", NA, NA, NA, ...
## $ Embarked (chr) "S", "C", "S", "S", "S", "Q", "S", "S", "S", "C", ...head(my_df, n = 10)## Source: local data frame [10 x 12]
##
## PassengerId Survived Pclass
## (int) (int) (int)
## 1 1 0 3
## 2 2 1 1
## 3 3 1 3
## 4 4 1 1
## 5 5 0 3
## 6 6 0 3
## 7 7 0 1
## 8 8 0 3
## 9 9 1 3
## 10 10 1 2
## Variables not shown: Name (chr), Sex (chr), Age (dbl), SibSp (int), Parch
## (int), Ticket (chr), Fare (dbl), Cabin (chr), Embarked (chr)Some type-checking and coercion
sapply(my_df, class)## PassengerId Survived Pclass Name Sex Age
## "integer" "integer" "integer" "character" "character" "numeric"
## SibSp Parch Ticket Fare Cabin Embarked
## "integer" "integer" "character" "numeric" "character" "character"#Convert "sex"from char->factor
my_df$Sex <- as.factor(my_df$Sex)
str(my_df)## Classes 'tbl_df', 'tbl' and 'data.frame': 1309 obs. of 12 variables:
## $ PassengerId: int 1 2 3 4 5 6 7 8 9 10 ...
## $ Survived : int 0 1 1 1 0 0 0 0 1 1 ...
## $ Pclass : int 3 1 3 1 3 3 1 3 3 2 ...
## $ Name : chr "Braund, Mr. Owen Harris" "Cumings, Mrs. John Bradley (Florence Briggs Thayer)" "Heikkinen, Miss. Laina" "Futrelle, Mrs. Jacques Heath (Lily May Peel)" ...
## $ Sex : Factor w/ 2 levels "female","male": 2 1 1 1 2 2 2 2 1 1 ...
## $ Age : num 22 38 26 35 35 NA 54 2 27 14 ...
## $ SibSp : int 1 1 0 1 0 0 0 3 0 1 ...
## $ Parch : int 0 0 0 0 0 0 0 1 2 0 ...
## $ Ticket : chr "A/5 21171" "PC 17599" "STON/O2. 3101282" "113803" ...
## $ Fare : num 7.25 71.28 7.92 53.1 8.05 ...
## $ Cabin : chr NA "C85" NA "C123" ...
## $ Embarked : chr "S" "C" "S" "S" ...This is not a massive data set, its easy to explore its integrity with a few lines of code. One of the must do’s is missing data analysis and correction. This is mainly because missing data affects model performance more specifically regression models. But whether regression or not, it’s recommended to analyze missing data and determine course of action to correct missingness. Several techniques for imputing missing data exist, however, in this post I will use a single predictive method from the mice package and a manual method inspired by some other bloggers.
Missing data & Correction
#If a variable is "numeric" check for numerical special values otherwise just for NAs
is_special <- function(x){
if(is.numeric(x)) !is.finite(x) else is.na(x)
}
sapply(my_df, is_special)Output too long hence not shown for brevity.
Missing_d <- function(x){sum(is.na(x))/length(x)*100} #USD to calculate % of missing data
apply(my_df, 2, Missing_d) #checking missing data variable-wise## PassengerId Survived Pclass Name Sex Age
## 0.00000000 31.93277311 0.00000000 0.00000000 0.00000000 20.09167303
## SibSp Parch Ticket Fare Cabin Embarked
## 0.00000000 0.00000000 0.00000000 0.07639419 77.46371276 0.15278839md.pattern(my_df)## PassengerId Pclass Sex SibSp Parch Fare Age Ticket Survived Name Cabin
## 521 1 1 1 1 1 1 1 1 1 0 0
## 231 1 1 1 1 1 1 1 1 0 0 0
## 140 1 1 1 1 1 1 0 1 1 0 0
## 193 1 1 1 1 1 1 1 0 1 0 0
## 64 1 1 1 1 1 1 0 1 0 0 0
## 100 1 1 1 1 1 1 1 0 0 0 0
## 37 1 1 1 1 1 1 0 0 1 0 0
## 1 1 1 1 1 1 0 1 1 0 0 0
## 22 1 1 1 1 1 1 0 0 0 0 0
## 0 0 0 0 0 1 263 352 418 1309 1309
## Embarked
## 521 0 3
## 231 0 4
## 140 0 4
## 193 0 4
## 64 0 5
## 100 0 5
## 37 0 5
## 1 0 5
## 22 0 6
## 1309 4961As seen Age has about 20% missing data with Cabin being the worst having ~77%. Embarked & Fare have rather minimal missing data also notice that the response variable appears to have some missing values, that’s just a reflection of the absence of labels in the test set which I merged to train! I will remove Cabin since imputing it would introduce a lot of bias into the data but Age missing values will be imputed & pre- and post-imputation distributions visually analyzed for consistent distributions. But before I do that, its good practice to get a visual intuition of missing data.
missmap(my_df, main = "Observed Vs Missing Data")
aggr_plot <- aggr(my_df, col=c('navyblue','red'), numbers=TRUE, sortVars=TRUE,
labels=names(data), cex.axis=.7, gap=3, ylab=c("Histogram of missing data","Pattern"))
##
## Variables sorted by number of missings:
## Variable Count
## Cabin 0.7746371276
## Survived 0.3193277311
## Age 0.2009167303
## Embarked 0.0015278839
## Fare 0.0007639419
## PassengerId 0.0000000000
## Pclass 0.0000000000
## Name 0.0000000000
## Sex 0.0000000000
## SibSp 0.0000000000
## Parch 0.0000000000
## Ticket 0.0000000000It is evident that this is a case of Missing data at random as evidenced in the visualizations above. As stated earlier, it probably wouldn’t assist much imputing Cabin’s missing data, additionally, the feature being categorical poses its own issues with respect to imputation. Age on the other end, seems to be rather useful & the fact that its a continuous variable with enough presence of data points, imputation would probably provide lift to model performance. To start the missing data correction phase, Fare and Embarked are dealt with manually and then Age later using a predictive technique.
1 Embarked
#1. Embarked
which(is.na(my_df$Embarked))## [1] 62 830#observations 62 and 830 are missing embarkment. Where did these voyagers come from?
#Find out how much they paid (fare)
embark_miss <- subset(my_df, PassengerId == 62 | PassengerId== 830)
#They both paid $80, belonged to class 1 Lets impute based on similar passengers
embark_fare <- my_df %>%
filter(PassengerId != 62 & PassengerId != 830)
#As inspired by some Kagglers
ggplot(data = embark_fare, aes(x = Embarked, y= Fare, fill = factor(Pclass))) +
geom_boxplot()+
geom_hline(aes(yintercept = 80),
colour = 'blue', linetype = 'dashed', lwd = 2)+
scale_y_continuous(labels = dollar_format())+
theme_few()
#As observed, $80-dollar dash line coincides with the median for class 1. We can infer that the
#obervations 62 and 830 embarked from "C"
#CORRECTION:replace NA with "C"
my_df$Embarked[c(62, 830)] <- "C"
sum(is.na(my_df$Embarked))## [1] 02 Fare
#2. FARE
which(is.na(my_df$Fare))## [1] 1044#voyager 1044 has missing fare
fare_miss <- subset(my_df, PassengerId == 1044)
#This passenger embarked the unsinkable at port "S" and was Class "3", use same approach as embark
fare_fix <- my_df%>%
filter(PassengerId != 1044)
#whats the median fare for class "3" passengers who emabarked at "S"?
fare_fix%>%
filter(Pclass == 3 & Embarked == 'S')%>%
summarise(avg_fare = round(median(Fare),digits = 2))## Source: local data frame [1 x 1]
##
## avg_fare
## (dbl)
## 1 8.05#median amount paid is ~ 8.05::Its safe to CORRECT the NA value with the median value
my_df$Fare[1044] <- 8.05
sum(is.na(my_df$Fare))## [1] 03 Predictive Imputation-Age
There are several methods of imputing missing data, best practice is to assess performance lift as a result of each method implemeted on data. In this post however, I will use a single method from the mice package leveraging the pmm (predictive mean matching) method.
mice_corr <- mice(my_df[c("Sex","Age")],m=5,maxit=50,meth='pmm',seed=500)##
## iter imp variable
## 1 1 Age
## 1 2 Age
## 1 3 Age
## 1 4 Age
## 1 5 Age
## 2 1 Age
## 2 2 Age
## 2 3 Age
## 2 4 Age
## 2 5 Age
## 3 1 Age
## 3 2 Age
## 3 3 Age
## 3 4 Age
## 3 5 Age
## 4 1 Age
## 4 2 Age
## 4 3 Age
## 4 4 Age
## 4 5 Age
## 5 1 Age
## 5 2 Age
## 5 3 Age
## 5 4 Age
## 5 5 Age
## 6 1 Age
## 6 2 Age
## 6 3 Age
## 6 4 Age
## 6 5 Age
## 7 1 Age
## 7 2 Age
## 7 3 Age
## 7 4 Age
## 7 5 Age
## 8 1 Age
## 8 2 Age
## 8 3 Age
## 8 4 Age
## 8 5 Age
## 9 1 Age
## 9 2 Age
## 9 3 Age
## 9 4 Age
## 9 5 Age
## 10 1 Age
## 10 2 Age
## 10 3 Age
## 10 4 Age
## 10 5 Age
## 11 1 Age
## 11 2 Age
## 11 3 Age
## 11 4 Age
## 11 5 Age
## 12 1 Age
## 12 2 Age
## 12 3 Age
## 12 4 Age
## 12 5 Age
## 13 1 Age
## 13 2 Age
## 13 3 Age
## 13 4 Age
## 13 5 Age
## 14 1 Age
## 14 2 Age
## 14 3 Age
## 14 4 Age
## 14 5 Age
## 15 1 Age
## 15 2 Age
## 15 3 Age
## 15 4 Age
## 15 5 Age
## 16 1 Age
## 16 2 Age
## 16 3 Age
## 16 4 Age
## 16 5 Age
## 17 1 Age
## 17 2 Age
## 17 3 Age
## 17 4 Age
## 17 5 Age
## 18 1 Age
## 18 2 Age
## 18 3 Age
## 18 4 Age
## 18 5 Age
## 19 1 Age
## 19 2 Age
## 19 3 Age
## 19 4 Age
## 19 5 Age
## 20 1 Age
## 20 2 Age
## 20 3 Age
## 20 4 Age
## 20 5 Age
## 21 1 Age
## 21 2 Age
## 21 3 Age
## 21 4 Age
## 21 5 Age
## 22 1 Age
## 22 2 Age
## 22 3 Age
## 22 4 Age
## 22 5 Age
## 23 1 Age
## 23 2 Age
## 23 3 Age
## 23 4 Age
## 23 5 Age
## 24 1 Age
## 24 2 Age
## 24 3 Age
## 24 4 Age
## 24 5 Age
## 25 1 Age
## 25 2 Age
## 25 3 Age
## 25 4 Age
## 25 5 Age
## 26 1 Age
## 26 2 Age
## 26 3 Age
## 26 4 Age
## 26 5 Age
## 27 1 Age
## 27 2 Age
## 27 3 Age
## 27 4 Age
## 27 5 Age
## 28 1 Age
## 28 2 Age
## 28 3 Age
## 28 4 Age
## 28 5 Age
## 29 1 Age
## 29 2 Age
## 29 3 Age
## 29 4 Age
## 29 5 Age
## 30 1 Age
## 30 2 Age
## 30 3 Age
## 30 4 Age
## 30 5 Age
## 31 1 Age
## 31 2 Age
## 31 3 Age
## 31 4 Age
## 31 5 Age
## 32 1 Age
## 32 2 Age
## 32 3 Age
## 32 4 Age
## 32 5 Age
## 33 1 Age
## 33 2 Age
## 33 3 Age
## 33 4 Age
## 33 5 Age
## 34 1 Age
## 34 2 Age
## 34 3 Age
## 34 4 Age
## 34 5 Age
## 35 1 Age
## 35 2 Age
## 35 3 Age
## 35 4 Age
## 35 5 Age
## 36 1 Age
## 36 2 Age
## 36 3 Age
## 36 4 Age
## 36 5 Age
## 37 1 Age
## 37 2 Age
## 37 3 Age
## 37 4 Age
## 37 5 Age
## 38 1 Age
## 38 2 Age
## 38 3 Age
## 38 4 Age
## 38 5 Age
## 39 1 Age
## 39 2 Age
## 39 3 Age
## 39 4 Age
## 39 5 Age
## 40 1 Age
## 40 2 Age
## 40 3 Age
## 40 4 Age
## 40 5 Age
## 41 1 Age
## 41 2 Age
## 41 3 Age
## 41 4 Age
## 41 5 Age
## 42 1 Age
## 42 2 Age
## 42 3 Age
## 42 4 Age
## 42 5 Age
## 43 1 Age
## 43 2 Age
## 43 3 Age
## 43 4 Age
## 43 5 Age
## 44 1 Age
## 44 2 Age
## 44 3 Age
## 44 4 Age
## 44 5 Age
## 45 1 Age
## 45 2 Age
## 45 3 Age
## 45 4 Age
## 45 5 Age
## 46 1 Age
## 46 2 Age
## 46 3 Age
## 46 4 Age
## 46 5 Age
## 47 1 Age
## 47 2 Age
## 47 3 Age
## 47 4 Age
## 47 5 Age
## 48 1 Age
## 48 2 Age
## 48 3 Age
## 48 4 Age
## 48 5 Age
## 49 1 Age
## 49 2 Age
## 49 3 Age
## 49 4 Age
## 49 5 Age
## 50 1 Age
## 50 2 Age
## 50 3 Age
## 50 4 Age
## 50 5 Agesummary(mice_corr)## Multiply imputed data set
## Call:
## mice(data = my_df[c("Sex", "Age")], m = 5, method = "pmm", maxit = 50,
## seed = 500)
## Number of multiple imputations: 5
## Missing cells per column:
## Sex Age
## 0 263
## Imputation methods:
## Sex Age
## "pmm" "pmm"
## VisitSequence:
## Age
## 2
## PredictorMatrix:
## Sex Age
## Sex 0 0
## Age 1 0
## Random generator seed value: 500mice_corr$imp$Age #quick check of imputed data## 1 2 3 4 5
## 6 23.0 16.00 23.00 21.00 3.00
## 18 31.0 0.83 21.00 22.00 61.00
## 20 21.0 55.00 33.00 15.00 35.00
## 27 62.0 51.00 28.00 0.42 22.00
## 29 36.0 29.00 30.50 54.00 38.00
## 30 31.0 19.00 20.00 22.00 25.00
## 32 28.0 21.00 2.00 18.50 24.00
## 33 26.0 22.00 24.00 22.00 14.00
## 37 38.0 49.00 30.00 50.00 27.00
## 43 42.0 23.00 22.00 23.00 19.00
## 46 21.0 34.00 24.00 30.50 36.50
## 47 34.0 18.00 51.00 21.00 32.00
## 48 39.0 22.00 33.00 39.00 17.00
## 49 18.0 28.00 42.00 28.00 21.00
## 56 26.0 34.50 46.00 32.00 28.00
## 65 32.0 56.00 21.00 35.00 23.00
## 66 32.0 17.00 28.00 24.00 34.00
## 77 34.0 18.00 23.00 28.50 29.00
## 78 36.0 27.00 26.00 32.00 35.00
## 83 23.0 24.00 9.00 31.00 2.00
## 88 8.0 32.00 25.00 46.00 2.00
## 96 16.0 31.00 34.50 47.00 25.00
## 102 22.0 32.00 18.50 24.00 3.00
## 108 39.0 55.00 23.00 52.00 31.00
## 110 27.0 47.00 20.00 50.00 19.00
## 122 61.0 6.00 22.00 29.00 21.00
## 127 22.0 29.00 23.00 30.00 13.00
## 129 18.0 32.00 44.00 25.00 21.00
## 141 45.0 35.00 24.00 6.00 15.00
## 155 40.0 26.00 30.00 19.00 23.00
## 159 25.0 30.00 32.00 28.00 47.00
## 160 30.0 26.00 42.00 30.50 42.00
## 167 16.0 58.00 39.00 33.00 25.00
## 169 33.0 27.00 41.00 27.00 0.42
## 177 27.0 44.00 33.00 23.00 25.00
## 181 24.0 11.00 30.00 9.00 18.00
## 182 25.0 40.00 36.00 25.00 34.00
## 186 25.0 32.00 64.00 0.92 36.00
## 187 8.0 1.00 31.00 18.00 38.00
## 197 33.0 49.00 29.00 16.00 31.00
## 199 30.0 45.00 30.00 19.00 24.00
## 202 28.0 40.00 39.00 32.00 20.00
## 215 41.0 2.00 21.00 26.00 44.00
## 224 31.0 50.00 9.00 40.00 55.00
## 230 16.0 39.00 54.00 23.00 47.00
## 236 13.0 18.00 23.00 45.00 24.00
## 241 24.0 15.00 37.00 24.00 28.00
## 242 54.0 26.00 39.00 45.00 29.00
## 251 20.0 52.00 17.00 27.00 54.00
## 257 19.0 17.00 45.00 42.00 26.00
## 261 20.0 14.00 34.00 42.00 0.83
## 265 47.0 36.00 21.00 41.00 18.00
## 271 35.0 21.00 61.00 19.00 39.00
## 275 9.0 56.00 5.00 36.00 31.00
## 278 23.0 32.00 36.50 44.00 2.00
## 285 19.0 40.00 55.00 59.00 22.00
## 296 30.0 9.00 39.00 0.83 25.00
## 299 24.0 48.00 36.00 26.00 29.00
## 301 25.0 28.00 24.00 31.00 29.00
## 302 4.0 21.00 19.00 51.00 24.00
## 304 17.0 17.00 24.00 47.00 4.00
## 305 20.0 31.00 28.00 13.00 65.00
## 307 40.0 35.00 55.00 2.00 40.00
## 325 20.0 27.00 60.00 27.00 56.00
## 331 26.0 45.00 36.00 50.00 36.00
## 335 4.0 7.00 31.00 20.00 36.00
## 336 21.0 65.00 23.00 27.00 48.00
## 348 17.0 47.00 48.00 39.00 20.00
## 352 45.5 51.00 40.50 26.00 36.50
## 355 35.0 44.00 50.00 30.00 21.00
## 359 49.0 21.00 24.00 41.00 76.00
## 360 22.0 63.00 28.00 50.00 1.00
## 365 6.0 34.00 21.00 28.00 44.00
## 368 24.0 17.00 47.00 53.00 47.00
## 369 20.0 26.00 9.00 29.00 28.00
## 376 20.0 47.00 51.00 32.00 18.00
## 385 42.0 35.00 34.00 59.00 24.50
## 389 34.0 23.00 16.00 16.00 29.00
## 410 60.0 22.00 22.00 29.00 15.00
## 411 27.0 25.00 37.00 21.00 28.00
## 412 42.0 23.00 64.00 50.00 28.00
## 414 20.5 26.00 17.00 24.00 22.00
## 416 27.0 0.75 54.00 39.00 53.00
## 421 20.0 36.00 10.00 18.00 45.00
## 426 50.0 22.00 54.00 28.00 31.00
## 429 26.0 49.00 29.00 38.00 46.00
## 432 24.0 0.92 18.00 2.00 22.00
## 445 18.0 29.00 17.00 28.00 25.00
## 452 31.0 32.00 19.00 24.00 25.00
## 455 27.0 26.00 20.00 39.00 21.00
## 458 18.0 39.00 30.00 24.00 40.00
## 460 27.0 16.00 32.00 28.50 20.00
## 465 23.0 24.00 19.00 46.00 22.00
## 467 24.0 41.00 34.00 35.00 49.00
## 469 35.0 43.00 26.00 48.00 33.00
## 471 17.0 33.00 20.00 28.00 6.00
## 476 35.0 19.00 38.00 48.00 53.00
## 482 34.0 28.00 53.00 21.00 25.00
## 486 56.0 24.00 15.00 13.00 28.00
## 491 28.0 28.00 3.00 17.00 50.00
## 496 33.0 18.00 24.00 24.00 25.00
## 498 27.0 27.00 29.00 30.00 19.00
## 503 17.0 2.00 18.00 47.00 2.00
## 508 1.0 22.00 21.00 2.00 6.00
## 512 34.0 32.00 8.00 3.00 70.50
## 518 49.0 32.00 30.00 32.00 18.00
## 523 30.0 34.00 40.00 36.00 22.00
## 525 21.0 67.00 21.00 47.00 50.00
## 528 28.0 46.00 0.75 47.00 46.00
## 532 31.0 45.00 39.00 36.00 29.00
## 534 18.0 17.00 37.00 31.00 18.50
## 539 25.0 29.00 21.00 26.00 21.00
## 548 38.5 31.00 26.00 26.00 23.00
## 553 14.0 31.00 47.00 41.00 2.00
## 558 35.0 32.00 20.00 28.00 30.00
## 561 32.0 48.00 18.00 16.00 32.00
## 564 16.0 47.00 33.00 38.00 30.00
## 565 24.0 42.00 17.00 60.00 18.00
## 569 14.0 54.00 20.00 22.00 23.50
## 574 26.0 28.00 24.00 38.00 2.00
## 579 36.0 31.00 30.00 22.00 35.00
## 585 14.5 42.00 40.00 0.83 28.00
## 590 36.0 24.00 18.00 36.00 58.00
## 594 33.0 59.00 17.00 38.00 39.00
## 597 22.0 36.00 21.00 27.00 3.00
## 599 40.5 19.00 21.00 19.00 20.00
## 602 17.0 25.00 38.50 26.00 50.00
## 603 39.0 17.00 22.00 14.00 55.00
## 612 42.0 29.00 47.00 38.00 19.00
## 613 58.0 22.00 32.00 22.00 20.00
## 614 29.0 24.00 39.00 28.50 20.00
## 630 32.0 23.00 37.00 52.00 45.00
## 634 62.0 0.83 28.00 32.50 27.00
## 640 40.0 48.00 49.00 33.00 6.00
## 644 39.0 23.00 42.00 19.00 48.00
## 649 25.0 43.00 49.00 40.00 19.00
## 651 35.0 33.00 23.00 26.00 20.50
## 654 48.0 30.00 40.00 29.00 33.00
## 657 1.0 32.00 28.50 21.00 21.00
## 668 36.0 25.00 31.00 18.50 29.00
## 670 40.0 47.00 22.00 21.00 22.00
## 675 20.0 25.00 0.42 29.00 34.00
## 681 57.0 45.00 31.00 16.00 28.00
## 693 37.0 19.00 15.00 59.00 24.00
## 698 22.0 51.00 42.00 43.00 31.00
## 710 39.0 20.00 20.00 65.00 36.00
## 712 44.0 49.00 22.00 8.00 28.00
## 719 17.0 31.00 36.00 35.00 47.00
## 728 1.0 4.00 54.00 63.00 20.00
## 733 19.0 32.00 16.00 19.00 37.00
## 739 28.0 47.00 48.00 31.00 51.00
## 740 70.5 24.00 28.50 27.00 14.00
## 741 20.0 22.00 1.00 46.00 19.00
## 761 30.0 32.00 25.00 35.00 28.00
## 767 19.0 27.00 2.00 62.00 29.00
## 769 51.0 62.00 18.00 41.00 21.00
## 774 37.0 39.00 30.00 28.00 24.00
## 777 34.0 19.00 31.00 67.00 48.00
## 779 31.0 54.00 42.00 26.00 33.00
## 784 8.0 40.00 24.00 21.00 22.00
## 791 44.0 54.00 38.00 51.00 47.00
## 793 33.0 15.00 21.00 41.00 29.00
## 794 61.0 23.00 35.00 28.00 30.00
## 816 25.0 0.83 21.00 32.00 32.50
## 826 22.0 22.00 26.00 50.00 18.00
## 827 39.0 27.00 29.00 31.00 17.00
## 829 36.0 36.00 25.00 80.00 29.00
## 833 19.0 56.00 39.00 27.00 42.00
## 838 36.0 25.00 16.00 39.00 51.00
## 840 26.0 36.00 31.00 23.00 30.00
## 847 20.0 30.00 28.00 30.00 16.00
## 850 22.0 22.00 20.00 24.00 0.17
## 860 21.0 36.00 38.00 25.00 36.00
## 864 23.0 33.00 33.00 41.00 27.00
## 869 80.0 6.00 24.00 24.00 25.00
## 879 29.0 22.00 42.00 25.00 61.00
## 889 3.0 26.00 19.00 28.00 44.00
## 902 20.0 26.00 26.00 39.00 27.00
## 914 2.0 49.00 23.00 35.00 19.00
## 921 25.0 57.00 45.00 22.00 30.00
## 925 39.0 35.00 63.00 13.00 33.00
## 928 42.0 18.00 15.00 45.00 17.00
## 931 42.0 29.00 2.00 25.00 28.00
## 933 27.0 64.00 34.00 52.00 26.00
## 939 30.0 17.00 61.00 40.00 29.00
## 946 19.0 24.00 14.50 41.00 16.00
## 950 28.0 21.00 24.00 26.00 16.00
## 957 42.0 45.00 18.00 22.00 42.00
## 968 42.0 21.00 25.00 32.00 28.00
## 975 21.0 54.00 21.00 51.00 32.00
## 976 26.0 30.00 60.50 29.00 42.00
## 977 54.0 21.00 27.00 30.00 40.00
## 980 34.0 13.00 22.00 45.00 18.00
## 983 1.0 70.00 37.00 29.00 28.00
## 985 42.0 50.00 49.00 27.00 13.00
## 994 61.0 14.00 45.00 57.00 60.00
## 999 18.0 32.00 27.00 49.00 48.00
## 1000 34.0 23.00 32.00 25.00 16.00
## 1003 53.0 11.00 17.00 18.50 25.00
## 1008 32.0 29.00 21.00 45.00 25.00
## 1013 14.0 11.00 24.00 25.00 41.00
## 1016 33.0 43.00 47.00 37.00 24.00
## 1019 45.0 25.00 31.00 45.00 32.50
## 1024 24.0 26.00 24.00 54.00 45.00
## 1025 42.0 32.00 40.00 35.00 20.00
## 1038 32.0 47.00 66.00 49.00 20.00
## 1040 9.0 48.00 39.00 21.00 1.00
## 1043 30.0 40.50 71.00 36.00 57.00
## 1052 47.0 18.00 26.00 23.00 30.00
## 1055 30.0 29.00 26.00 30.00 24.00
## 1060 43.0 14.00 15.00 22.00 22.00
## 1062 35.0 22.00 32.00 24.00 18.00
## 1065 17.0 22.00 55.00 63.00 19.00
## 1075 36.0 26.00 55.00 51.00 54.00
## 1080 33.0 24.00 30.00 17.00 8.00
## 1083 20.5 40.00 14.00 40.00 26.00
## 1091 30.0 2.00 56.00 11.00 20.00
## 1092 1.0 2.00 4.00 31.00 31.00
## 1097 4.0 36.00 13.00 34.50 40.50
## 1103 26.0 26.00 0.33 25.00 37.00
## 1108 39.0 39.00 49.00 33.00 60.00
## 1111 28.0 22.00 36.00 28.50 32.50
## 1117 2.0 47.00 21.00 17.00 53.00
## 1119 18.0 54.00 45.00 27.00 21.00
## 1125 25.0 27.00 23.00 32.00 29.00
## 1135 22.0 26.00 35.00 24.00 27.00
## 1136 3.0 46.00 29.00 8.00 24.00
## 1141 58.0 64.00 26.00 27.00 54.00
## 1147 2.0 31.00 36.50 34.00 37.00
## 1148 20.0 35.00 19.00 27.00 32.00
## 1157 70.0 22.00 27.00 17.00 32.00
## 1158 40.0 32.00 46.00 31.00 58.00
## 1159 18.0 49.00 49.00 49.00 2.00
## 1160 28.0 29.00 45.00 29.00 21.00
## 1163 18.0 56.00 41.00 50.00 28.00
## 1165 21.0 2.00 19.00 22.00 2.00
## 1166 24.0 23.00 23.00 17.00 30.00
## 1174 27.0 19.00 10.00 39.00 30.00
## 1178 28.0 30.00 41.00 32.00 0.92
## 1180 44.0 22.00 20.00 45.50 36.00
## 1181 11.5 26.00 62.00 21.00 42.00
## 1182 20.0 32.00 9.00 62.00 33.00
## 1184 34.0 42.00 46.00 21.00 28.00
## 1189 9.0 17.00 25.00 10.00 45.00
## 1193 36.5 24.00 28.50 35.00 36.00
## 1196 17.0 41.00 22.00 17.00 28.00
## 1204 31.0 19.00 28.00 42.00 54.00
## 1224 35.0 20.00 21.00 19.00 26.50
## 1231 34.0 36.00 24.00 23.00 31.00
## 1234 21.0 29.00 14.50 26.00 36.00
## 1236 22.0 25.00 67.00 17.00 10.00
## 1249 51.0 18.00 18.00 42.00 39.00
## 1250 24.0 44.00 45.00 27.00 27.00
## 1257 49.0 24.00 28.00 29.00 21.00
## 1258 17.0 52.00 23.00 28.00 40.00
## 1272 50.0 41.00 19.00 47.00 42.00
## 1274 11.0 35.00 28.00 21.00 39.00
## 1276 47.0 28.00 25.00 74.00 47.00
## 1300 41.0 18.00 52.00 20.00 14.00
## 1302 31.0 19.00 21.00 18.00 22.00
## 1305 36.0 20.00 63.00 60.00 30.00
## 1308 48.0 20.00 21.00 22.00 20.00
## 1309 28.0 39.00 19.00 31.00 30.00mice_corr <- complete(mice_corr,1) #save the newly imputed data choosing the first of
#five data sets (m=5)!!
#Check before & after distributions of the AGE variable
par(mfrow=c(1,2))
hist(my_df$Age, freq=F, main='Age: Original Data',
col='darkgreen', ylim=c(0,0.04))
hist(mice_corr$Age, freq=F, main='Age: MICE imputation',
col='lightgreen', ylim=c(0,0.04))
#Looks great!! Visual inspection shows NO change in distribution!
#Replace original age values with imputed values
my_df$Age <- mice_corr$Age
sum(is.na(my_df$Age))## [1] 0In the above code chunks, I performed imputation then verified before-and-after distributions using the histogram shown above then finally replaced the original with imputed Age variable.
my_df$Cabin <- NULL #remove CabinFeature engineering & Visualizations
Missing Age, Fare & Embark values have been successfully dealt with. Some features like Age can be decomposed to adequately expose the underlying structure. This consequently provides performance lift to models. Intuitively, passangers can be grouped into Child, Adult and Ederly by defining some rules, I will also define another variable Status (married/single). Title can be engineered too to expose its struture to the algorithms. In this post, I will focus on the 2 features Age & Name…it would also be interesting to compare model performance with and without these 2 features engineered to evaluate the significance of engineering these features.
1 Age
#Before I decopose AGE, lets do some visual analyses related to the age variable
my_hist <- hist(my_df$Age, breaks = 100, plot = F )
my_col <- ifelse(my_hist$breaks < 18, rgb(0.2,0.8,0.5,0.5), ifelse(my_hist$breaks >= 65, "purple"
, rgb(0.2,0.2,0.2,0.2)))
plot(my_hist, col = my_col, border =F, xlab = "color by age group",main ="", xlim = c(0,100))
#I will define age group as child < 18, 18>adult<65 and elderly >= 65. As can be seen there was more #adults than the 2 age groups
posn_jitter <- position_jitter(0.5, 0)
ggplot(my_df[1:891,], aes(x = factor(Pclass), y = Age, col = factor(Sex))) +
geom_jitter(size = 2, alpha = 0.5, position = posn_jitter) +
facet_grid(. ~ Survived)
#It didnt help being a middle aged male in class 3!! On the contrary, majority females in class 1
#survived
#Decompose age
my_df$Group[my_df$Age < 18] <- 'Child'
my_df$Group[my_df$Age >= 18 & my_df$Age < 65] <- 'Adult'
my_df$Group[my_df$Age >= 65] <- 'Elderly'
#Survival per age group
table(my_df$Group, my_df$Survived)##
## 0 1
## Adult 469 271
## Child 68 70
## Elderly 12 1The adult age group had many who perished (being the majority age group), notice the child age bracket almost tied between survival and death. The elderly barely survived!!!Let’s examine survival by class and age group.
table(my_df$Survived, my_df$Pclass,my_df$Group)## , , = Adult
##
##
## 1 2 3
## 0 74 93 302
## 1 121 65 85
##
## , , = Child
##
##
## 1 2 3
## 0 1 2 65
## 1 14 22 34
##
## , , = Elderly
##
##
## 1 2 3
## 0 5 2 5
## 1 1 0 0Most third class adults perished & majority first class survived (oh that’s not obvious!) and Only 1 child from first class perished. I would like to know if this child’s parents (I assume they were onboard) survived or persihed. The one elderly who survived was ofcourse from…(you guess it!) first class!
2 Name
Some basic string manipulation using stringr. I will clean the Name variable a bit by pulling titles from names and rearranging it in a more intuitive order i.e ‘first-middle-last’. Then I will coerce some character variables to factors.
my_df$Title <- stringr::str_replace_all(my_df$Name,'(.*,)|(\\..*)', '')
my_df$Name <- stringr::str_replace_all(my_df$Name, "(.*),.*\\. (.*)", "\\2 \\1")factors <- c('Embarked', 'Group', 'Title', 'Pclass')#left survived out...
my_df[factors] <- lapply(my_df[factors], as.factor)
sapply(my_df, class)## PassengerId Survived Pclass Name Sex Age
## "integer" "integer" "factor" "character" "factor" "numeric"
## SibSp Parch Ticket Fare Embarked Group
## "integer" "integer" "character" "numeric" "factor" "factor"
## Title
## "factor"The above wraps up some featue engineering. Before proceeding to further steps, I will remove some less useful variables and split the data back to train & test sets.
my_df$Ticket <- NULL
my_df$Name <- NULL
train <- my_df[1:891,]
test <- my_df[892:1309,]
#remove Survive from test set which is all NULL resulted from merging the 2 sets
test$Survived <- NULL
passenger_id <- test[,1]#pull passenger_id for submission
test <- test[,-1]#remove passenger_id from test set
train <- train[,-1]#remove passenger_id from train set its irrevalent
train$Survived <- ifelse(train$Survived == 1, 'Yes', 'No')
train$Survived <- as.factor(train$Survived)Feature Importance Analysis
This step is important more specifically for huge data sets. I like to evaluate variable importance and deteremine the right course of action given the use case. Removing ‘less’ important variables may not always be the best course of action, there are techniques that can be used to sort of ‘transform’ zero or near-zero-variance variables to more useful variables. In this post, I will rank features by importance and ignore the least important ones. One of the cons of removing the ‘least’ important is ofcourse loss of data, but more optimized run time could be considered a pro. So let’s proceed by randomly shuffling the data for a fair distribution then rank features by importance.
set.seed(10)
shuffled <- sample(nrow(train))
train <- train[shuffled,]set.seed(34)
control <- trainControl(method="cv", number=10, repeats=3)
model <- train(Survived~., data = train,trControl=control)
importance <- varImp(model, scale=FALSE)
print(importance)## rf variable importance
##
## only 20 most important variables shown (out of 28)
##
## Overall
## Fare 85.9436
## Age 77.8449
## Title Mr 66.3821
## Sexmale 58.5182
## Pclass3 27.7527
## SibSp 19.9088
## Parch 9.3094
## EmbarkedS 8.5218
## Title Mrs 6.6505
## Pclass2 5.5530
## Title Miss 5.2738
## Title Master 4.4991
## GroupChild 3.6696
## EmbarkedQ 3.1245
## Title Rev 2.7242
## Title Dr 1.1829
## GroupElderly 0.8836
## Title Major 0.6034
## Title Col 0.5053
## Title Don 0.3626plot(importance, main = "feature importance")
model building (Wide Cardinality)
It is best practice to fit multiple models within the problem domain then evaluate their performance and make a selection based on the winner!Caret makes this step easy by providing a flexible technique of creating a trainControl object that can be used as an interface for fitting multiple models by changing ‘method’ in the train function.
So lets fit some models!!First I will create a trainControl object that will be a platform for ALL models in order to compare apples-to-apples!
set.seed(5048)
myControl <- trainControl(method = "repeatedcv", number = 10,repeats = 3, classProbs = T,
verboseIter = F, summaryFunction = twoClassSummary)
metric <- 'ROC'
#1.RF
set.seed(5048)
RF_model <- train(Survived ~.,tuneLength = 3, data = train, method = "rf",
metric = metric,trControl = myControl)
print(RF_model)## Random Forest
##
## 891 samples
## 9 predictor
## 2 classes: 'No', 'Yes'
##
## No pre-processing
## Resampling: Cross-Validated (10 fold, repeated 3 times)
## Summary of sample sizes: 802, 802, 802, 802, 802, 802, ...
## Resampling results across tuning parameters:
##
## mtry ROC Sens Spec
## 2 0.8578662 0.8785971 0.7035014
## 15 0.8738719 0.9004938 0.7445098
## 28 0.8649648 0.8695511 0.7533053
##
## ROC was used to select the optimal model using the largest value.
## The final value used for the model was mtry = 15.#2.rpart
set.seed(5048)
RP_model <- train(Survived ~.,tuneLength = 3, data = train, method = "rpart",
metric = metric,trControl = myControl)
print(RP_model)## CART
##
## 891 samples
## 9 predictor
## 2 classes: 'No', 'Yes'
##
## No pre-processing
## Resampling: Cross-Validated (10 fold, repeated 3 times)
## Summary of sample sizes: 802, 802, 802, 802, 802, 802, ...
## Resampling results across tuning parameters:
##
## cp ROC Sens Spec
## 0.01461988 0.8215177 0.8695062 0.7161625
## 0.05263158 0.7843943 0.8215264 0.7347059
## 0.43274854 0.6573739 0.8615825 0.4531653
##
## ROC was used to select the optimal model using the largest value.
## The final value used for the model was cp = 0.01461988.#3.logit
set.seed(5048)
Grid<- expand.grid(decay=c(0.0001,0.00001,0.000001))
LR_model <- train(Survived~., data = train, method = 'multinom',metric = metric,trControl=myControl,
tuneGrid=Grid, MaxNWts=2000)## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 339.200226
## iter 20 value 329.873627
## iter 30 value 329.625048
## iter 40 value 329.621136
## final value 329.620363
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 339.198162
## iter 20 value 329.864270
## iter 30 value 329.590089
## iter 30 value 329.590086
## iter 30 value 329.590086
## final value 329.590086
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 339.197956
## iter 20 value 329.863331
## iter 30 value 329.583671
## final value 329.583441
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 337.699668
## iter 20 value 327.080218
## iter 30 value 326.913510
## iter 40 value 326.907423
## iter 50 value 326.904435
## final value 326.903707
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 337.697561
## iter 20 value 327.065929
## iter 30 value 326.866492
## iter 40 value 326.863446
## final value 326.863369
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 337.697351
## iter 20 value 327.064493
## iter 30 value 326.856139
## final value 326.855929
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 336.653709
## iter 20 value 324.663806
## iter 30 value 324.488954
## final value 324.487792
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 336.651693
## iter 20 value 324.649968
## iter 30 value 324.450452
## iter 30 value 324.450452
## iter 30 value 324.450452
## final value 324.450452
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 336.651492
## iter 20 value 324.648577
## iter 30 value 324.444282
## final value 324.444227
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 327.451453
## iter 20 value 317.051305
## iter 30 value 316.953430
## iter 40 value 316.949427
## final value 316.946476
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 327.449127
## iter 20 value 317.032999
## iter 30 value 316.907653
## iter 30 value 316.907651
## iter 30 value 316.907651
## final value 316.907651
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 327.448894
## iter 20 value 317.031150
## iter 30 value 316.898478
## final value 316.898422
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 330.028133
## iter 20 value 313.921621
## iter 30 value 313.498832
## iter 40 value 313.492375
## final value 313.489349
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 330.024108
## iter 20 value 313.911855
## iter 30 value 313.453908
## iter 40 value 313.452395
## final value 313.450445
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 330.023705
## iter 20 value 313.910876
## iter 30 value 313.445286
## final value 313.443781
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 321.517308
## iter 20 value 310.023704
## iter 30 value 309.853879
## iter 40 value 309.846010
## iter 50 value 309.842505
## final value 309.841668
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 321.513687
## iter 20 value 310.009989
## iter 30 value 309.806149
## iter 40 value 309.801784
## final value 309.801701
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 321.513325
## iter 20 value 310.008610
## iter 30 value 309.794888
## final value 309.794699
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 322.232379
## iter 20 value 313.050196
## iter 30 value 312.809784
## iter 40 value 312.802883
## iter 50 value 312.798704
## final value 312.798131
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 322.229409
## iter 20 value 313.033778
## iter 30 value 312.752813
## iter 40 value 312.748338
## final value 312.748321
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 322.229112
## iter 20 value 313.032128
## iter 30 value 312.739888
## final value 312.739617
## converged
## # weights: 30 (29 variable)
## initial value 555.210892
## iter 10 value 338.610859
## iter 20 value 325.168554
## iter 30 value 324.670909
## iter 40 value 324.661701
## iter 50 value 324.658641
## final value 324.657558
## converged
## # weights: 30 (29 variable)
## initial value 555.210892
## iter 10 value 338.606908
## iter 20 value 325.158609
## iter 30 value 324.619827
## iter 40 value 324.617786
## final value 324.615360
## converged
## # weights: 30 (29 variable)
## initial value 555.210892
## iter 10 value 338.606513
## iter 20 value 325.157612
## iter 30 value 324.609519
## final value 324.608025
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 343.938227
## iter 20 value 331.789031
## iter 30 value 331.424004
## final value 331.422235
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 343.936354
## iter 20 value 331.779137
## iter 30 value 331.385886
## final value 331.385874
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 343.936167
## iter 20 value 331.778145
## iter 30 value 331.379746
## final value 331.379487
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 329.444106
## iter 20 value 318.755630
## iter 30 value 318.551612
## iter 40 value 318.547507
## final value 318.544178
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 329.441932
## iter 20 value 318.741869
## iter 30 value 318.505868
## iter 40 value 318.503889
## final value 318.503792
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 329.441715
## iter 20 value 318.740486
## iter 30 value 318.496459
## final value 318.496244
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 324.751312
## iter 20 value 314.657445
## iter 30 value 314.389495
## final value 314.388415
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 324.748391
## iter 20 value 314.644708
## iter 30 value 314.348624
## iter 30 value 314.348623
## iter 30 value 314.348623
## final value 314.348623
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 324.748099
## iter 20 value 314.643430
## iter 30 value 314.342114
## final value 314.342007
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 336.712256
## iter 20 value 326.725192
## iter 30 value 326.525885
## iter 40 value 326.516137
## iter 50 value 326.510940
## final value 326.509627
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 336.710062
## iter 20 value 326.708654
## iter 30 value 326.468629
## iter 40 value 326.462189
## final value 326.462165
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 336.709843
## iter 20 value 326.706991
## iter 30 value 326.453952
## final value 326.453589
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 333.031299
## iter 20 value 324.730260
## iter 30 value 324.395328
## iter 40 value 324.383828
## iter 50 value 324.379906
## final value 324.378404
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 333.029020
## iter 20 value 324.720001
## iter 30 value 324.344950
## iter 40 value 324.340866
## final value 324.340025
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 333.028792
## iter 20 value 324.718972
## iter 30 value 324.333450
## final value 324.332892
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 341.474207
## iter 20 value 330.251732
## iter 30 value 329.951721
## iter 40 value 329.947554
## final value 329.945464
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 341.471366
## iter 20 value 330.239768
## iter 30 value 329.906144
## final value 329.906130
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 341.471082
## iter 20 value 330.238568
## iter 30 value 329.897149
## final value 329.896669
## converged
## # weights: 30 (29 variable)
## initial value 555.210892
## iter 10 value 329.799977
## iter 20 value 319.990660
## iter 30 value 319.782905
## iter 40 value 319.763044
## iter 50 value 319.756996
## final value 319.753931
## converged
## # weights: 30 (29 variable)
## initial value 555.210892
## iter 10 value 329.797781
## iter 20 value 319.979865
## iter 30 value 319.724335
## iter 40 value 319.716521
## final value 319.716238
## converged
## # weights: 30 (29 variable)
## initial value 555.210892
## iter 10 value 329.797562
## iter 20 value 319.978781
## iter 30 value 319.709342
## final value 319.709058
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 325.509616
## iter 20 value 314.777216
## iter 30 value 314.382699
## final value 314.381626
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 325.507388
## iter 20 value 314.767293
## iter 30 value 314.346479
## final value 314.346474
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 325.507165
## iter 20 value 314.766298
## iter 30 value 314.340820
## final value 314.340645
## converged
## # weights: 30 (29 variable)
## initial value 555.210892
## iter 10 value 334.290557
## iter 20 value 324.282832
## iter 30 value 323.948168
## iter 40 value 323.944925
## final value 323.942720
## converged
## # weights: 30 (29 variable)
## initial value 555.210892
## iter 10 value 334.287822
## iter 20 value 324.270539
## iter 30 value 323.902198
## final value 323.902182
## converged
## # weights: 30 (29 variable)
## initial value 555.210892
## iter 10 value 334.287548
## iter 20 value 324.269306
## iter 30 value 323.893480
## final value 323.892978
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 329.097871
## iter 20 value 319.753006
## iter 30 value 319.531876
## iter 40 value 319.528096
## final value 319.524798
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 329.095626
## iter 20 value 319.741127
## iter 30 value 319.489408
## iter 30 value 319.489406
## iter 30 value 319.489406
## final value 319.489406
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 329.095401
## iter 20 value 319.739934
## iter 30 value 319.481162
## final value 319.480971
## converged
## # weights: 30 (29 variable)
## initial value 556.597186
## iter 10 value 329.931441
## iter 20 value 318.081752
## iter 30 value 317.784330
## iter 40 value 317.779539
## iter 50 value 317.776797
## final value 317.776308
## converged
## # weights: 30 (29 variable)
## initial value 556.597186
## iter 10 value 329.929096
## iter 20 value 318.070199
## iter 30 value 317.738543
## final value 317.738531
## converged
## # weights: 30 (29 variable)
## initial value 556.597186
## iter 10 value 329.928862
## iter 20 value 318.069040
## iter 30 value 317.729353
## final value 317.728887
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 332.951617
## iter 20 value 319.522659
## iter 30 value 319.257113
## iter 40 value 319.252022
## iter 50 value 319.249628
## final value 319.249015
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 332.949573
## iter 20 value 319.509250
## iter 30 value 319.210041
## final value 319.210030
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 332.949369
## iter 20 value 319.507904
## iter 30 value 319.200338
## final value 319.199922
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 334.813405
## iter 20 value 324.500172
## iter 30 value 324.331590
## iter 40 value 324.324239
## iter 50 value 324.320701
## final value 324.319589
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 334.811305
## iter 20 value 324.485498
## iter 30 value 324.281553
## iter 40 value 324.278126
## final value 324.278108
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 334.811095
## iter 20 value 324.484023
## iter 30 value 324.270487
## final value 324.270312
## converged
## # weights: 30 (29 variable)
## initial value 555.210892
## iter 10 value 342.055388
## iter 20 value 327.645581
## final value 327.391445
## converged
## # weights: 30 (29 variable)
## initial value 555.210892
## iter 10 value 342.052509
## iter 20 value 327.634378
## iter 30 value 327.355318
## iter 30 value 327.355314
## iter 30 value 327.355314
## final value 327.355314
## converged
## # weights: 30 (29 variable)
## initial value 555.210892
## iter 10 value 342.052222
## iter 20 value 327.633255
## iter 30 value 327.349609
## final value 327.349483
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 318.160201
## iter 20 value 308.137432
## iter 30 value 307.863196
## iter 40 value 307.850792
## iter 50 value 307.845190
## final value 307.843085
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 318.156762
## iter 20 value 308.123668
## iter 30 value 307.803853
## iter 40 value 307.796212
## final value 307.796028
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 318.156418
## iter 20 value 308.122286
## iter 30 value 307.788179
## final value 307.787623
## converged
## # weights: 30 (29 variable)
## initial value 556.597186
## iter 10 value 338.720826
## iter 20 value 327.644544
## iter 30 value 327.409805
## iter 40 value 327.398845
## iter 50 value 327.394440
## final value 327.393436
## converged
## # weights: 30 (29 variable)
## initial value 556.597186
## iter 10 value 338.718970
## iter 20 value 327.633173
## iter 30 value 327.360658
## iter 40 value 327.355730
## final value 327.354620
## converged
## # weights: 30 (29 variable)
## initial value 556.597186
## iter 10 value 338.718784
## iter 20 value 327.632032
## iter 30 value 327.348304
## final value 327.347737
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 356.528515
## iter 20 value 328.036681
## iter 30 value 327.379484
## iter 40 value 327.371069
## iter 50 value 327.368675
## final value 327.367938
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 356.524855
## iter 20 value 328.028227
## iter 30 value 327.330713
## iter 40 value 327.328445
## final value 327.325881
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 356.524489
## iter 20 value 328.027381
## iter 30 value 327.321596
## final value 327.318988
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 340.964676
## iter 20 value 326.341665
## iter 30 value 326.064250
## iter 40 value 326.057516
## final value 326.053830
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 340.961452
## iter 20 value 326.331120
## iter 30 value 326.018831
## iter 40 value 326.016653
## final value 326.015262
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 340.961129
## iter 20 value 326.330062
## iter 30 value 326.009059
## final value 326.008471
## converged
## # weights: 30 (29 variable)
## initial value 555.210892
## iter 10 value 327.455840
## iter 20 value 319.548931
## iter 30 value 319.373031
## iter 40 value 319.353161
## iter 50 value 319.347269
## final value 319.345282
## converged
## # weights: 30 (29 variable)
## initial value 555.210892
## iter 10 value 327.453115
## iter 20 value 319.533613
## iter 30 value 319.312048
## iter 40 value 319.300734
## final value 319.300637
## converged
## # weights: 30 (29 variable)
## initial value 555.210892
## iter 10 value 327.452842
## iter 20 value 319.532070
## iter 30 value 319.293417
## final value 319.293234
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 320.185045
## iter 20 value 305.678067
## iter 30 value 305.488938
## final value 305.487784
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 320.182547
## iter 20 value 305.661284
## iter 30 value 305.444787
## final value 305.444779
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 320.182297
## iter 20 value 305.659597
## iter 30 value 305.437922
## final value 305.437767
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 331.069895
## iter 20 value 322.591479
## iter 30 value 322.520229
## final value 322.518308
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 331.067713
## iter 20 value 322.576433
## final value 322.486259
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 331.067495
## iter 20 value 322.574915
## iter 30 value 322.480425
## final value 322.480400
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 331.960141
## iter 20 value 321.779845
## iter 30 value 321.637854
## iter 40 value 321.633458
## iter 50 value 321.630713
## final value 321.629781
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 331.957910
## iter 20 value 321.765056
## iter 30 value 321.593688
## iter 40 value 321.591913
## final value 321.591824
## converged
## # weights: 30 (29 variable)
## initial value 555.904039
## iter 10 value 331.957687
## iter 20 value 321.763567
## iter 30 value 321.584758
## final value 321.584659
## converged
## # weights: 30 (29 variable)
## initial value 617.594138
## iter 10 value 372.226605
## iter 20 value 358.037279
## iter 30 value 357.744138
## iter 40 value 357.737679
## iter 50 value 357.734468
## final value 357.733860
## convergedprint(LR_model)## Penalized Multinomial Regression
##
## 891 samples
## 9 predictor
## 2 classes: 'No', 'Yes'
##
## No pre-processing
## Resampling: Cross-Validated (10 fold, repeated 3 times)
## Summary of sample sizes: 802, 802, 802, 802, 802, 802, ...
## Resampling results across tuning parameters:
##
## decay ROC Sens Spec
## 1e-06 0.8563907 0.8755892 0.742521
## 1e-05 0.8567565 0.8749832 0.742521
## 1e-04 0.8568468 0.8755892 0.742521
##
## ROC was used to select the optimal model using the largest value.
## The final value used for the model was decay = 1e-04.#4.glmnet
set.seed(5048)
GN_model <- train(Survived~.,train,tuneGrid = expand.grid(alpha = 0:1,lambda = seq(0.0001, 1, length = 20)),
method = "glmnet", metric = metric,trControl = myControl)
print(GN_model)## glmnet
##
## 891 samples
## 9 predictor
## 2 classes: 'No', 'Yes'
##
## No pre-processing
## Resampling: Cross-Validated (10 fold, repeated 3 times)
## Summary of sample sizes: 802, 802, 802, 802, 802, 802, ...
## Resampling results across tuning parameters:
##
## alpha lambda ROC Sens Spec
## 0 0.00010000 0.8598042 0.8786195 0.7376471
## 0 0.05272632 0.8599021 0.8792368 0.7288796
## 0 0.10535263 0.8585473 0.8798316 0.7191036
## 0 0.15797895 0.8574646 0.8743659 0.7151821
## 0 0.21060526 0.8559011 0.8725477 0.7112605
## 0 0.26323158 0.8548500 0.8725477 0.7083193
## 0 0.31585789 0.8543002 0.8731538 0.7073389
## 0 0.36848421 0.8532351 0.8755668 0.7053782
## 0 0.42111053 0.8527179 0.8785971 0.7024650
## 0 0.47373684 0.8519562 0.8870932 0.6926611
## 0 0.52636316 0.8512096 0.8968013 0.6721849
## 0 0.57898947 0.8504136 0.9028620 0.6585434
## 0 0.63161579 0.8499344 0.9095511 0.6400280
## 0 0.68424211 0.8491861 0.9156453 0.6263305
## 0 0.73686842 0.8487267 0.9210999 0.6087955
## 0 0.78949474 0.8483737 0.9283726 0.5921849
## 0 0.84212105 0.8478794 0.9392929 0.5794958
## 0 0.89474737 0.8475410 0.9417172 0.5639216
## 0 0.94737368 0.8470101 0.9478002 0.5492717
## 0 1.00000000 0.8468505 0.9557015 0.5376471
## 1 0.00010000 0.8580994 0.8761953 0.7435014
## 1 0.05272632 0.8405712 0.8251964 0.7279552
## 1 0.10535263 0.8412925 0.8494613 0.6821289
## 1 0.15797895 0.7923443 0.8524916 0.6811485
## 1 0.21060526 0.7909540 1.0000000 0.0000000
## 1 0.26323158 0.7021192 1.0000000 0.0000000
## 1 0.31585789 0.5000000 1.0000000 0.0000000
## 1 0.36848421 0.5000000 1.0000000 0.0000000
## 1 0.42111053 0.5000000 1.0000000 0.0000000
## 1 0.47373684 0.5000000 1.0000000 0.0000000
## 1 0.52636316 0.5000000 1.0000000 0.0000000
## 1 0.57898947 0.5000000 1.0000000 0.0000000
## 1 0.63161579 0.5000000 1.0000000 0.0000000
## 1 0.68424211 0.5000000 1.0000000 0.0000000
## 1 0.73686842 0.5000000 1.0000000 0.0000000
## 1 0.78949474 0.5000000 1.0000000 0.0000000
## 1 0.84212105 0.5000000 1.0000000 0.0000000
## 1 0.89474737 0.5000000 1.0000000 0.0000000
## 1 0.94737368 0.5000000 1.0000000 0.0000000
## 1 1.00000000 0.5000000 1.0000000 0.0000000
##
## ROC was used to select the optimal model using the largest value.
## The final values used for the model were alpha = 0 and lambda = 0.05272632.#5.SVM
set.seed(5048)
SVM_model <- train(Survived~.,data = train, method = "svmRadial",metric = metric,
trControl = myControl)
print(SVM_model)## Support Vector Machines with Radial Basis Function Kernel
##
## 891 samples
## 9 predictor
## 2 classes: 'No', 'Yes'
##
## No pre-processing
## Resampling: Cross-Validated (10 fold, repeated 3 times)
## Summary of sample sizes: 802, 802, 802, 802, 802, 802, ...
## Resampling results across tuning parameters:
##
## C ROC Sens Spec
## 0.25 0.7693646 0.8397082 0.5008683
## 0.50 0.7879707 0.8572840 0.5056863
## 1.00 0.7969022 0.8245230 0.5717647
##
## Tuning parameter 'sigma' was held constant at a value of 0.01187786
## ROC was used to select the optimal model using the largest value.
## The final values used for the model were sigma = 0.01187786 and C = 1.#6.naive bayes
set.seed(5048)
NB_model <- train(Survived~., data = train, method="nb", metric=metric,
trControl= myControl)
print(NB_model)## Naive Bayes
##
## 891 samples
## 9 predictor
## 2 classes: 'No', 'Yes'
##
## No pre-processing
## Resampling: Cross-Validated (10 fold, repeated 3 times)
## Summary of sample sizes: 802, 802, 802, 802, 802, 802, ...
## Resampling results across tuning parameters:
##
## usekernel ROC Sens Spec
## FALSE NaN NaN NaN
## TRUE 0.8418008 0.5417172 0.8563585
##
## Tuning parameter 'fL' was held constant at a value of 0
## Tuning
## parameter 'adjust' was held constant at a value of 1
## ROC was used to select the optimal model using the largest value.
## The final values used for the model were fL = 0, usekernel = TRUE
## and adjust = 1.#7.adaboost
set.seed(5048)
AB_model <- train(Survived~., data = train, method="adaboost", metric=metric,
trControl= myControl)
print(AB_model)## AdaBoost Classification Trees
##
## 891 samples
## 9 predictor
## 2 classes: 'No', 'Yes'
##
## No pre-processing
## Resampling: Cross-Validated (10 fold, repeated 3 times)
## Summary of sample sizes: 802, 802, 802, 802, 802, 802, ...
## Resampling results across tuning parameters:
##
## nIter method ROC Sens Spec
## 50 Adaboost.M1 0.8595000 0.8470595 0.7396359
## 50 Real adaboost 0.7278513 0.8525365 0.7104762
## 100 Adaboost.M1 0.8605329 0.8373288 0.7474790
## 100 Real adaboost 0.6854587 0.8579686 0.7193557
## 150 Adaboost.M1 0.8597469 0.8379461 0.7493838
## 150 Real adaboost 0.6782580 0.8567565 0.7212325
##
## ROC was used to select the optimal model using the largest value.
## The final values used for the model were nIter = 100 and method
## = Adaboost.M1.#8.Shrinkage Discrimant Analysis
set.seed(5048)
SDA_model <- train(Survived~., data = train, method="sda", metric=metric,
trControl=myControl)## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0437
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0437
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0437
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.044
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.044
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.044
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0215
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0323
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0215
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0323
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0215
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0323
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0436
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0436
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0436
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0407
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0407
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0407
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0424
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0424
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0424
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0523
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0523
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0523
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 801
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0495
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 801
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0495
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 801
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0495
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0427
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0427
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0427
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0415
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0415
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0415
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0386
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0386
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0386
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0424
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0424
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0424
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0479
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0479
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0479
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0443
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0443
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0443
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 801
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0459
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 801
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0459
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 801
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0459
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0397
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0397
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0397
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 801
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0428
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 801
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0428
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 801
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0428
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0411
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0411
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0411
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 803
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0497
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 803
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0497
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 803
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0497
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0434
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0434
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0434
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0336
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0336
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0336
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 801
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.041
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 801
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.041
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 801
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.041
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0429
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0429
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0429
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 803
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0425
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 803
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0425
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 803
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0425
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0465
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0465
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0465
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0472
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0472
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0472
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 801
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0445
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 801
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0445
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 801
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0217
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0445
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.043
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.043
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.043
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0463
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0463
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0463
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0432
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0432
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0.5
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 802
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.022
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0432
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 1
## Prediction uses 28 features.
## Prediction uses 28 features.
## Number of variables: 28
## Number of observations: 891
## Number of classes: 2
##
## Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.0197
## Estimating variances (pooled across classes)
## Estimating optimal shrinkage intensity lambda.var (variance vector): 0.0394
##
##
## Computing inverse correlation matrix (pooled across classes)
## Specified shrinkage intensity lambda (correlation matrix): 0print(SDA_model)## Shrinkage Discriminant Analysis
##
## 891 samples
## 9 predictor
## 2 classes: 'No', 'Yes'
##
## No pre-processing
## Resampling: Cross-Validated (10 fold, repeated 3 times)
## Summary of sample sizes: 802, 802, 802, 802, 802, 802, ...
## Resampling results across tuning parameters:
##
## lambda ROC Sens Spec
## 0.0 0.8618544 0.8737710 0.7386275
## 0.5 0.8582613 0.8482604 0.7337815
## 1.0 0.8394343 0.8312570 0.7268908
##
## Tuning parameter 'diagonal' was held constant at a value of FALSE
## ROC was used to select the optimal model using the largest value.
## The final values used for the model were diagonal = FALSE and lambda = 0.#9.glm
set.seed(5048)
GLM_model <- train(Survived~., data = train, method="glm", metric=metric,
trControl=myControl)
print(GLM_model)## Generalized Linear Model
##
## 891 samples
## 9 predictor
## 2 classes: 'No', 'Yes'
##
## No pre-processing
## Resampling: Cross-Validated (10 fold, repeated 3 times)
## Summary of sample sizes: 802, 802, 802, 802, 802, 802, ...
## Resampling results:
##
## ROC Sens Spec
## 0.8544141 0.8743771 0.7395798
##
## #10.xgbTree
set.seed(5048)
XGB.grid <- expand.grid(nrounds = 100,#max number of Iter
eta = c(0.1,0.4, 1),#step size/learning rate
max_depth = seq(5,10),#try seq from 5 to10
gamma = c(0.1,0.5,1.0),#min loss fuc-try small to large
colsample_bytree = c(0.4,0.8,1),
#subsample = 0.5,#prevent overfitting
min_child_weight = 1)
XGB_model <- train(Survived~., data = train, method="xgbTree", metric=metric,
tuneGrid = XGB.grid,trControl=myControl)
print(XGB_model)## eXtreme Gradient Boosting
##
## 891 samples
## 9 predictor
## 2 classes: 'No', 'Yes'
##
## No pre-processing
## Resampling: Cross-Validated (10 fold, repeated 3 times)
## Summary of sample sizes: 802, 802, 802, 802, 802, 802, ...
## Resampling results across tuning parameters:
##
## eta max_depth gamma colsample_bytree ROC Sens Spec
## 0.1 5 0.1 0.4 0.8770200 0.8913805 0.7454902
## 0.1 5 0.1 0.8 0.8754003 0.8913356 0.7387115
## 0.1 5 0.1 1.0 0.8730337 0.8919416 0.7377591
## 0.1 5 0.5 0.4 0.8741267 0.8858923 0.7435854
## 0.1 5 0.5 0.8 0.8749973 0.8919529 0.7339216
## 0.1 5 0.5 1.0 0.8738976 0.8907407 0.7386555
## 0.1 5 1.0 0.4 0.8762817 0.8919529 0.7445658
## 0.1 5 1.0 0.8 0.8720327 0.8956004 0.7318768
## 0.1 5 1.0 1.0 0.8689714 0.8931762 0.7250420
## 0.1 6 0.1 0.4 0.8779395 0.8889226 0.7387115
## 0.1 6 0.1 0.8 0.8733061 0.8883053 0.7416807
## 0.1 6 0.1 1.0 0.8727448 0.8883053 0.7455742
## 0.1 6 0.5 0.4 0.8763529 0.8828844 0.7485154
## 0.1 6 0.5 0.8 0.8730889 0.8846689 0.7474230
## 0.1 6 0.5 1.0 0.8720997 0.8907520 0.7406162
## 0.1 6 1.0 0.4 0.8785112 0.8907520 0.7416807
## 0.1 6 1.0 0.8 0.8724027 0.8907520 0.7445658
## 0.1 6 1.0 1.0 0.8695349 0.8876992 0.7270868
## 0.1 7 0.1 0.4 0.8795537 0.8907407 0.7534454
## 0.1 7 0.1 0.8 0.8723437 0.8834792 0.7485154
## 0.1 7 0.1 1.0 0.8711740 0.8828620 0.7465266
## 0.1 7 0.5 0.4 0.8804450 0.8925926 0.7514846
## 0.1 7 0.5 0.8 0.8735936 0.8858923 0.7552661
## 0.1 7 0.5 1.0 0.8715459 0.8852637 0.7348179
## 0.1 7 1.0 0.4 0.8781328 0.8858923 0.7436695
## 0.1 7 1.0 0.8 0.8726814 0.8919641 0.7435574
## 0.1 7 1.0 1.0 0.8695151 0.8907407 0.7299720
## 0.1 8 0.1 0.4 0.8778011 0.8895623 0.7543137
## 0.1 8 0.1 0.8 0.8716130 0.8834568 0.7387395
## 0.1 8 0.1 1.0 0.8699751 0.8810438 0.7406162
## 0.1 8 0.5 0.4 0.8805813 0.8828620 0.7563025
## 0.1 8 0.5 0.8 0.8717825 0.8810438 0.7455462
## 0.1 8 0.5 1.0 0.8689664 0.8816386 0.7425770
## 0.1 8 1.0 0.4 0.8798932 0.8871380 0.7445658
## 0.1 8 1.0 0.8 0.8746576 0.8901571 0.7396919
## 0.1 8 1.0 1.0 0.8681939 0.8852750 0.7377311
## 0.1 9 0.1 0.4 0.8792333 0.8810662 0.7562745
## 0.1 9 0.1 0.8 0.8695766 0.8822559 0.7426331
## 0.1 9 0.1 1.0 0.8681395 0.8773962 0.7397199
## 0.1 9 0.5 0.4 0.8787565 0.8852974 0.7563305
## 0.1 9 0.5 0.8 0.8721337 0.8828732 0.7445378
## 0.1 9 0.5 1.0 0.8698041 0.8779910 0.7425770
## 0.1 9 1.0 0.4 0.8791294 0.8901347 0.7464706
## 0.1 9 1.0 0.8 0.8751922 0.8883053 0.7397479
## 0.1 9 1.0 1.0 0.8683852 0.8840516 0.7367507
## 0.1 10 0.1 0.4 0.8779130 0.8859259 0.7466387
## 0.1 10 0.1 0.8 0.8704751 0.8798092 0.7396639
## 0.1 10 0.1 1.0 0.8677968 0.8713019 0.7386835
## 0.1 10 0.5 0.4 0.8785817 0.8852750 0.7475070
## 0.1 10 0.5 0.8 0.8727809 0.8828395 0.7494678
## 0.1 10 0.5 1.0 0.8673040 0.8773737 0.7435574
## 0.1 10 1.0 0.4 0.8792213 0.8840741 0.7514286
## 0.1 10 1.0 0.8 0.8731551 0.8901347 0.7426331
## 0.1 10 1.0 1.0 0.8691274 0.8840629 0.7397479
## 0.4 5 0.1 0.4 0.8701244 0.8761167 0.7494398
## 0.4 5 0.1 0.8 0.8699534 0.8695062 0.7514006
## 0.4 5 0.1 1.0 0.8690678 0.8682604 0.7503922
## 0.4 5 0.5 0.4 0.8761020 0.8755668 0.7591877
## 0.4 5 0.5 0.8 0.8717598 0.8780022 0.7436415
## 0.4 5 0.5 1.0 0.8720020 0.8901010 0.7455182
## 0.4 5 1.0 0.4 0.8758565 0.8871044 0.7465266
## 0.4 5 1.0 0.8 0.8731762 0.8901459 0.7455462
## 0.4 5 1.0 1.0 0.8718831 0.8949832 0.7387395
## 0.4 6 0.1 0.4 0.8715537 0.8670932 0.7475350
## 0.4 6 0.1 0.8 0.8698248 0.8694949 0.7514846
## 0.4 6 0.1 1.0 0.8668897 0.8664646 0.7543417
## 0.4 6 0.5 0.4 0.8717412 0.8737486 0.7522969
## 0.4 6 0.5 0.8 0.8742857 0.8785859 0.7513445
## 0.4 6 0.5 1.0 0.8702840 0.8834231 0.7474790
## 0.4 6 1.0 0.4 0.8763193 0.8834680 0.7494678
## 0.4 6 1.0 0.8 0.8724466 0.8877104 0.7503081
## 0.4 6 1.0 1.0 0.8725429 0.8864983 0.7426050
## 0.4 7 0.1 0.4 0.8706705 0.8646689 0.7485434
## 0.4 7 0.1 0.8 0.8667877 0.8616274 0.7543417
## 0.4 7 0.1 1.0 0.8671466 0.8610101 0.7533613
## 0.4 7 0.5 0.4 0.8762658 0.8773962 0.7456022
## 0.4 7 0.5 0.8 0.8718277 0.8780135 0.7484874
## 0.4 7 0.5 1.0 0.8680977 0.8731762 0.7270308
## 0.4 7 1.0 0.4 0.8778235 0.8871044 0.7426611
## 0.4 7 1.0 0.8 0.8711383 0.8858923 0.7476471
## 0.4 7 1.0 1.0 0.8737236 0.8840516 0.7435854
## 0.4 8 0.1 0.4 0.8711014 0.8646240 0.7582913
## 0.4 8 0.1 0.8 0.8686517 0.8622110 0.7592157
## 0.4 8 0.1 1.0 0.8657235 0.8622334 0.7504202
## 0.4 8 0.5 0.4 0.8717729 0.8798429 0.7465826
## 0.4 8 0.5 0.8 0.8691987 0.8731650 0.7377311
## 0.4 8 0.5 1.0 0.8679243 0.8743659 0.7406443
## 0.4 8 1.0 0.4 0.8792320 0.8804714 0.7485154
## 0.4 8 1.0 0.8 0.8707583 0.8749719 0.7465826
## 0.4 8 1.0 1.0 0.8689148 0.8864983 0.7347619
## 0.4 9 0.1 0.4 0.8688506 0.8622110 0.7572829
## 0.4 9 0.1 0.8 0.8655634 0.8604265 0.7543978
## 0.4 9 0.1 1.0 0.8664954 0.8604265 0.7533613
## 0.4 9 0.5 0.4 0.8741043 0.8658586 0.7514286
## 0.4 9 0.5 0.8 0.8696126 0.8718967 0.7475350
## 0.4 9 0.5 1.0 0.8669141 0.8737710 0.7406443
## 0.4 9 1.0 0.4 0.8743352 0.8761953 0.7387955
## 0.4 9 1.0 0.8 0.8699083 0.8816386 0.7466106
## 0.4 9 1.0 1.0 0.8703292 0.8864871 0.7270308
## 0.4 10 0.1 0.4 0.8696781 0.8634119 0.7533613
## 0.4 10 0.1 0.8 0.8692521 0.8585859 0.7513445
## 0.4 10 0.1 1.0 0.8657611 0.8604153 0.7523810
## 0.4 10 0.5 0.4 0.8747608 0.8768350 0.7563866
## 0.4 10 0.5 0.8 0.8660321 0.8670819 0.7357143
## 0.4 10 0.5 1.0 0.8708634 0.8707295 0.7405882
## 0.4 10 1.0 0.4 0.8729491 0.8846801 0.7386555
## 0.4 10 1.0 0.8 0.8731182 0.8804153 0.7398599
## 0.4 10 1.0 1.0 0.8684086 0.8852974 0.7358263
## 1.0 5 0.1 0.4 0.8627152 0.8482379 0.7417367
## 1.0 5 0.1 0.8 0.8619993 0.8463749 0.7475630
## 1.0 5 0.1 1.0 0.8640709 0.8427722 0.7591597
## 1.0 5 0.5 0.4 0.8632145 0.8598092 0.7348459
## 1.0 5 0.5 0.8 0.8626045 0.8634231 0.7456863
## 1.0 5 0.5 1.0 0.8644114 0.8609989 0.7416527
## 1.0 5 1.0 0.4 0.8659079 0.8695511 0.7337535
## 1.0 5 1.0 0.8 0.8611989 0.8670707 0.7387675
## 1.0 5 1.0 1.0 0.8653418 0.8725365 0.7387675
## 1.0 6 0.1 0.4 0.8613921 0.8512121 0.7348459
## 1.0 6 0.1 0.8 0.8603287 0.8543210 0.7465266
## 1.0 6 0.1 1.0 0.8633673 0.8616049 0.7493838
## 1.0 6 0.5 0.4 0.8616148 0.8554994 0.7435014
## 1.0 6 0.5 0.8 0.8606632 0.8670707 0.7436134
## 1.0 6 0.5 1.0 0.8619959 0.8725253 0.7465266
## 1.0 6 1.0 0.4 0.8663571 0.8658698 0.7426050
## 1.0 6 1.0 0.8 0.8618939 0.8743547 0.7388796
## 1.0 6 1.0 1.0 0.8578680 0.8689001 0.7348179
## 1.0 7 0.1 0.4 0.8617985 0.8464310 0.7426891
## 1.0 7 0.1 0.8 0.8615643 0.8519080 0.7543417
## 1.0 7 0.1 1.0 0.8630143 0.8513244 0.7495238
## 1.0 7 0.5 0.4 0.8582012 0.8537262 0.7407843
## 1.0 7 0.5 0.8 0.8612468 0.8603928 0.7486275
## 1.0 7 0.5 1.0 0.8594892 0.8561953 0.7358824
## 1.0 7 1.0 0.4 0.8625924 0.8585410 0.7445378
## 1.0 7 1.0 0.8 0.8652301 0.8682267 0.7379272
## 1.0 7 1.0 1.0 0.8619204 0.8664646 0.7221569
## 1.0 8 0.1 0.4 0.8594713 0.8415937 0.7494678
## 1.0 8 0.1 0.8 0.8594280 0.8458586 0.7543417
## 1.0 8 0.1 1.0 0.8617940 0.8555668 0.7494398
## 1.0 8 0.5 0.4 0.8615279 0.8597755 0.7358263
## 1.0 8 0.5 0.8 0.8596818 0.8573962 0.7514006
## 1.0 8 0.5 1.0 0.8571568 0.8586195 0.7416527
## 1.0 8 1.0 0.4 0.8638918 0.8658361 0.7377031
## 1.0 8 1.0 0.8 0.8663975 0.8549158 0.7359104
## 1.0 8 1.0 1.0 0.8599928 0.8664646 0.7309244
## 1.0 9 0.1 0.4 0.8595313 0.8403816 0.7484034
## 1.0 9 0.1 0.8 0.8621719 0.8519192 0.7494958
## 1.0 9 0.1 1.0 0.8604203 0.8506958 0.7494398
## 1.0 9 0.5 0.4 0.8622858 0.8482492 0.7377031
## 1.0 9 0.5 0.8 0.8583992 0.8604040 0.7376751
## 1.0 9 0.5 1.0 0.8592445 0.8585859 0.7288515
## 1.0 9 1.0 0.4 0.8606443 0.8719641 0.7436134
## 1.0 9 1.0 0.8 0.8670578 0.8664759 0.7523249
## 1.0 9 1.0 1.0 0.8597146 0.8670595 0.7365546
## 1.0 10 0.1 0.4 0.8624199 0.8488664 0.7495518
## 1.0 10 0.1 0.8 0.8645495 0.8463749 0.7514846
## 1.0 10 0.1 1.0 0.8629077 0.8531538 0.7524090
## 1.0 10 0.5 0.4 0.8688883 0.8512907 0.7475630
## 1.0 10 0.5 0.8 0.8610938 0.8640404 0.7465266
## 1.0 10 0.5 1.0 0.8601990 0.8640853 0.7368627
## 1.0 10 1.0 0.4 0.8605737 0.8591919 0.7377311
## 1.0 10 1.0 0.8 0.8654751 0.8755668 0.7396078
## 1.0 10 1.0 1.0 0.8574628 0.8664534 0.7455462
##
## Tuning parameter 'nrounds' was held constant at a value of 100
##
## Tuning parameter 'min_child_weight' was held constant at a value of 1
## ROC was used to select the optimal model using the largest value.
## The final values used for the model were nrounds = 100, max_depth = 8,
## eta = 0.1, gamma = 0.5, colsample_bytree = 0.4 and min_child_weight = 1.Model Performance Evaluation & Final Candidate Selection
#Using ROC as measure
#create resample object assigning models to their descriptive names
resample_results <- resamples(list(RF=RF_model,CART=RP_model,GLMNET=GN_model,
SDA=SDA_model,SVM=SVM_model, GLM = GLM_model, LR=LR_model,
ADABoost = AB_model, NaiveBayes=NB_model,xgbTree =XGB_model))
#Check summary on all metrics
summary(resample_results, metric = c('ROC','Sens','Spec'))##
## Call:
## summary.resamples(object = resample_results, metric = c("ROC",
## "Sens", "Spec"))
##
## Models: RF, CART, GLMNET, SDA, SVM, GLM, LR, ADABoost, NaiveBayes, xgbTree
## Number of resamples: 30
##
## ROC
## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
## RF 0.7128 0.8419 0.8866 0.8739 0.9120 0.9439 0
## CART 0.6997 0.8001 0.8281 0.8215 0.8549 0.8947 0
## GLMNET 0.7201 0.8219 0.8594 0.8599 0.9067 0.9353 0
## SDA 0.7072 0.8313 0.8682 0.8619 0.9024 0.9540 0
## SVM 0.6463 0.7587 0.7850 0.7969 0.8380 0.8861 0
## GLM 0.7190 0.8226 0.8600 0.8544 0.8953 0.9239 0
## LR 0.7190 0.8231 0.8603 0.8568 0.8967 0.9390 0
## ADABoost 0.7270 0.8232 0.8671 0.8605 0.8959 0.9631 0
## NaiveBayes 0.7024 0.8119 0.8477 0.8418 0.8851 0.9198 0
## xgbTree 0.7519 0.8441 0.8889 0.8806 0.9232 0.9471 0
##
## Sens
## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
## RF 0.8000 0.8727 0.8909 0.9005 0.9409 0.9818 0
## CART 0.7455 0.8364 0.8727 0.8695 0.9087 0.9455 0
## GLMNET 0.7818 0.8591 0.8889 0.8792 0.9045 0.9636 0
## SDA 0.7818 0.8545 0.8808 0.8738 0.8909 0.9630 0
## SVM 0.7407 0.7818 0.8348 0.8245 0.8545 0.9455 0
## GLM 0.7818 0.8545 0.8727 0.8744 0.9091 0.9630 0
## LR 0.7818 0.8545 0.8727 0.8756 0.9091 0.9630 0
## ADABoost 0.7455 0.8000 0.8364 0.8373 0.8545 0.9636 0
## NaiveBayes 0.0000 0.4227 0.5727 0.5417 0.7818 0.9455 0
## xgbTree 0.7636 0.8545 0.8818 0.8829 0.9091 0.9818 0
##
## Spec
## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
## RF 0.5294 0.6838 0.7647 0.7445 0.7941 0.9412 0
## CART 0.5000 0.6788 0.7353 0.7162 0.7714 0.8235 0
## GLMNET 0.5294 0.6788 0.7353 0.7289 0.7884 0.9118 0
## SDA 0.5588 0.6788 0.7353 0.7386 0.7941 0.9412 0
## SVM 0.3235 0.5000 0.5588 0.5718 0.6546 0.7714 0
## GLM 0.5588 0.6788 0.7353 0.7396 0.8000 0.9412 0
## LR 0.5588 0.6788 0.7353 0.7425 0.8000 0.9412 0
## ADABoost 0.5588 0.7059 0.7353 0.7475 0.7941 0.9118 0
## NaiveBayes 0.5588 0.7664 0.8971 0.8564 0.9632 1.0000 0
## xgbTree 0.5882 0.7059 0.7647 0.7563 0.7985 0.9412 0#Analyze MODEL performance
bwplot(resample_results, metric = metric)
bwplot(resample_results, metric = c('ROC','Sens','Spec'))
densityplot(resample_results, metric = metric, auto.key = list(columns = 3))
As seen from the model performance visualizations above, extreme gradient boosting performed great with RandomForest coming in second based on ROC/AUC values. So the xgbTree model will be used to perform prediction. Furthermore, in order to provide more optimal performance, an ensamble method called Stacking can be employed in which a set of algorithms are used as level 0 models to predict with and their output used by a level 1 model for optimal prediction. This will be demonstrated in my next blog post. Below is the prediction of survival on the test set with the first 25 records shown from the output.Thank you for reading and hope you learned something!
prediction <- data.frame(passenger_id,predict(XGB_model,test,type = "raw"))
head(prediction, n = 25)## PassengerId predict.XGB_model..test..type....raw..
## 1 892 No
## 2 893 No
## 3 894 No
## 4 895 No
## 5 896 Yes
## 6 897 No
## 7 898 Yes
## 8 899 No
## 9 900 Yes
## 10 901 No
## 11 902 No
## 12 903 No
## 13 904 Yes
## 14 905 No
## 15 906 Yes
## 16 907 Yes
## 17 908 No
## 18 909 No
## 19 910 Yes
## 20 911 No
## 21 912 No
## 22 913 Yes
## 23 914 Yes
## 24 915 No
## 25 916 YesGood Luck!
More reading: Kaggle