CSUEB STATS 6620 - Machine Learning with R - Spring 2017 - Prof E. Suess

Gui Larangeira - HW 6 - Random Forests - 05/22/2017

We tried applying RFs to the Credit data from ISLR book, but for some reason the algorithm did not converge. So we are using the same German Credit data once more. We do a 70/30 split between training and testing.

Step 3 Training a simple Random Forest:

model <- randomForest(default ~ . , data = train, ntree=1000, mtry=5)
model <- randomForest(default ~ . , data = train, ntree=1000, mtry=5)
model

Call:
 randomForest(formula = default ~ ., data = train, ntree = 1000,      mtry = 5) 
               Type of random forest: classification
                     Number of trees: 1000
No. of variables tried at each split: 5

        OOB estimate of  error rate: 24.43%
Confusion matrix:
     no yes class.error
no  450  41  0.08350305
yes 130  79  0.62200957

Step 4 - Evaluating Model Performance

After training the model, we evaluate it’s performance on the test data:

sum(pred==test$default) / nrow(test)

[1] 0.76

A accuracy of 0.76 leaves room for improvement so we follow on to step 5, boosting and tuning the RF (could be Radio Frequency but meaning Rand Forest!)

Step 5 - Improving Model Performance

We use Cross-Validation for tuning and C5.0 for boosting.

m_c50 <- train(default ~ ., data = train, method = "C5.0",
               metric = "Kappa", trControl = ctrl,
               tuneGrid = grid_c50)

<U+393C><U+3E31>!<U+393C><U+3E32> not meaningful for factors

And again we look to evaluate the performance of the trained model on the test data:

sum(pred==test$default) / nrow(test)

[1] 0.7233333

After a pretty lenghty number-crunching session, our results have actually worsened!