DATA 622 -Assignment 4:Final Project -Community Demographics and Crime Analysis

Random forests are one the most popular machine learning algorithms. They can be used for classification or regression, deal with a large number of features, and generally are so successful because they provide a good predictive performance, low incidence of over-fitting, and easy interpret-ability.

In creating the best random forest model, we want to minimize the OOB error rate by finding the optimal number of variables selected at each split, known as the mtry. The below code finds the optimal mtry to use in our random forest model.

train2 <- as.data.frame(train)
oob.err=double(101)
test.err=double(101)
for (mtry in 1:101){
  fit=randomForest(ViolentCrimesPerPop~.,data=train2,mtry=mtry,ntree=400, proximity=TRUE)
 oob.err[mtry]=fit$mse[400]
 pred=predict(fit, train2)
 test.err[mtry]=with(train2, mean((ViolentCrimesPerPop-pred)^2))
 cat(mtry," ")}

1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  

40  

41  

42  

43  

44  

45  

46  

47  

48  

49  

50  

51  

52  

53  

54  

55  

56  

57  

58  

59  

60  

61  

62  

63  

64  

65  

66  

67  

68  

69  

70  

71  

72  

73  

74  

75  

76  

77  

78  

79  

80  

81  

82  

83  

84  

85  

86  

87  

88  

89  

90  

91  

92  

93  

94  

95  

96  

97  

98  

99  

100  

101  

matplot(1:mtry,cbind(test.err, oob.err),pch=19, col=c("red", "blue"), type="b",ylab
="Mean Squared Error")
legend("topright",legend=c("00B", "Test"),pch=19,col=c("red", "blue"))

Ideally OOB error and test errors should line up - but in this case the test error seems to be higher than the OOB.

Variable Importance

varImpPlot(fit)

## Call Variable Importance Plot Function
printVarImportance("Random Forest", varImp(fit), 10)

The above graph illustrates the importance of the variables used to predict the Violent Crimes Per Population. The Mean Decrease Accuracy displays how much the model accuracy decreases if we drop the variable. Here, Percentage of kids born to never married is regarded as the most important variable by a wide margin. The Mean Decrease Gini graph displays the variable importance on the Gini impurity index used for splitting trees. Again, Percentage of kids born to never married is the clear leader.

print(fit)

Call:
 randomForest(formula = ViolentCrimesPerPop ~ ., data = train2,      mtry = mtry, ntree = 400, proximity = TRUE) 
               Type of random forest: regression
                     Number of trees: 400
No. of variables tried at each split: 39

          Mean of squared residuals: 0.01920442
                    % Var explained: 65.2

Prediction Performance

rfPredPerf <- postResample(pred = pred, obs = y_test)
rfPredPerf['Family'] <- 'Trees & Boosting'
rfPredPerf['Model'] <- 'Random Forest'

Model10: Gradient Boosting Method The boosting method is uses trees in a sequential pattern. The successive tree is developed using information from the previous tree to minimize its error. The boosting method has three tuning parameters, number of trees, shrinkage parameter, and number of splits in a tree.

We will be using the stochastic gradient boosting in our model. This approach resamples the observations and columns in each iteration. We’ll use the caret workflow, which invokes the xgboost package, to automatically adjust the model parameter values, and fit the final best boosted tree that explains the best our data.

Here, we can see our parameters that are used in the model -

Model Summary

modelSummaryDF <- rbind(ridgePredPerf, lassoPredPerf, elnetPredPerf, nnetPredPerf, knnPredPerf, svmRPredPerf, marsPredPerf, dtreePredPerf, rfPredPerf)

SummaryDF <- modelSummaryDF[, c(4, 5, 1, 2, 3)]



rownames(SummaryDF) <- NULL

SummaryDF %>% kable() %>% kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive")) %>% 
  scroll_box(width="100%",height="500px")