train <- read.csv("train_wk4.csv")
test <- read.csv("test_wk4.csv")
str(train)
## 'data.frame': 19622 obs. of 160 variables:
## $ X : int 1 2 3 4 5 6 7 8 9 10 ...
## $ user_name : chr "carlitos" "carlitos" "carlitos" "carlitos" ...
## $ raw_timestamp_part_1 : int 1323084231 1323084231 1323084231 1323084232 1323084232 1323084232 1323084232 1323084232 1323084232 1323084232 ...
## $ raw_timestamp_part_2 : int 788290 808298 820366 120339 196328 304277 368296 440390 484323 484434 ...
## $ cvtd_timestamp : chr "5/12/2011 11:23" "5/12/2011 11:23" "5/12/2011 11:23" "5/12/2011 11:23" ...
## $ new_window : chr "no" "no" "no" "no" ...
## $ num_window : int 11 11 11 12 12 12 12 12 12 12 ...
## $ roll_belt : num 1.41 1.41 1.42 1.48 1.48 1.45 1.42 1.42 1.43 1.45 ...
## $ pitch_belt : num 8.07 8.07 8.07 8.05 8.07 8.06 8.09 8.13 8.16 8.17 ...
## $ yaw_belt : num -94.4 -94.4 -94.4 -94.4 -94.4 -94.4 -94.4 -94.4 -94.4 -94.4 ...
## $ total_accel_belt : int 3 3 3 3 3 3 3 3 3 3 ...
## $ kurtosis_roll_belt : chr "" "" "" "" ...
## $ kurtosis_picth_belt : chr "" "" "" "" ...
## $ kurtosis_yaw_belt : chr "" "" "" "" ...
## $ skewness_roll_belt : chr "" "" "" "" ...
## $ skewness_roll_belt.1 : chr "" "" "" "" ...
## $ skewness_yaw_belt : chr "" "" "" "" ...
## $ max_roll_belt : num NA NA NA NA NA NA NA NA NA NA ...
## $ max_picth_belt : int NA NA NA NA NA NA NA NA NA NA ...
## $ max_yaw_belt : chr "" "" "" "" ...
## $ min_roll_belt : num NA NA NA NA NA NA NA NA NA NA ...
## $ min_pitch_belt : int NA NA NA NA NA NA NA NA NA NA ...
## $ min_yaw_belt : chr "" "" "" "" ...
## $ amplitude_roll_belt : num NA NA NA NA NA NA NA NA NA NA ...
## $ amplitude_pitch_belt : int NA NA NA NA NA NA NA NA NA NA ...
## $ amplitude_yaw_belt : chr "" "" "" "" ...
## $ var_total_accel_belt : num NA NA NA NA NA NA NA NA NA NA ...
## $ avg_roll_belt : num NA NA NA NA NA NA NA NA NA NA ...
## $ stddev_roll_belt : num NA NA NA NA NA NA NA NA NA NA ...
## $ var_roll_belt : num NA NA NA NA NA NA NA NA NA NA ...
## $ avg_pitch_belt : num NA NA NA NA NA NA NA NA NA NA ...
## $ stddev_pitch_belt : num NA NA NA NA NA NA NA NA NA NA ...
## $ var_pitch_belt : num NA NA NA NA NA NA NA NA NA NA ...
## $ avg_yaw_belt : num NA NA NA NA NA NA NA NA NA NA ...
## $ stddev_yaw_belt : num NA NA NA NA NA NA NA NA NA NA ...
## $ var_yaw_belt : num NA NA NA NA NA NA NA NA NA NA ...
## $ gyros_belt_x : num 0 0.02 0 0.02 0.02 0.02 0.02 0.02 0.02 0.03 ...
## $ gyros_belt_y : num 0 0 0 0 0.02 0 0 0 0 0 ...
## $ gyros_belt_z : num -0.02 -0.02 -0.02 -0.03 -0.02 -0.02 -0.02 -0.02 -0.02 0 ...
## $ accel_belt_x : int -21 -22 -20 -22 -21 -21 -22 -22 -20 -21 ...
## $ accel_belt_y : int 4 4 5 3 2 4 3 4 2 4 ...
## $ accel_belt_z : int 22 22 23 21 24 21 21 21 24 22 ...
## $ magnet_belt_x : int -3 -7 -2 -6 -6 0 -4 -2 1 -3 ...
## $ magnet_belt_y : int 599 608 600 604 600 603 599 603 602 609 ...
## $ magnet_belt_z : int -313 -311 -305 -310 -302 -312 -311 -313 -312 -308 ...
## $ roll_arm : num -128 -128 -128 -128 -128 -128 -128 -128 -128 -128 ...
## $ pitch_arm : num 22.5 22.5 22.5 22.1 22.1 22 21.9 21.8 21.7 21.6 ...
## $ yaw_arm : num -161 -161 -161 -161 -161 -161 -161 -161 -161 -161 ...
## $ total_accel_arm : int 34 34 34 34 34 34 34 34 34 34 ...
## $ var_accel_arm : num NA NA NA NA NA NA NA NA NA NA ...
## $ avg_roll_arm : num NA NA NA NA NA NA NA NA NA NA ...
## $ stddev_roll_arm : num NA NA NA NA NA NA NA NA NA NA ...
## $ var_roll_arm : num NA NA NA NA NA NA NA NA NA NA ...
## $ avg_pitch_arm : num NA NA NA NA NA NA NA NA NA NA ...
## $ stddev_pitch_arm : num NA NA NA NA NA NA NA NA NA NA ...
## $ var_pitch_arm : num NA NA NA NA NA NA NA NA NA NA ...
## $ avg_yaw_arm : num NA NA NA NA NA NA NA NA NA NA ...
## $ stddev_yaw_arm : num NA NA NA NA NA NA NA NA NA NA ...
## $ var_yaw_arm : num NA NA NA NA NA NA NA NA NA NA ...
## $ gyros_arm_x : num 0 0.02 0.02 0.02 0 0.02 0 0.02 0.02 0.02 ...
## $ gyros_arm_y : num 0 -0.02 -0.02 -0.03 -0.03 -0.03 -0.03 -0.02 -0.03 -0.03 ...
## $ gyros_arm_z : num -0.02 -0.02 -0.02 0.02 0 0 0 0 -0.02 -0.02 ...
## $ accel_arm_x : int -288 -290 -289 -289 -289 -289 -289 -289 -288 -288 ...
## $ accel_arm_y : int 109 110 110 111 111 111 111 111 109 110 ...
## $ accel_arm_z : int -123 -125 -126 -123 -123 -122 -125 -124 -122 -124 ...
## $ magnet_arm_x : int -368 -369 -368 -372 -374 -369 -373 -372 -369 -376 ...
## $ magnet_arm_y : int 337 337 344 344 337 342 336 338 341 334 ...
## $ magnet_arm_z : int 516 513 513 512 506 513 509 510 518 516 ...
## $ kurtosis_roll_arm : chr "" "" "" "" ...
## $ kurtosis_picth_arm : chr "" "" "" "" ...
## $ kurtosis_yaw_arm : chr "" "" "" "" ...
## $ skewness_roll_arm : chr "" "" "" "" ...
## $ skewness_pitch_arm : chr "" "" "" "" ...
## $ skewness_yaw_arm : chr "" "" "" "" ...
## $ max_roll_arm : num NA NA NA NA NA NA NA NA NA NA ...
## $ max_picth_arm : num NA NA NA NA NA NA NA NA NA NA ...
## $ max_yaw_arm : int NA NA NA NA NA NA NA NA NA NA ...
## $ min_roll_arm : num NA NA NA NA NA NA NA NA NA NA ...
## $ min_pitch_arm : num NA NA NA NA NA NA NA NA NA NA ...
## $ min_yaw_arm : int NA NA NA NA NA NA NA NA NA NA ...
## $ amplitude_roll_arm : num NA NA NA NA NA NA NA NA NA NA ...
## $ amplitude_pitch_arm : num NA NA NA NA NA NA NA NA NA NA ...
## $ amplitude_yaw_arm : int NA NA NA NA NA NA NA NA NA NA ...
## $ roll_dumbbell : num 13.1 13.1 12.9 13.4 13.4 ...
## $ pitch_dumbbell : num -70.5 -70.6 -70.3 -70.4 -70.4 ...
## $ yaw_dumbbell : num -84.9 -84.7 -85.1 -84.9 -84.9 ...
## $ kurtosis_roll_dumbbell : chr "" "" "" "" ...
## $ kurtosis_picth_dumbbell : chr "" "" "" "" ...
## $ kurtosis_yaw_dumbbell : chr "" "" "" "" ...
## $ skewness_roll_dumbbell : chr "" "" "" "" ...
## $ skewness_pitch_dumbbell : chr "" "" "" "" ...
## $ skewness_yaw_dumbbell : chr "" "" "" "" ...
## $ max_roll_dumbbell : num NA NA NA NA NA NA NA NA NA NA ...
## $ max_picth_dumbbell : num NA NA NA NA NA NA NA NA NA NA ...
## $ max_yaw_dumbbell : chr "" "" "" "" ...
## $ min_roll_dumbbell : num NA NA NA NA NA NA NA NA NA NA ...
## $ min_pitch_dumbbell : num NA NA NA NA NA NA NA NA NA NA ...
## $ min_yaw_dumbbell : chr "" "" "" "" ...
## $ amplitude_roll_dumbbell : num NA NA NA NA NA NA NA NA NA NA ...
## [list output truncated]
train <- train[,-c(1:7)]
test <- test[,-c(1:7)]
train$classe <- as.factor(train$classe)
remove <- c("kurtosis", "skewness", "yaw")
cols_to_remove <- colnames(train)[grep(paste(remove, collapse = "|"),
colnames(train))]
train <- train[,!(colnames(train) %in% cols_to_remove)]
test <- test[,!(colnames(test) %in% cols_to_remove)]
train <- train[,colSums(is.na(train))==0]
test <- test[,colSums(is.na(test))==0]
library(ggplot2)
ggplot(train, aes(x = classe))+
geom_histogram(stat = "count", col = "black", fill = "purple")+
theme_classic()
## Warning in geom_histogram(stat = "count", col = "black", fill = "purple"):
## Ignoring unknown parameters: `binwidth`, `bins`, and `pad`
library(tree)
## Warning: package 'tree' was built under R version 4.3.2
model_tree <- tree(classe~., data = train, method = "class")
summary(model_tree)
##
## Classification tree:
## tree(formula = classe ~ ., data = train, method = "class")
## Variables actually used in tree construction:
## [1] "roll_belt" "pitch_forearm" "roll_forearm"
## [4] "magnet_dumbbell_x" "magnet_dumbbell_y" "magnet_dumbbell_z"
## [7] "pitch_belt" "accel_dumbbell_z" "accel_dumbbell_x"
## [10] "total_accel_dumbbell" "accel_forearm_x"
## Number of terminal nodes: 18
## Residual mean deviance: 1.689 = 33100 / 19600
## Misclassification error rate: 0.3322 = 6518 / 19622
plot(model_tree)
text(model_tree, pretty = 0, cex = 0.5)
library(caret)
## Warning: package 'caret' was built under R version 4.3.2
## Loading required package: lattice
set.seed(123)
pred_tree <- predict(model_tree, train, type = "class")
result_train_tree <- confusionMatrix(pred_tree, train$classe)
result_train_tree$overall[[1]]
## [1] 0.6678218
result_train_tree$overall[[3]]
## [1] 0.6611817
result_train_tree$overall[[4]]
## [1] 0.6744122
The 95% confidence interval for the accuracy of the tree model spans 0.66 to 0.67, this gives us some idea of the out of sample error rate.
cv_tree <- cv.tree(model_tree, FUN = prune.misclass)
cv_tree
## $size
## [1] 18 17 16 15 14 13 11 8 7 6 2 1
##
## $dev
## [1] 6597 6597 6826 6833 7118 7306 8517 8725 9124 9692 12429 14042
##
## $k
## [1] -Inf 0.0000 133.0000 145.0000 200.0000 227.0000 280.0000
## [8] 296.3333 431.0000 575.0000 686.7500 1617.0000
##
## $method
## [1] "misclass"
##
## attr(,"class")
## [1] "prune" "tree.sequence"
After performing cross validation, we can see the 17 leaf tree is the best performing model. We will prune the tree to include this number of nodes.
prune_model_tree <- prune.misclass(model_tree, best = 17)
Let’s see if pruning the tree improves accuracy
install.packages("caret")
## Warning: package 'caret' is in use and will not be installed
library(caret)
pred <- predict(prune_model_tree, train, type = "class")
pred_result <- confusionMatrix(pred, train$classe)
pred_result$overall[[1]]
## [1] 0.6678218
The accuracy is similar between the initial model and the pruned model.
Next we’ll build a random forest model.
when bagging random forest models we typically want to use a subset of predictors to build each tree, the square root of the number of predictors is usually used to build each tree, so we will set mtry = 7.
library(randomForest)
## Warning: package 'randomForest' was built under R version 4.3.2
## randomForest 4.7-1.1
## Type rfNews() to see new features/changes/bug fixes.
##
## Attaching package: 'randomForest'
## The following object is masked from 'package:ggplot2':
##
## margin
set.seed(123)
model_rf <- randomForest(classe~., data = train,
mtry = 7, importance = T)
As the trees in a random forest model are fit to bootstrapped subsets of predictors, there is no need to perform cross-validation on our random forest model
library(caret)
pred <- predict(model_rf, train, type = "class")
result <- confusionMatrix(pred, train$classe)
result$overall[[1]]
## [1] 1
result$overall[[3]]
## [1] 0.999812
result$overall[[4]]
## [1] 1
As we can see the confidence interval around the accuracy when predicting using the random forest model ranges from 0.99 to 1.00, resulting in better performance of this model compared with the classification tree model.
pred_test <- predict(model_rf, test)
pred_test
## 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
## B A B A A E D B A A B C B A E E A B B B
## Levels: A B C D E
We can clearly see the random forest model (Accuracy = 0.99 to 1.00) performed better than the classification tree model (Accuracy = 0.66 to 0.67). We performed cross validation across a range of k’s when building the tree model, the ifnal model result in 16 node’s. Given each tree within the random forest model is built using a subset of predictors, we do not need to perform cross-validation to estimate the error rate. Using the random fores model we would expect, in the worst case, our out of sample error rate to be 0.001 (0.1%). Given we are predicting 20 cases in the test data set, we would be confiden all of these cases have been predicted correctly.