Read in data

train <- read.csv("train_wk4.csv")
test <- read.csv("test_wk4.csv")

Check dataframe structure & remove unnecessary variables

str(train)
## 'data.frame':    19622 obs. of  160 variables:
##  $ X                       : int  1 2 3 4 5 6 7 8 9 10 ...
##  $ user_name               : chr  "carlitos" "carlitos" "carlitos" "carlitos" ...
##  $ raw_timestamp_part_1    : int  1323084231 1323084231 1323084231 1323084232 1323084232 1323084232 1323084232 1323084232 1323084232 1323084232 ...
##  $ raw_timestamp_part_2    : int  788290 808298 820366 120339 196328 304277 368296 440390 484323 484434 ...
##  $ cvtd_timestamp          : chr  "5/12/2011 11:23" "5/12/2011 11:23" "5/12/2011 11:23" "5/12/2011 11:23" ...
##  $ new_window              : chr  "no" "no" "no" "no" ...
##  $ num_window              : int  11 11 11 12 12 12 12 12 12 12 ...
##  $ roll_belt               : num  1.41 1.41 1.42 1.48 1.48 1.45 1.42 1.42 1.43 1.45 ...
##  $ pitch_belt              : num  8.07 8.07 8.07 8.05 8.07 8.06 8.09 8.13 8.16 8.17 ...
##  $ yaw_belt                : num  -94.4 -94.4 -94.4 -94.4 -94.4 -94.4 -94.4 -94.4 -94.4 -94.4 ...
##  $ total_accel_belt        : int  3 3 3 3 3 3 3 3 3 3 ...
##  $ kurtosis_roll_belt      : chr  "" "" "" "" ...
##  $ kurtosis_picth_belt     : chr  "" "" "" "" ...
##  $ kurtosis_yaw_belt       : chr  "" "" "" "" ...
##  $ skewness_roll_belt      : chr  "" "" "" "" ...
##  $ skewness_roll_belt.1    : chr  "" "" "" "" ...
##  $ skewness_yaw_belt       : chr  "" "" "" "" ...
##  $ max_roll_belt           : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ max_picth_belt          : int  NA NA NA NA NA NA NA NA NA NA ...
##  $ max_yaw_belt            : chr  "" "" "" "" ...
##  $ min_roll_belt           : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ min_pitch_belt          : int  NA NA NA NA NA NA NA NA NA NA ...
##  $ min_yaw_belt            : chr  "" "" "" "" ...
##  $ amplitude_roll_belt     : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ amplitude_pitch_belt    : int  NA NA NA NA NA NA NA NA NA NA ...
##  $ amplitude_yaw_belt      : chr  "" "" "" "" ...
##  $ var_total_accel_belt    : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ avg_roll_belt           : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ stddev_roll_belt        : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ var_roll_belt           : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ avg_pitch_belt          : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ stddev_pitch_belt       : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ var_pitch_belt          : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ avg_yaw_belt            : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ stddev_yaw_belt         : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ var_yaw_belt            : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ gyros_belt_x            : num  0 0.02 0 0.02 0.02 0.02 0.02 0.02 0.02 0.03 ...
##  $ gyros_belt_y            : num  0 0 0 0 0.02 0 0 0 0 0 ...
##  $ gyros_belt_z            : num  -0.02 -0.02 -0.02 -0.03 -0.02 -0.02 -0.02 -0.02 -0.02 0 ...
##  $ accel_belt_x            : int  -21 -22 -20 -22 -21 -21 -22 -22 -20 -21 ...
##  $ accel_belt_y            : int  4 4 5 3 2 4 3 4 2 4 ...
##  $ accel_belt_z            : int  22 22 23 21 24 21 21 21 24 22 ...
##  $ magnet_belt_x           : int  -3 -7 -2 -6 -6 0 -4 -2 1 -3 ...
##  $ magnet_belt_y           : int  599 608 600 604 600 603 599 603 602 609 ...
##  $ magnet_belt_z           : int  -313 -311 -305 -310 -302 -312 -311 -313 -312 -308 ...
##  $ roll_arm                : num  -128 -128 -128 -128 -128 -128 -128 -128 -128 -128 ...
##  $ pitch_arm               : num  22.5 22.5 22.5 22.1 22.1 22 21.9 21.8 21.7 21.6 ...
##  $ yaw_arm                 : num  -161 -161 -161 -161 -161 -161 -161 -161 -161 -161 ...
##  $ total_accel_arm         : int  34 34 34 34 34 34 34 34 34 34 ...
##  $ var_accel_arm           : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ avg_roll_arm            : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ stddev_roll_arm         : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ var_roll_arm            : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ avg_pitch_arm           : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ stddev_pitch_arm        : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ var_pitch_arm           : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ avg_yaw_arm             : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ stddev_yaw_arm          : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ var_yaw_arm             : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ gyros_arm_x             : num  0 0.02 0.02 0.02 0 0.02 0 0.02 0.02 0.02 ...
##  $ gyros_arm_y             : num  0 -0.02 -0.02 -0.03 -0.03 -0.03 -0.03 -0.02 -0.03 -0.03 ...
##  $ gyros_arm_z             : num  -0.02 -0.02 -0.02 0.02 0 0 0 0 -0.02 -0.02 ...
##  $ accel_arm_x             : int  -288 -290 -289 -289 -289 -289 -289 -289 -288 -288 ...
##  $ accel_arm_y             : int  109 110 110 111 111 111 111 111 109 110 ...
##  $ accel_arm_z             : int  -123 -125 -126 -123 -123 -122 -125 -124 -122 -124 ...
##  $ magnet_arm_x            : int  -368 -369 -368 -372 -374 -369 -373 -372 -369 -376 ...
##  $ magnet_arm_y            : int  337 337 344 344 337 342 336 338 341 334 ...
##  $ magnet_arm_z            : int  516 513 513 512 506 513 509 510 518 516 ...
##  $ kurtosis_roll_arm       : chr  "" "" "" "" ...
##  $ kurtosis_picth_arm      : chr  "" "" "" "" ...
##  $ kurtosis_yaw_arm        : chr  "" "" "" "" ...
##  $ skewness_roll_arm       : chr  "" "" "" "" ...
##  $ skewness_pitch_arm      : chr  "" "" "" "" ...
##  $ skewness_yaw_arm        : chr  "" "" "" "" ...
##  $ max_roll_arm            : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ max_picth_arm           : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ max_yaw_arm             : int  NA NA NA NA NA NA NA NA NA NA ...
##  $ min_roll_arm            : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ min_pitch_arm           : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ min_yaw_arm             : int  NA NA NA NA NA NA NA NA NA NA ...
##  $ amplitude_roll_arm      : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ amplitude_pitch_arm     : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ amplitude_yaw_arm       : int  NA NA NA NA NA NA NA NA NA NA ...
##  $ roll_dumbbell           : num  13.1 13.1 12.9 13.4 13.4 ...
##  $ pitch_dumbbell          : num  -70.5 -70.6 -70.3 -70.4 -70.4 ...
##  $ yaw_dumbbell            : num  -84.9 -84.7 -85.1 -84.9 -84.9 ...
##  $ kurtosis_roll_dumbbell  : chr  "" "" "" "" ...
##  $ kurtosis_picth_dumbbell : chr  "" "" "" "" ...
##  $ kurtosis_yaw_dumbbell   : chr  "" "" "" "" ...
##  $ skewness_roll_dumbbell  : chr  "" "" "" "" ...
##  $ skewness_pitch_dumbbell : chr  "" "" "" "" ...
##  $ skewness_yaw_dumbbell   : chr  "" "" "" "" ...
##  $ max_roll_dumbbell       : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ max_picth_dumbbell      : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ max_yaw_dumbbell        : chr  "" "" "" "" ...
##  $ min_roll_dumbbell       : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ min_pitch_dumbbell      : num  NA NA NA NA NA NA NA NA NA NA ...
##  $ min_yaw_dumbbell        : chr  "" "" "" "" ...
##  $ amplitude_roll_dumbbell : num  NA NA NA NA NA NA NA NA NA NA ...
##   [list output truncated]
train <- train[,-c(1:7)]
test <- test[,-c(1:7)]

Convert classe to a factor variable

train$classe <- as.factor(train$classe)

Remove columns with missing data

remove <- c("kurtosis", "skewness", "yaw")
cols_to_remove <- colnames(train)[grep(paste(remove, collapse = "|"),
                                       colnames(train))]
train <- train[,!(colnames(train) %in% cols_to_remove)]
test <- test[,!(colnames(test) %in% cols_to_remove)]
train <- train[,colSums(is.na(train))==0]
test <- test[,colSums(is.na(test))==0]

Plot classe variable

library(ggplot2)
ggplot(train, aes(x = classe))+
  geom_histogram(stat = "count", col = "black", fill = "purple")+
  theme_classic()
## Warning in geom_histogram(stat = "count", col = "black", fill = "purple"):
## Ignoring unknown parameters: `binwidth`, `bins`, and `pad`

Build a tree classification model

library(tree)
## Warning: package 'tree' was built under R version 4.3.2
model_tree <- tree(classe~., data = train, method = "class")
summary(model_tree)
## 
## Classification tree:
## tree(formula = classe ~ ., data = train, method = "class")
## Variables actually used in tree construction:
##  [1] "roll_belt"            "pitch_forearm"        "roll_forearm"        
##  [4] "magnet_dumbbell_x"    "magnet_dumbbell_y"    "magnet_dumbbell_z"   
##  [7] "pitch_belt"           "accel_dumbbell_z"     "accel_dumbbell_x"    
## [10] "total_accel_dumbbell" "accel_forearm_x"     
## Number of terminal nodes:  18 
## Residual mean deviance:  1.689 = 33100 / 19600 
## Misclassification error rate: 0.3322 = 6518 / 19622

Plot the tree model

plot(model_tree)
text(model_tree, pretty = 0, cex = 0.5)

Predict on training data

library(caret)
## Warning: package 'caret' was built under R version 4.3.2
## Loading required package: lattice
set.seed(123)
pred_tree <- predict(model_tree, train, type = "class")
result_train_tree <- confusionMatrix(pred_tree, train$classe)
result_train_tree$overall[[1]]
## [1] 0.6678218
result_train_tree$overall[[3]]
## [1] 0.6611817
result_train_tree$overall[[4]]
## [1] 0.6744122

The 95% confidence interval for the accuracy of the tree model spans 0.66 to 0.67, this gives us some idea of the out of sample error rate.

Perform cross-validation

cv_tree <- cv.tree(model_tree, FUN = prune.misclass)
cv_tree
## $size
##  [1] 18 17 16 15 14 13 11  8  7  6  2  1
## 
## $dev
##  [1]  6597  6597  6826  6833  7118  7306  8517  8725  9124  9692 12429 14042
## 
## $k
##  [1]      -Inf    0.0000  133.0000  145.0000  200.0000  227.0000  280.0000
##  [8]  296.3333  431.0000  575.0000  686.7500 1617.0000
## 
## $method
## [1] "misclass"
## 
## attr(,"class")
## [1] "prune"         "tree.sequence"

Prune tree

After performing cross validation, we can see the 17 leaf tree is the best performing model. We will prune the tree to include this number of nodes.

prune_model_tree <- prune.misclass(model_tree, best = 17)

Predict on the training data again

Let’s see if pruning the tree improves accuracy

install.packages("caret")
## Warning: package 'caret' is in use and will not be installed
library(caret)
pred <- predict(prune_model_tree, train, type = "class")
pred_result <- confusionMatrix(pred, train$classe)
pred_result$overall[[1]]
## [1] 0.6678218

The accuracy is similar between the initial model and the pruned model.

Next we’ll build a random forest model.

Random forest model

when bagging random forest models we typically want to use a subset of predictors to build each tree, the square root of the number of predictors is usually used to build each tree, so we will set mtry = 7.

library(randomForest)
## Warning: package 'randomForest' was built under R version 4.3.2
## randomForest 4.7-1.1
## Type rfNews() to see new features/changes/bug fixes.
## 
## Attaching package: 'randomForest'
## The following object is masked from 'package:ggplot2':
## 
##     margin
set.seed(123)
model_rf <- randomForest(classe~., data = train, 
                         mtry = 7, importance = T)

Predict using the random forest model

As the trees in a random forest model are fit to bootstrapped subsets of predictors, there is no need to perform cross-validation on our random forest model

library(caret)
pred <- predict(model_rf, train, type = "class")
result <- confusionMatrix(pred, train$classe)
result$overall[[1]]
## [1] 1
result$overall[[3]]
## [1] 0.999812
result$overall[[4]]
## [1] 1

As we can see the confidence interval around the accuracy when predicting using the random forest model ranges from 0.99 to 1.00, resulting in better performance of this model compared with the classification tree model.

Lets predict the test cases

pred_test <- predict(model_rf, test)
pred_test
##  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 
##  B  A  B  A  A  E  D  B  A  A  B  C  B  A  E  E  A  B  B  B 
## Levels: A B C D E

Final verdict

We can clearly see the random forest model (Accuracy = 0.99 to 1.00) performed better than the classification tree model (Accuracy = 0.66 to 0.67). We performed cross validation across a range of k’s when building the tree model, the ifnal model result in 16 node’s. Given each tree within the random forest model is built using a subset of predictors, we do not need to perform cross-validation to estimate the error rate. Using the random fores model we would expect, in the worst case, our out of sample error rate to be 0.001 (0.1%). Given we are predicting 20 cases in the test data set, we would be confiden all of these cases have been predicted correctly.