Peer-graded Assignment: Prediction Assignment Writeup
Gaspare Mattarella
9/5/2020
Introduction
This report is prepared as one of the requirement in Practical Machine Learning online course by Johns Hopkins University. The basic goal of this assignment is to predict the manner of the subject (6 participants) performed some exercise. For this assignment, in order to predict the manner of the subject did the exercise decision tree and random forest method will be performed to determine the best prediction. The best prediction is determined by the highest accuracy.
Background
Using devices such as Jawbone Up, Nike FuelBand, and Fitbit it is now possible to collect a large amount of data about personal activity relatively inexpensively. These type of devices are part of the quantified self movement ??? a group of enthusiasts who take measurements about themselves regularly to improve their health, to find patterns in their behavior, or because they are tech geeks. One thing that people regularly do is quantify how much of a particular activity they do, but they rarely quantify how well they do it. In this project, your goal will be to use data from accelerometers on the belt, forearm, arm, and dumbell of 6 participants. They were asked to perform barbell lifts correctly and incorrectly in 5 different ways. More information is available from the website here
Pre-processing
This segment includes packages required and uploading data into r.
Packages needed to perform tree classification, bagging, random forest are as follows:
library(caret)
library(readr)
library(RColorBrewer)
library(RGtk2)
library(rpart)
library(rpart.plot)
library(rattle)
library(randomForest)
library(gbm)Data Cleaning
Many columns of the data set contain the same value accros the lines. These “near-zero variance predictors”" bring almost no information to our model and will make computing unnecessarily longer. Others are entirely filled with NA values. Finnaly, the six first variables do not concern fitness motions whatsoever. They also need to be remove before we start fitting our model.
w <- sapply(training_pml,function(x) mean(is.na(x)) > 0.95)
b <- sapply(testing_pml,function(x) mean(is.na(x)) > 0.95)
training_pml <- training_pml[,w == F]
testing_pml <- testing_pml[,w == F]inTrain <- createDataPartition(y= training_pml$classe, p = .65, list =F)
train <- training_pml[inTrain,]
test <- training_pml[-inTrain,]train <- train[-c(1:7)]
test <- test[-c(1:7)]The first model we’re goning to fit is a simple Tree Classification:
tree <- train(factor(classe)~ ., method= "rpart", data = train,
metric = "Accuracy",
preProcess=c("center", "scale"),
trControl=trainControl(method = "cv"
, number = 4
, p= 0.60
, allowParallel = TRUE))
tree## CART
##
## 12757 samples
## 51 predictor
## 5 classes: 'A', 'B', 'C', 'D', 'E'
##
## Pre-processing: centered (51), scaled (51)
## Resampling: Cross-Validated (4 fold)
## Summary of sample sizes: 9566, 9569, 9568, 9568
## Resampling results across tuning parameters:
##
## cp Accuracy Kappa
## 0.03088719 0.4878902 0.3303340
## 0.03132530 0.4878902 0.3303340
## 0.06243154 0.4172581 0.2208464
##
## Accuracy was used to select the optimal model using the largest value.
## The final value used for the model was cp = 0.0313253.fancyRpartPlot(tree$finalModel)
As we can see the accuricy is the model is just 51% which makes evident this is not a proper model to fit this kind of data.
The next models we’re going to fit are the “bagging” and the random forest:
baggi <- train(factor(classe)~ ., method= "treebag", data = train,
metric = "Accuracy",
preProcess=c("center", "scale"),
trControl=trainControl(method = "cv"
, number = 4
, p= 0.60
, allowParallel = TRUE))
baggi## Bagged CART
##
## 12757 samples
## 51 predictor
## 5 classes: 'A', 'B', 'C', 'D', 'E'
##
## Pre-processing: centered (51), scaled (51)
## Resampling: Cross-Validated (4 fold)
## Summary of sample sizes: 9567, 9568, 9569, 9567
## Resampling results:
##
## Accuracy Kappa
## 0.981187 0.9762009bag_pred <- predict(baggi, test)
confusionMatrix(bag_pred,factor(test$classe))## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 1944 22 0 4 0
## B 8 1293 18 0 10
## C 1 9 1163 22 7
## D 0 0 14 1099 7
## E 0 4 2 0 1238
##
## Overall Statistics
##
## Accuracy : 0.9814
## 95% CI : (0.9779, 0.9844)
## No Information Rate : 0.2845
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.9764
##
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 0.9954 0.9736 0.9716 0.9769 0.9810
## Specificity 0.9947 0.9935 0.9931 0.9963 0.9989
## Pos Pred Value 0.9868 0.9729 0.9676 0.9813 0.9952
## Neg Pred Value 0.9982 0.9937 0.9940 0.9955 0.9957
## Prevalence 0.2845 0.1934 0.1744 0.1639 0.1838
## Detection Rate 0.2832 0.1883 0.1694 0.1601 0.1803
## Detection Prevalence 0.2870 0.1936 0.1751 0.1631 0.1812
## Balanced Accuracy 0.9950 0.9836 0.9824 0.9866 0.9900Random_Forest <- randomForest(factor(classe) ~ .,
data = train,
ntree=100
, metric = "Accuracy"
, preProcess=c("center", "scale")
, trControl=trainControl(method = "cv"
, number = 4
, p= 0.60
, allowParallel = TRUE))Both the models show much higher degrees of accuracy in the cross validation tests, both close to \(100\%\).
plot(Random_Forest)
rf_pred <- predict(Random_Forest, test)
confusionMatrix(rf_pred,factor(test$classe))## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 1952 9 0 0 0
## B 1 1314 18 0 0
## C 0 5 1179 25 2
## D 0 0 0 1099 6
## E 0 0 0 1 1254
##
## Overall Statistics
##
## Accuracy : 0.9902
## 95% CI : (0.9876, 0.9924)
## No Information Rate : 0.2845
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.9877
##
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 0.9995 0.9895 0.9850 0.9769 0.9937
## Specificity 0.9982 0.9966 0.9944 0.9990 0.9998
## Pos Pred Value 0.9954 0.9857 0.9736 0.9946 0.9992
## Neg Pred Value 0.9998 0.9975 0.9968 0.9955 0.9986
## Prevalence 0.2845 0.1934 0.1744 0.1639 0.1838
## Detection Rate 0.2843 0.1914 0.1717 0.1601 0.1827
## Detection Prevalence 0.2857 0.1942 0.1764 0.1610 0.1828
## Balanced Accuracy 0.9988 0.9930 0.9897 0.9879 0.9967Testing the model
We already applied to the final test set the same features selection method that we used for the training set so we can now directly test on the new data:
print(predict(Random_Forest, newdata=testing_pml))## 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
## B A B A A E D B A A B C B A E E A B B B
## Levels: A B C D E