Using devices such as Jawbone Up, Nike FuelBand, and Fitbit it is now possible to collect a large amount of data about personal activity relatively inexpensively. These type of devices are part of the quantified self movement ??? a group of enthusiasts who take measurements about themselves regularly to improve their health, to find patterns in their behavior, or because they are tech geeks. One thing that people regularly do is quantify how much of a particular activity they do, but they rarely quantify how well they do it. In this project, your goal will be to use data from accelerometers on the belt, forearm, arm, and dumbell of 6 participants. They were asked to perform barbell lifts correctly and incorrectly in 5 different ways. More information is available from the website here: http://groupware.les.inf.puc-rio.br/har (see the section on the Weight Lifting Exercise Dataset).
The training data for this project are available here: https://d396qusza40orc.cloudfront.net/predmachlearn/pml-training.csv
The test data are available here: https://d396qusza40orc.cloudfront.net/predmachlearn/pml-testing.csv
The data for this project come from this source: http://groupware.les.inf.puc-rio.br/har. If you use the document you create for this class for any purpose please cite them as they have been very generous in allowing their data to be used for this kind of assignment.
The goal of your project is to predict the manner in which they did the exercise. This is the “classe” variable in the training set. You may use any of the other variables to predict with. You should create a report describing how you built your model, how you used cross validation, what you think the expected out of sample error is, and why you made the choices you did. You will also use your prediction model to predict 20 different test cases.
set.seed(123) # set seed for reproducability
# load libraries
library(caret)## Loading required package: lattice## Loading required package: ggplot2# The caret package provides the ability for implementing 200+ machine learning algorithms.
# import data
train = read.csv(file="pml-training.csv", na.strings=c("NA", "#DIV/0!", ""))
test = read.csv(file="pml-testing.csv", na.strings=c("NA", "#DIV/0!", ""))
# clean data
train <- train[, colSums(is.na(train)) == 0] # removes all columns that are NA only
test <- test[, colSums(is.na(test)) == 0] # removes all columns that are NA only
train <- train[, -c(1:7)] # removes irrelevant columns ("X", "user_name", timestamps, etc)
test <- test[, -c(1:7)] # removes irrelevant columns ("X", "user_name", timestamps, etc)
table(train$classe)##
## A B C D E
## 5580 3797 3422 3216 3607# subset data for cross validation
# Approach:
# 1) Use the training set to split it into two subgroups: subtraining and subtest set
# 2) Build a model on the subtraining set
# 3) Evaluate performance on subtest set
subset <- createDataPartition(y = train$classe, p = 0.75, list=FALSE)
subtrain <- train[subset,] # used for preparing models
subtest <- train[-subset,] # used for evaluating the models performance on unseen data
dim(subtrain); dim(subtest)## [1] 14718 53## [1] 4904 53I am going to predict “classe” using a random forest (“rf”), decision tree (“rpart”), Boosting (“gbm”) and K nearest neighbours (“knn”) model.I chose these models because “classe” is a factor variable.
# Step 1: train model to predict "classe" variable with all other variables
# Step 2: predict "classe" variable on cross validation set
# Step 3: validate outcome (Confusion Matrix)
ModelFitRPART <- train(classe~., method="rpart", data=subtrain)
predRPART <- predict(ModelFitRPART,newdata=subtest)
confusionMatrix(predRPART, subtest$classe)## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 1262 378 418 356 144
## B 20 312 26 141 101
## C 107 259 411 307 248
## D 0 0 0 0 0
## E 6 0 0 0 408
##
## Overall Statistics
##
## Accuracy : 0.488
## 95% CI : (0.4739, 0.5021)
## No Information Rate : 0.2845
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.3307
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 0.9047 0.32877 0.48070 0.0000 0.45283
## Specificity 0.6307 0.92718 0.77254 1.0000 0.99850
## Pos Pred Value 0.4934 0.52000 0.30856 NaN 0.98551
## Neg Pred Value 0.9433 0.85200 0.87570 0.8361 0.89020
## Prevalence 0.2845 0.19352 0.17435 0.1639 0.18373
## Detection Rate 0.2573 0.06362 0.08381 0.0000 0.08320
## Detection Prevalence 0.5216 0.12235 0.27162 0.0000 0.08442
## Balanced Accuracy 0.7677 0.62797 0.62662 0.5000 0.72567ModelFitRF <- train(classe~., method="rf", data=subtrain, trControl=trainControl(method="cv"), number = 10)
predRF <- predict(ModelFitRF,newdata=subtest)
confusionMatrix(predRF, subtest$classe)## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 1394 2 0 0 0
## B 1 945 7 0 0
## C 0 2 848 16 0
## D 0 0 0 786 1
## E 0 0 0 2 900
##
## Overall Statistics
##
## Accuracy : 0.9937
## 95% CI : (0.991, 0.9957)
## No Information Rate : 0.2845
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.992
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 0.9993 0.9958 0.9918 0.9776 0.9989
## Specificity 0.9994 0.9980 0.9956 0.9998 0.9995
## Pos Pred Value 0.9986 0.9916 0.9792 0.9987 0.9978
## Neg Pred Value 0.9997 0.9990 0.9983 0.9956 0.9998
## Prevalence 0.2845 0.1935 0.1743 0.1639 0.1837
## Detection Rate 0.2843 0.1927 0.1729 0.1603 0.1835
## Detection Prevalence 0.2847 0.1943 0.1766 0.1605 0.1839
## Balanced Accuracy 0.9994 0.9969 0.9937 0.9887 0.9992ModelFitKNN <- train(classe~., method="knn", data=subtrain)
predKNN <- predict(ModelFitKNN,newdata=subtest)
confusionMatrix(predKNN, subtest$classe)## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 1342 40 16 11 18
## B 15 835 28 4 34
## C 11 35 784 72 28
## D 25 20 14 710 26
## E 2 19 13 7 795
##
## Overall Statistics
##
## Accuracy : 0.9107
## 95% CI : (0.9024, 0.9185)
## No Information Rate : 0.2845
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.887
## Mcnemar's Test P-Value : < 2.2e-16
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 0.9620 0.8799 0.9170 0.8831 0.8824
## Specificity 0.9758 0.9795 0.9639 0.9793 0.9898
## Pos Pred Value 0.9404 0.9116 0.8430 0.8931 0.9510
## Neg Pred Value 0.9848 0.9714 0.9821 0.9771 0.9739
## Prevalence 0.2845 0.1935 0.1743 0.1639 0.1837
## Detection Rate 0.2737 0.1703 0.1599 0.1448 0.1621
## Detection Prevalence 0.2910 0.1868 0.1896 0.1621 0.1705
## Balanced Accuracy 0.9689 0.9297 0.9405 0.9312 0.9361ModelFitGBM <- train(classe~., method="gbm", data=subtrain, verbose = F, trControl = trainControl(method = "cv", number = 10))
predGBM <- predict(ModelFitGBM,newdata=subtest)
confusionMatrix(predGBM, subtest$classe)## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 1373 32 0 0 4
## B 11 896 30 5 7
## C 6 17 813 31 4
## D 5 3 11 760 13
## E 0 1 1 8 873
##
## Overall Statistics
##
## Accuracy : 0.9615
## 95% CI : (0.9557, 0.9667)
## No Information Rate : 0.2845
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.9512
## Mcnemar's Test P-Value : 1.231e-06
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 0.9842 0.9442 0.9509 0.9453 0.9689
## Specificity 0.9897 0.9866 0.9857 0.9922 0.9975
## Pos Pred Value 0.9744 0.9442 0.9334 0.9596 0.9887
## Neg Pred Value 0.9937 0.9866 0.9896 0.9893 0.9930
## Prevalence 0.2845 0.1935 0.1743 0.1639 0.1837
## Detection Rate 0.2800 0.1827 0.1658 0.1550 0.1780
## Detection Prevalence 0.2873 0.1935 0.1776 0.1615 0.1801
## Balanced Accuracy 0.9870 0.9654 0.9683 0.9687 0.9832# For comparison here are all resulting accuracies on the subtest set:
ACC <- c(confusionMatrix(predRPART, subtest$classe)$overall[1],confusionMatrix(predRF, subtest$classe)$overall[1], confusionMatrix(predKNN, subtest$classe)$overall[1], confusionMatrix(predGBM, subtest$classe)$overall[1])
# RPART, RF, KNN, GBM
ACC## Accuracy Accuracy Accuracy Accuracy
## 0.4879690 0.9936786 0.9106852 0.9614600# Estimated out-of-sample error on the subtest set:
ERR <- as.numeric(c(1-confusionMatrix(predRPART, subtest$classe)$overall[1],1-confusionMatrix(predRF, subtest$classe)$overall[1], 1-confusionMatrix(predKNN, subtest$classe)$overall[1], 1-confusionMatrix(predGBM, subtest$classe)$overall[1]))
ERR## [1] 0.51203100 0.00632137 0.08931485 0.03853997Random Forest has the best results in predicting the class variable and will be chosen for predicting the classe variable on the test set
# Predicting classe variable for test data set using RF
test_predRF <- predict(ModelFitRF,newdata=test)
test_predRF## [1] B A B A A E D B A A B C B A E E A B B B
## Levels: A B C D E