Practical Machine Learning Course Project
Gita Guru
20 February 2015
Background
Using devices such as Jawbone Up, Nike FuelBand, and Fitbit it is now possible to collect a large amount of data about personal activity relatively inexpensively. These type of devices are part of the quantified self movement a group of enthusiasts who take measurements about themselves regularly to improve their health, to find patterns in their behavior, or because they are tech geeks. One thing that people regularly do is quantify how much of a particular activity they do, but they rarely quantify how well they do it. In this project, our goal will be to use data from accelerometers on the belt, forearm, arm, and dumbell of 6 participants. They were asked to perform barbell lifts correctly and incorrectly in 5 different ways. More information is available from the website here (see the section on the Weight Lifting Exercise Dataset).
Data
The training data for this project are available here:
https://d396qusza40orc.cloudfront.net/predmachlearn/pml-training.csv
The test data are available here:
https://d396qusza40orc.cloudfront.net/predmachlearn/pml-testing.csv
The data for this project come from this source: http://groupware.les.inf.puc-rio.br/har. If you use the document you create for this class for any purpose please cite them as they have been very generous in allowing their data to be used for this kind of assignment.
The Project
The goal of your project is to predict the manner in which they did the exercise. This is the “classe” variable in the training set. You may use any of the other variables to predict with. You should create a report describing how you built your model, how you used cross validation, what you think the expected out of sample error is, and why you made the choices you did. You will also use your prediction model to predict 20 different test cases.
Assignment:
Your submission should consist of a link to a Github repo with your R markdown and compiled HTML file describing your analysis. Please constrain the text of the writeup to < 2000 words and the number of figures to be less than 5. It will make it easier for the graders if you submit a repo with a gh-pages branch so the HTML page can be viewed online.
You should also apply your machine learning algorithm to the 20 test cases available in the test data above. Please submit your predictions in appropriate format to the programming assignment for automated grading. See the programming assignment for additional details.
Data Processing
Installing packages, loading libraries, and setting the seed for reproduceability:
echo = TRUE # Always make code visible
library(caret)## Loading required package: lattice
## Loading required package: ggplot2library(randomForest) #Random forest for classification and regression## randomForest 4.6-10
## Type rfNews() to see new features/changes/bug fixes.library(rpart) # Regressive Partitioning and Regression trees
library(rpart.plot) # Decision Tree plot
# setting the overall seed for reproduceability
set.seed(1234)Download Data
First we read the generated csv file. If the data already exists in the working environment, we do not need to load it again. Otherwise, we read the csv file.
if (!"trainFullData" %in% ls()) {
print("Loading training data")
trainFullData <- read.csv("data/pml-training.csv", sep = ",")
}## [1] "Loading training data"dim(trainFullData)## [1] 19622 160if (!"testFullData" %in% ls()) {
print("Loading testing data")
testFullData <- read.csv("data/pml-testing.csv", sep = ",")
}## [1] "Loading testing data"dim(testFullData)## [1] 20 160Clean the data
We remove the columns which has values just NA from both the datasets. Then we remove the columns which are not necessary for our prediction model.
sum(complete.cases(trainFullData))## [1] 406traindt <- trainFullData[, colSums(is.na(trainFullData)) == 0]
testdt <- testFullData[, colSums(is.na(testFullData)) == 0]
dim(traindt)## [1] 19622 93dim(testdt)## [1] 20 60# # Some variables are irrelevant to our current project: user_name, raw_timestamp_part_1, raw_timestamp_part_,2 cvtd_timestamp, new_window, and num_window (columns 1 to 7). We can delete these variables
classe <- traindt$classe
trainRemove <- grepl("^X|timestamp|window", names(traindt))
traindt <- traindt[, !trainRemove]
trainCleaned <- traindt[, sapply(traindt, is.numeric)]
trainCleaned$classe <- classe
testRemove <- grepl("^X|timestamp|window", names(testdt))
testdt <- testdt[, !testRemove]
testCleaned <- testdt[, sapply(testdt, is.numeric)]
dim(trainCleaned) ##This is the final training data## [1] 19622 53dim(testCleaned) ##This is the final Testing data## [1] 20 53So, finally we have train data has 19622 observations and 53 variables. And, test data has 20 observations and 53 variables.
Prediction modeling
Now we will try to find a better fitting prediction model with the cleaned training data available.
For that we need to partition it further as train and test data for prediction.
In order to perform cross-validation, the training data set is partionned into 2 sets: subTraining (70%) and subTest (30%). This will be performed using random subsampling without replacement.
set.seed(23442) # For reproducibile purpose
inTrain <- createDataPartition(trainCleaned$classe, p=0.70, list=F)
subtrainData <- trainCleaned[inTrain, ]
subtestData <- trainCleaned[-inTrain, ]
dim(subtrainData)## [1] 13737 53dim(subtestData)## [1] 5885 53We will try different prediction models until we get highest accuracy percentage.
Prediction 1 : Decision Tree
#Model
modelDecTree <- rpart(classe ~ ., data=subtrainData, method="class")
#Prediction 1
predDecTree <- predict(modelDecTree, subtestData, type="class")
# Plot of the Decision Tree
rpart.plot(modelDecTree, main="Classification Tree", extra=102, under=TRUE, faclen=0)
# testing results with confusionMatrix
confusionMatrix(predDecTree, subtestData$classe)## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 1522 236 33 79 34
## B 56 598 45 29 62
## C 36 203 866 109 141
## D 53 77 81 653 95
## E 7 25 1 94 750
##
## Overall Statistics
##
## Accuracy : 0.7458
## 95% CI : (0.7345, 0.7569)
## No Information Rate : 0.2845
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.6773
## Mcnemar's Test P-Value : < 2.2e-16
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 0.9092 0.5250 0.8441 0.6774 0.6932
## Specificity 0.9093 0.9595 0.8994 0.9378 0.9736
## Pos Pred Value 0.7994 0.7570 0.6391 0.6809 0.8552
## Neg Pred Value 0.9618 0.8938 0.9647 0.9369 0.9337
## Prevalence 0.2845 0.1935 0.1743 0.1638 0.1839
## Detection Rate 0.2586 0.1016 0.1472 0.1110 0.1274
## Detection Prevalence 0.3235 0.1342 0.2302 0.1630 0.1490
## Balanced Accuracy 0.9092 0.7423 0.8717 0.8076 0.8334#Finding the accuracy of the model
accuracyPredTree <- postResample(predDecTree, subtestData$classe)
accuracyPredTree## Accuracy Kappa
## 0.7457944 0.6773038# Finding the out of sample error
oosePredTree <- 1 - as.numeric(confusionMatrix(subtestData$classe, predDecTree)$overall[1])
oosePredTree## [1] 0.2542056So, the estimated accuracy of the model is 74.58% and the estimated out-of-sample error is 25.42%.
Prediction 2 : Random Forest Model
#Model
modelRanFor <- randomForest(classe ~ ., data=subtrainData, method="class")
#Prediction 2
predRanFor <- predict(modelRanFor, subtestData, type="class")
# testing results with confusionMatrix
confusionMatrix(predRanFor, subtestData$classe)## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 1672 4 0 0 0
## B 2 1134 6 0 0
## C 0 1 1019 9 0
## D 0 0 1 954 6
## E 0 0 0 1 1076
##
## Overall Statistics
##
## Accuracy : 0.9949
## 95% CI : (0.9927, 0.9966)
## No Information Rate : 0.2845
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.9936
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 0.9988 0.9956 0.9932 0.9896 0.9945
## Specificity 0.9991 0.9983 0.9979 0.9986 0.9998
## Pos Pred Value 0.9976 0.9930 0.9903 0.9927 0.9991
## Neg Pred Value 0.9995 0.9989 0.9986 0.9980 0.9988
## Prevalence 0.2845 0.1935 0.1743 0.1638 0.1839
## Detection Rate 0.2841 0.1927 0.1732 0.1621 0.1828
## Detection Prevalence 0.2848 0.1941 0.1749 0.1633 0.1830
## Balanced Accuracy 0.9989 0.9970 0.9956 0.9941 0.9971#Finding the accuracy of the model
accuracyRanFor <- postResample(predRanFor, subtestData$classe)
accuracyRanFor## Accuracy Kappa
## 0.9949023 0.9935517# Finding the out of sample error
ooseRanFor <- 1 - as.numeric(confusionMatrix(subtestData$classe, predRanFor)$overall[1])
ooseRanFor## [1] 0.005097706So, the estimated accuracy of the model is 99.49% and the estimated out-of-sample error is 0.51%.
plot(modelRanFor, log="y", main="Error rate over Rainforest Model")
legend("topright", legend=unique(subtestData$classe), col=unique(as.numeric(subtestData$classe)), pch=19)
varImpPlot(modelRanFor, main=" Average Importance plots")
INFERENCE
As expected, Random Forest algorithm performed better than Decision Trees. Accuracy for Random Forest model was 99.49% compared to 74.58% for Decision Tree model.
Out-of-sample error for Random Forest was 0.51% compared to 25.42% for Decision Tree Model.
The random Forest model is chosen.
Submission
# # predict outcome levels on the original Testing data set using Random Forest algorithm
# predictfinal <- predict(modelRanFor, testCleaned, type="class")
# predictfinal
#
# # Write files for submission
# pml_write_files = function(x){
# n = length(x)
# for(i in 1:n){
# filename = paste0("problem_id_",i,".txt")
# write.table(x[i],file=filename,quote=FALSE,row.names=FALSE,col.names=FALSE)
# }
# }
#
# pml_write_files(predictfinal)