Coursera Machine Learning
Filipe Rigueiro
28 November 2017
Introduction
Objectives
One thing that people regularly do is quantify how much of a particular activity they do, but they rarely quantify how well they do it. In this project, your goal will be to use data from accelerometers on the belt, forearm, arm, and dumbell of 6 participants.
Background
Using devices such as Jawbone Up, Nike FuelBand, and Fitbit it is now possible to collect a large amount of data about personal activity relatively inexpensively. These type of devices are part of the quantified self movement - a group of enthusiasts who take measurements about themselves regularly to improve their health, to find patterns in their behavior, or because they are tech geeks. One thing that people regularly do is quantify how much of a particular activity they do, but they rarely quantify how well they do it. In this project, your goal will be to use data from accelerometers on the belt, forearm, arm, and dumbell of 6 participants. They were asked to perform barbell lifts correctly and incorrectly in 5 different ways. More information is available from the website here: http://groupware.les.inf.puc-rio.br/har (see the section on the Weight Lifting Exercise Dataset).
Data Loading
Data
The training data for this project are available here:
https://d396qusza40orc.cloudfront.net/predmachlearn/pml-training.csv
The test data are available here:
https://d396qusza40orc.cloudfront.net/predmachlearn/pml-testing.csv
Data description
The data for this project come from this source: http://groupware.les.inf.puc-rio.br/har.
Velloso, E.; Bulling, A.; Gellersen, H.; Ugulino, W.; Fuks, H. “Qualitative Activity Recognition of Weight Lifting Exercises. Proceedings of 4th International Conference in Cooperation with SIGCHI (Augmented Human ’13)”. Stuttgart, Germany: ACM SIGCHI, 2013.
Thanks to the above mentioned authors for being so generous.
A short description of the datasets content from the authors’ website:
“Six young health participants were asked to perform one set of 10 repetitions of the Unilateral Dumbbell Biceps Curl in five different fashions: exactly according to the specification (Class A), throwing the elbows to the front (Class B), lifting the dumbbell only halfway (Class C), lowering the dumbbell only halfway (Class D) and throwing the hips to the front (Class E).
Class A corresponds to the specified execution of the exercise, while the other 4 classes correspond to common mistakes. Participants were supervised by an experienced weight lifter to make sure the execution complied to the manner they were supposed to simulate. The exercises were performed by six male participants aged between 20-28 years, with little weight lifting experience. We made sure that all participants could easily simulate the mistakes in a safe and controlled manner by using a relatively light dumbbell (1.25kg)."
Preparation
Loading required packages into R.
rm(list=ls())
setwd("C:/Users/FILIPE/Desktop/Coursera")
library(knitr)
library(caret)## Loading required package: lattice## Loading required package: ggplot2library(rpart)
library(rpart.plot)
library(rattle)## Rattle: A free graphical interface for data science with R.
## Version 5.1.0 Copyright (c) 2006-2017 Togaware Pty Ltd.
## Type 'rattle()' to shake, rattle, and roll your data.library(randomForest)## randomForest 4.6-12## Type rfNews() to see new features/changes/bug fixes.##
## Attaching package: 'randomForest'## The following object is masked from 'package:rattle':
##
## importance## The following object is masked from 'package:ggplot2':
##
## marginlibrary(corrplot)## corrplot 0.84 loadedset.seed(2525)Partioning the training set into two.
# set the URL for the download
UrlTrain <- "https://d396qusza40orc.cloudfront.net/predmachlearn/pml-training.csv"
UrlTest <- "https://d396qusza40orc.cloudfront.net/predmachlearn/pml-testing.csv"
# download the datasets
training <- read.csv(url(UrlTrain))
testing <- read.csv(url(UrlTest))
# create a partition with the training dataset
inTrain <- createDataPartition(training$classe, p=0.7, list=FALSE)
TrainSet <- training[inTrain, ]
TestSet <- training[-inTrain, ]
dim(TrainSet)## [1] 13737 160dim(TestSet)## [1] 5885 160Both created datasets have 160 variables. Those variables have many NAs, that can be removed with the cleaning procedures below. The Near Zero variance (NZeroV) variables are also removed and the ID variables as well.
Cleaning the data
NZeroV <- nearZeroVar(TrainSet)
TrainSet <- TrainSet[, -NZeroV]
TestSet <- TestSet[, -NZeroV]
dim(TrainSet)## [1] 13737 108# remove variables that are mostly NA
AllNAs <- sapply(TrainSet, function(x) mean(is.na(x))) > 0.95
TrainSet <- TrainSet[, AllNAs==FALSE]
TestSet <- TestSet[, AllNAs==FALSE]
dim(TrainSet)## [1] 13737 59# remove identification only variables (columns 1 to 5)
TrainSet <- TrainSet[, -(1:5)]
TestSet <- TestSet[, -(1:5)]
dim(TrainSet)## [1] 13737 54dim(TestSet)## [1] 5885 54After the cleaning the datasets have 54 variables.
Analysis of Correlation
We can visualize the correlation of the variables with a handy plot.
corMatrix <- cor(TrainSet[, -54])
corrplot(corMatrix, order = "FPC", method = "color", type = "lower",
tl.cex = 0.8, tl.col = rgb(0, 0, 0))
The highly correlated variables are shown in darker colors in the graph above. To make an evem more compact analysis, a PCA (Principal Components Analysis) could be performed as pre-processing step to the datasets.
Prediction Model
I will use three methods to model the regressions (in the Train dataset) and the best one (with higher accuracy when applied to the Test dataset) will be used for the quiz predictions. The methods are: Random Forests, Decision Tree and Generalized Boosted Model, as described below. A Confusion Matrix is plotted at the end of each analysis to better visualize the accuracy of the models.
1 Random Forest
set.seed(2525)
controlRF <- trainControl(method="cv", number=3, verboseIter=FALSE)
modFitRandForest <- train(classe ~ ., data=TrainSet, method="rf",
trControl=controlRF)
modFitRandForest$finalModel##
## Call:
## randomForest(x = x, y = y, mtry = param$mtry)
## Type of random forest: classification
## Number of trees: 500
## No. of variables tried at each split: 27
##
## OOB estimate of error rate: 0.22%
## Confusion matrix:
## A B C D E class.error
## A 3905 0 0 0 1 0.0002560164
## B 6 2651 1 0 0 0.0026335591
## C 0 3 2393 0 0 0.0012520868
## D 0 0 12 2239 1 0.0057726465
## E 0 0 0 6 2519 0.0023762376# prediction on Test dataset
predictRandForest <- predict(modFitRandForest, newdata=TestSet)
confMatRandForest <- confusionMatrix(predictRandForest, TestSet$classe)
confMatRandForest## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 1673 3 0 0 0
## B 1 1136 2 0 0
## C 0 0 1024 4 0
## D 0 0 0 960 1
## E 0 0 0 0 1081
##
## Overall Statistics
##
## Accuracy : 0.9981
## 95% CI : (0.9967, 0.9991)
## No Information Rate : 0.2845
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.9976
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 0.9994 0.9974 0.9981 0.9959 0.9991
## Specificity 0.9993 0.9994 0.9992 0.9998 1.0000
## Pos Pred Value 0.9982 0.9974 0.9961 0.9990 1.0000
## Neg Pred Value 0.9998 0.9994 0.9996 0.9992 0.9998
## Prevalence 0.2845 0.1935 0.1743 0.1638 0.1839
## Detection Rate 0.2843 0.1930 0.1740 0.1631 0.1837
## Detection Prevalence 0.2848 0.1935 0.1747 0.1633 0.1837
## Balanced Accuracy 0.9993 0.9984 0.9986 0.9978 0.9995# plot matrix results
plot(confMatRandForest$table, col = confMatRandForest$byClass,
main = paste("Random Forest - Accuracy =",
round(confMatRandForest$overall['Accuracy'], 4)))
#2 Decision Tree Method
set.seed(2525)
modFitDecTree <- rpart(classe ~ ., data=TrainSet, method="class")
fancyRpartPlot(modFitDecTree)
# prediction on Test dataset
predictDecTree <- predict(modFitDecTree, newdata=TestSet, type="class")
confMatDecTree <- confusionMatrix(predictDecTree, TestSet$classe)
confMatDecTree## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 1502 212 52 72 66
## B 66 664 18 94 172
## C 12 59 898 126 64
## D 88 160 54 637 152
## E 6 44 4 35 628
##
## Overall Statistics
##
## Accuracy : 0.7356
## 95% CI : (0.7241, 0.7468)
## No Information Rate : 0.2845
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.6643
## Mcnemar's Test P-Value : < 2.2e-16
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 0.8973 0.5830 0.8752 0.6608 0.5804
## Specificity 0.9045 0.9263 0.9463 0.9077 0.9815
## Pos Pred Value 0.7889 0.6548 0.7748 0.5839 0.8759
## Neg Pred Value 0.9568 0.9025 0.9729 0.9318 0.9122
## Prevalence 0.2845 0.1935 0.1743 0.1638 0.1839
## Detection Rate 0.2552 0.1128 0.1526 0.1082 0.1067
## Detection Prevalence 0.3235 0.1723 0.1969 0.1854 0.1218
## Balanced Accuracy 0.9009 0.7546 0.9108 0.7843 0.7809plot(confMatDecTree$table, col = confMatDecTree$byClass,
main = paste("Decision Tree - Accuracy =",
round(confMatDecTree$overall['Accuracy'], 4)))
#3 Generalized Booster Model Method
# model fit
set.seed(2525)
controlGBM <- trainControl(method = "repeatedcv", number = 5, repeats = 1)
modFitGBM <- train(classe ~ ., data=TrainSet, method = "gbm",
trControl = controlGBM, verbose = FALSE)## Loading required package: survival##
## Attaching package: 'survival'## The following object is masked from 'package:caret':
##
## cluster## Loading required package: splines## Loading required package: parallel## Loaded gbm 2.1.3modFitGBM$finalModel## A gradient boosted model with multinomial loss function.
## 150 iterations were performed.
## There were 53 predictors of which 42 had non-zero influence.# prediction on Test dataset
predictGBM <- predict(modFitGBM, newdata=TestSet)
confMatGBM <- confusionMatrix(predictGBM, TestSet$classe)
confMatGBM## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 1670 9 0 0 0
## B 3 1121 6 8 3
## C 0 9 1019 17 2
## D 0 0 1 939 11
## E 1 0 0 0 1066
##
## Overall Statistics
##
## Accuracy : 0.9881
## 95% CI : (0.985, 0.9907)
## No Information Rate : 0.2845
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.985
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 0.9976 0.9842 0.9932 0.9741 0.9852
## Specificity 0.9979 0.9958 0.9942 0.9976 0.9998
## Pos Pred Value 0.9946 0.9825 0.9733 0.9874 0.9991
## Neg Pred Value 0.9990 0.9962 0.9986 0.9949 0.9967
## Prevalence 0.2845 0.1935 0.1743 0.1638 0.1839
## Detection Rate 0.2838 0.1905 0.1732 0.1596 0.1811
## Detection Prevalence 0.2853 0.1939 0.1779 0.1616 0.1813
## Balanced Accuracy 0.9977 0.9900 0.9937 0.9858 0.9925# plot matrix results
plot(confMatGBM$table, col = confMatGBM$byClass,
main = paste("GBM - Accuracy =", round(confMatGBM$overall['Accuracy'], 4)))
#Applying the correct Model
The acuracy of the models is as follows:
Random forest: 0.9981 Decision Tree: 0.7356 GBM: 0.9881
In this case The Random Forest Model is the model with the best accuracy with 0.9981
I will use the modFitRandomForest to predict the 20 quiz results.
predictTEST <- predict(modFitRandForest, newdata=testing)
predictTEST## [1] B A B A A E D B A A B C B A E E A B B B
## Levels: A B C D E