Practical Machine Learning Project
Sreekantha Thimmareddy
Sunday, April 26, 2015
Background
Using devices such as Jawbone Up, Nike FuelBand, and Fitbit it is now possible to collect a large amount of data about personal activity relatively inexpensively. In this project, your goal will be to use data from accelerometers on the belt, forearm, arm, and dumbell of 6 participants. They were asked to perform barbell lifts correctly and incorrectly in 5 different ways.
Data The training data for this project are available here:
https://d396qusza40orc.cloudfront.net/predmachlearn/pml-training.csv
The test data are available here:
https://d396qusza40orc.cloudfront.net/predmachlearn/pml-testing.csv
What to submit
The goal of the project is to predict the manner in which they did the exercise.
submission should consist of a link to a Github repo with your R markdown and compiled HTML file describing your analysis. Please constrain the text of the writeup to < 2000 words and the number of figures to be less than 5.
You should also apply your machine learning algorithm to the 20 test cases available in the test data above. Please submit your predictions in appropriate format to the programming assignment for automated grading.
For this assignment I analyzed the provided data to determine what activity an individual perform. To do this I made use of caret and randomForest, this allowed me to generate correct answers for each of the 20 test data cases provided in this assignment. I made use of a seed value for consistent results.
library(data.table)## Warning: package 'data.table' was built under R version 3.1.3library(caret)## Warning: package 'caret' was built under R version 3.1.3## Loading required package: lattice
## Loading required package: ggplot2## Warning: package 'ggplot2' was built under R version 3.1.3library(ggplot2)
library(knitr)## Warning: package 'knitr' was built under R version 3.1.3library(xtable)## Warning: package 'xtable' was built under R version 3.1.3library(randomForest)## Warning: package 'randomForest' was built under R version 3.1.3## randomForest 4.6-10
## Type rfNews() to see new features/changes/bug fixes.library(RCurl)## Warning: package 'RCurl' was built under R version 3.1.3## Loading required package: bitops## Warning: package 'bitops' was built under R version 3.1.2library(Hmisc)## Warning: package 'Hmisc' was built under R version 3.1.3## Loading required package: grid
## Loading required package: survival
## Loading required package: splines
##
## Attaching package: 'survival'
##
## The following object is masked from 'package:caret':
##
## cluster
##
## Loading required package: Formula## Warning: package 'Formula' was built under R version 3.1.2##
## Attaching package: 'Hmisc'
##
## The following object is masked from 'package:randomForest':
##
## combine
##
## The following objects are masked from 'package:xtable':
##
## label, label<-
##
## The following objects are masked from 'package:base':
##
## format.pval, round.POSIXt, trunc.POSIXt, unitslibrary(foreach)## Warning: package 'foreach' was built under R version 3.1.3library(doParallel)## Warning: package 'doParallel' was built under R version 3.1.3## Loading required package: iterators## Warning: package 'iterators' was built under R version 3.1.3## Loading required package: parallellibrary(gridExtra)## Warning: package 'gridExtra' was built under R version 3.1.3options(warn=-1)
set.seed(2048)First, I loaded the data both from the provided training and test data provided by COURSERA. Some values contained a “#DIV/0!” that I replaced with an NA value. I also casted all columns 8 to the end to be numeric.
URLTrain <- "http://d396qusza40orc.cloudfront.net/predmachlearn/pml-training.csv"
t1 <- getURL(URLTrain)
URLEvaluate <- "http://d396qusza40orc.cloudfront.net/predmachlearn/pml-testing.csv"
e1 <- getURL(URLEvaluate)
training_data <- read.csv(textConnection(t1), na.strings=c("#DIV/0!"))
evaluation_data <- read.csv(textConnection(e1), na.strings=c("#DIV/0!"))
for(i in c(8:ncol(training_data)-1)) {training_data[,i] = as.numeric(as.character(training_data[,i]))}
for(i in c(8:ncol(evaluation_data)-1)) {evaluation_data[,i] = as.numeric(as.character(evaluation_data[,i]))}Some columns were mostly blank. These did not contribute well to the prediction. I chose a feature set that only included complete columns. We also remove user name, timestamps and windows.
Determine and display out feature set.
feature_set <- colnames(training_data[colSums(is.na(training_data)) == 0])[-(1:7)]
model_data <- training_data[feature_set]
feature_set## [1] "roll_belt" "pitch_belt" "yaw_belt"
## [4] "total_accel_belt" "gyros_belt_x" "gyros_belt_y"
## [7] "gyros_belt_z" "accel_belt_x" "accel_belt_y"
## [10] "accel_belt_z" "magnet_belt_x" "magnet_belt_y"
## [13] "magnet_belt_z" "roll_arm" "pitch_arm"
## [16] "yaw_arm" "total_accel_arm" "gyros_arm_x"
## [19] "gyros_arm_y" "gyros_arm_z" "accel_arm_x"
## [22] "accel_arm_y" "accel_arm_z" "magnet_arm_x"
## [25] "magnet_arm_y" "magnet_arm_z" "roll_dumbbell"
## [28] "pitch_dumbbell" "yaw_dumbbell" "total_accel_dumbbell"
## [31] "gyros_dumbbell_x" "gyros_dumbbell_y" "gyros_dumbbell_z"
## [34] "accel_dumbbell_x" "accel_dumbbell_y" "accel_dumbbell_z"
## [37] "magnet_dumbbell_x" "magnet_dumbbell_y" "magnet_dumbbell_z"
## [40] "roll_forearm" "pitch_forearm" "yaw_forearm"
## [43] "total_accel_forearm" "gyros_forearm_x" "gyros_forearm_y"
## [46] "gyros_forearm_z" "accel_forearm_x" "accel_forearm_y"
## [49] "accel_forearm_z" "magnet_forearm_x" "magnet_forearm_y"
## [52] "magnet_forearm_z" "classe"idx <- createDataPartition(y=model_data$classe, p=0.60, list=FALSE )
training <- model_data[idx,]
testing <- model_data[-idx,]Exploratory data analysis
Exploratory data analysis showed multiple class-based clusters. Based on clusters distribution I decided to use random forests
Model building and testing
We now build 5 random forests with 150 trees each. We make use of parallel processing to build this model. I found several examples of how to perform parallel processing with random forests in R, this provided a great speedup. Provide error reports for both training and test data.
registerDoParallel()
x <- training[-ncol(training)]
y <- training$classe
rf <- foreach(ntree=rep(150, 6), .combine=randomForest::combine, .packages='randomForest') %dopar% {
randomForest(x, y, ntree=ntree)
}
predictions1 <- predict(rf, newdata=training)
confusionMatrix(predictions1,training$classe)## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 3348 0 0 0 0
## B 0 2279 0 0 0
## C 0 0 2054 0 0
## D 0 0 0 1930 0
## E 0 0 0 0 2165
##
## Overall Statistics
##
## Accuracy : 1
## 95% CI : (0.9997, 1)
## No Information Rate : 0.2843
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 1
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 1.0000 1.0000 1.0000 1.0000 1.0000
## Specificity 1.0000 1.0000 1.0000 1.0000 1.0000
## Pos Pred Value 1.0000 1.0000 1.0000 1.0000 1.0000
## Neg Pred Value 1.0000 1.0000 1.0000 1.0000 1.0000
## Prevalence 0.2843 0.1935 0.1744 0.1639 0.1838
## Detection Rate 0.2843 0.1935 0.1744 0.1639 0.1838
## Detection Prevalence 0.2843 0.1935 0.1744 0.1639 0.1838
## Balanced Accuracy 1.0000 1.0000 1.0000 1.0000 1.0000predictions2 <- predict(rf, newdata=testing)
confusionMatrix(predictions2,testing$classe)## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 2230 3 0 0 0
## B 1 1514 5 0 0
## C 0 1 1363 16 2
## D 0 0 0 1267 7
## E 1 0 0 3 1433
##
## Overall Statistics
##
## Accuracy : 0.995
## 95% CI : (0.9932, 0.9965)
## No Information Rate : 0.2845
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.9937
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 0.9991 0.9974 0.9963 0.9852 0.9938
## Specificity 0.9995 0.9991 0.9971 0.9989 0.9994
## Pos Pred Value 0.9987 0.9961 0.9863 0.9945 0.9972
## Neg Pred Value 0.9996 0.9994 0.9992 0.9971 0.9986
## Prevalence 0.2845 0.1935 0.1744 0.1639 0.1838
## Detection Rate 0.2842 0.1930 0.1737 0.1615 0.1826
## Detection Prevalence 0.2846 0.1937 0.1761 0.1624 0.1832
## Balanced Accuracy 0.9993 0.9982 0.9967 0.9921 0.9966Conclusions and Test Data Submit
As can be seen from the confusion matrix this model is very accurate. I did experiment with PCA and other models, but did not get as good of accuracy. Because my test data was around 99% accurate I expected nearly all of the submitted test cases to be correct. It turned out they were all correct.
Prepare the submission. (using COURSERA provided code)
pml_write_files = function(x){
n = length(x)
for(i in 1:n){
filename = paste0("problem_id_",i,".txt")
write.table(x[i],file=filename,quote=FALSE,row.names=FALSE,col.names=FALSE)
}
}
x <- evaluation_data
x <- x[feature_set[feature_set!='classe']]
answers <- predict(rf, newdata=x)
answers## 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
## B A B A A E D B A A B C B A E E A B B B
## Levels: A B C D Epml_write_files(answers)