Practical Machine Learning
Konstantinos
5 January 2018
Introduction
Using devices such as Jawbone Up, Nike FuelBand, and Fitbit it is now possible to collect a large amount of data about personal activity relatively inexpensively. These type of devices are part of the quantified self movement – a group of enthusiasts who take measurements about themselves regularly to improve their health, to find patterns in their behavior, or because they are tech geeks. One thing that people regularly do is quantify how much of a particular activity they do, but they rarely quantify how well they do it. In this project, your goal will be to use data from accelerometers on the belt, forearm, arm, and dumbell of 6 participants. They were asked to perform barbell lifts correctly and incorrectly in 5 different ways. More information is available from the website here: http://web.archive.org/web/20161224072740/http:/groupware.les.inf.puc-rio.br/har (see the section on the Weight Lifting Exercise Dataset). # Data
library(randomForest)
library(caret)
library(dplyr)
library(rpart)Read Dataset
- Read the dataset
- Remove ID variable
- Remove variables full of NAs
train <- read.csv("pml-training.csv",header=T,sep=",",na.strings = c('NA',''))
test <- read.csv("pml-testing.csv",header=T,sep=",",na.strings = c('NA',''))
train <- train[,(colSums(is.na(train)) == 0)]
test <- test[,(colSums(is.na(test)) == 0)]
train <- select(train, -X )
test <- select(test, -X )- Validation data set. 75% of the train set will be used to train the model and the rest 25% to validate
s <- createDataPartition(train$user_name, p = 0.75, list = FALSE)
train_set <- train[s,]
val_set <- train[-s,]Model
- Train Random Forest model with all variables and count how much time does it take to run
time = Sys.time()
rf_all <- randomForest(classe~.,train_set)
time1 = Sys.time()
rf_all_time <- time1-time
rf_all##
## Call:
## randomForest(formula = classe ~ ., data = train_set)
## Type of random forest: classification
## Number of trees: 500
## No. of variables tried at each split: 7
##
## OOB estimate of error rate: 0.1%
## Confusion matrix:
## A B C D E class.error
## A 4208 1 0 0 0 0.0002375861
## B 2 2846 0 0 0 0.0007022472
## C 0 2 2560 2 0 0.0015600624
## D 0 0 3 2397 1 0.0016659725
## E 0 0 0 4 2692 0.0014836795- Accuracy, Confusion Matrix and other Metrics
confusionMatrix(val_set$classe,predict(rf_all, val_set[ ,-59] ))## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 1371 0 0 0 0
## B 0 948 1 0 0
## C 0 4 852 2 0
## D 0 0 2 813 0
## E 0 0 0 0 911
##
## Overall Statistics
##
## Accuracy : 0.9982
## 95% CI : (0.9965, 0.9992)
## No Information Rate : 0.2796
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.9977
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 1.0000 0.9958 0.9965 0.9975 1.0000
## Specificity 1.0000 0.9997 0.9985 0.9995 1.0000
## Pos Pred Value 1.0000 0.9989 0.9930 0.9975 1.0000
## Neg Pred Value 1.0000 0.9990 0.9993 0.9995 1.0000
## Prevalence 0.2796 0.1941 0.1743 0.1662 0.1858
## Detection Rate 0.2796 0.1933 0.1737 0.1658 0.1858
## Detection Prevalence 0.2796 0.1935 0.1750 0.1662 0.1858
## Balanced Accuracy 1.0000 0.9978 0.9975 0.9985 1.0000Variable Selection
- See the Importance of the variables from the next plot
varImpPlot(rf_all,cex=.8)
- I played around with the number of variables. I concluded to the obvious model. WITH ONLY 4 VARIABLES. After the 4 variables with most importance we can see from the plot that there is a big decrease.
col_index <- varImp(rf_all) %>%
mutate(names = row.names(.)) %>%
arrange(-Overall)
imp_names <- col_index$names[1:4]
time = Sys.time()
rf_few <- randomForest(train_set[,imp_names],train_set[,59])
time1 = Sys.time()
rf_few_time <- time1-time- Bellow are the results. The same great accuracy with only 4 variables. Why is this imprortant?? Faster model.
confusionMatrix(val_set$classe,predict(rf_few, val_set[,imp_names]))## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 1371 0 0 0 0
## B 0 949 0 0 0
## C 0 2 856 0 0
## D 0 0 0 815 0
## E 0 0 0 0 911
##
## Overall Statistics
##
## Accuracy : 0.9996
## 95% CI : (0.9985, 1)
## No Information Rate : 0.2796
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.9995
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 1.0000 0.9979 1.0000 1.0000 1.0000
## Specificity 1.0000 1.0000 0.9995 1.0000 1.0000
## Pos Pred Value 1.0000 1.0000 0.9977 1.0000 1.0000
## Neg Pred Value 1.0000 0.9995 1.0000 1.0000 1.0000
## Prevalence 0.2796 0.1939 0.1746 0.1662 0.1858
## Detection Rate 0.2796 0.1935 0.1746 0.1662 0.1858
## Detection Prevalence 0.2796 0.1935 0.1750 0.1662 0.1858
## Balanced Accuracy 1.0000 0.9989 0.9998 1.0000 1.0000The error is 0.03%
Faster model
The model with all variables took 1.1850679 minutes to run. The model withh 4 variables took 9.0795481 seconds to run.
Conclusions and prediction
- First lets predict the test set
testing <- rbind(train[100, -59] , test[,-59])
row.names(testing) <- c(100, 1:20)
predict(rf_all,testing[-1,])## 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
## B A B A A E D B A A B C B A E E A B B B
## Levels: A B C D E- I run from rpart a single tree just to visualise the model with 4 variables
tree <- rpart(classe~., data = train_set[,c(imp_names,'classe')])
plot(tree)
text(tree, use.n=F, all=F, cex=.5)