Classification Trees: Alternative Solution to Classification Methods of Linear Discriminant Analysis
Prana Ugiana Gio
Elly Rosmaini Siregar
Faigiziduhu Bu’ulolo
Department of Mathematics
University of Sumatera Utara
3rd, International Seminar on Operational Research
Medan, August 21-23, 2015
Linear Classification Based on LDA
The use of LDA will generate a discriminant function, which is a linear function. This function can be used to classify an object enter into one of group of dependent variable. Consider the following data.
#Read or open data "plot interior.csv"
data=read.csv("plot interior.csv")
#Show the data
data
## Y X1 X2
## 1 A 1.0 0.5
## 2 A 1.2 0.6
## 3 A 1.4 0.5
## 4 B 1.5 0.6
## 5 A 2.0 0.6
## 6 A 2.5 0.9
## 7 A 2.3 1.1
## 8 A 2.4 1.1
## 9 A 2.1 0.6
## 10 A 1.9 0.8
## 11 B 1.0 0.8
## 12 B 1.2 1.0
## 13 B 1.4 1.2
## 14 B 1.5 1.0
## 15 B 2.0 1.7
## 16 A 2.5 2.2
## 17 B 2.3 2.0
## 18 B 2.4 2.1
## 19 B 2.1 1.9
## 20 B 1.9 1.5
#Plot Data
library(ggplot2)
ggplot(data, aes(X1, X2)) + geom_point(aes(color = Y,
shape = Y)) + geom_text(data = NULL, x = 1.6, y = 2, label = "In general, green point (tree angle) is positioned above,", colour="blue") + geom_text(data = NULL, x = 1.5, y = 1.9, label = "whereas red point (circle) is positioned below", colour="blue")
Linear Classification Based on LDA
#Perform LDA
library(MASS) #Load package 'MASS'
## Warning: package 'MASS' was built under R version 3.2.2
fit.LDA = lda( Y ~ X1 + X2, data)
fit.LDA
## Call:
## lda(Y ~ X1 + X2, data = data)
##
## Prior probabilities of groups:
## A B
## 0.5 0.5
##
## Group means:
## X1 X2
## A 1.93 0.89
## B 1.73 1.38
##
## Coefficients of linear discriminants:
## LD1
## X1 -2.683033
## X2 2.937740
#Perform classification
class.LDA.C = predict(fit.LDA, data[,c(2,3)])$class
class.LDA.C
## [1] B B A A A A A A A A B B B B B B B B B B
## Levels: A B
table(data[,1],class.LDA.C)
## class.LDA.C
## A B
## A 7 3
## B 1 9
Classification Based on Classification Trees
#Perform classification trees
library(rpart)
tree <- rpart(Y ~ X1 + X2, data)
library(rpart.plot)
prp(tree, faclen = 0, cex = 0.8, extra = 1)
#Prediction
treePrediction <- predict(tree, data, type = "class")
treePrediction
## 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
## A A A A A A A A A A A A B A B B B B B B
## Levels: A B
#load package caret
library(caret)
## Loading required package: lattice
## Warning: package 'lattice' was built under R version 3.2.2
confusionMatrix(treePrediction, data$Y)
## Confusion Matrix and Statistics
##
## Reference
## Prediction A B
## A 9 4
## B 1 6
##
## Accuracy : 0.75
## 95% CI : (0.509, 0.9134)
## No Information Rate : 0.5
## P-Value [Acc > NIR] : 0.02069
##
## Kappa : 0.5
## Mcnemar's Test P-Value : 0.37109
##
## Sensitivity : 0.9000
## Specificity : 0.6000
## Pos Pred Value : 0.6923
## Neg Pred Value : 0.8571
## Prevalence : 0.5000
## Detection Rate : 0.4500
## Detection Prevalence : 0.6500
## Balanced Accuracy : 0.7500
##
## 'Positive' Class : A
##
