Pruning

pruneFit <- cv.tree(classTreeFit, FUN = prune.misclass)
summary(pruneFit)

##        Length Class  Mode     
## size   6      -none- numeric  
## dev    6      -none- numeric  
## k      6      -none- numeric  
## method 1      -none- character

#plot the accuracy of the predictions based on number of branches
plot(pruneFit$size, pruneFit$dev, type = "b")

#pruneFit has two columns that indicate size of tree and deviation
#Here we combine the two, and order them such that the best size count is at the top
#We then take that top row value for size and set it to a variable to be used later to create the 
#    final pruned tree
dfPruneFit <- cbind(size=pruneFit$size,dev=pruneFit$dev)
dfPruneFit <- data.frame(dfPruneFit)
dfPruneFit <- dfPruneFit %>% group_by(size)%>%arrange(size)%>%arrange(dev)
dfPruneFit

## # A tibble: 6 × 2
## # Groups:   size [6]
##    size   dev
##   <dbl> <dbl>
## 1     7    46
## 2     9    48
## 3     2    52
## 4    16    55
## 5    17    55
## 6     1    66

bestVal <- dfPruneFit$size[1]
bestVal

## [1] 7

Predictions and Tree #1

Using our pruned predictions on the test data, below is our resulting decision tree. It is simple and clear, but as we will see it performs quite poorly. Residual mean deviance still remains high.

#prune.misclass is a abbreviated form of prune.tree(method = "misclass") that takes a value 'k' 
#    to create the tree with that number of branches
pruneFitFinal <- prune.misclass(classTreeFit, best = bestVal)
summary(pruneFitFinal)

## 
## Classification tree:
## snip.tree(tree = classTreeFit, nodes = c(4L, 3L, 94L))
## Number of terminal nodes:  7 
## Residual mean deviance:  0.8756 = 124.3 / 142 
## Misclassification error rate: 0.1745 = 26 / 149

prunePred <- predict(pruneFitFinal, dplyr::select(dataTest, -"win"), type = "class")

prunePred

##  [1] FALSE TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  FALSE FALSE FALSE
## [13] FALSE TRUE  FALSE TRUE  TRUE  TRUE  TRUE  FALSE TRUE  TRUE  FALSE TRUE 
## [25] FALSE TRUE  TRUE  FALSE TRUE  TRUE  TRUE  FALSE FALSE FALSE FALSE FALSE
## [37] TRUE  FALSE FALSE TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  FALSE FALSE TRUE 
## [49] TRUE 
## Levels: FALSE TRUE

cm <- confusionMatrix(prunePred,dataTest$win)

cm

## Confusion Matrix and Statistics
## 
##           Reference
## Prediction FALSE TRUE
##      FALSE    15    4
##      TRUE      7   23
##                                           
##                Accuracy : 0.7755          
##                  95% CI : (0.6338, 0.8823)
##     No Information Rate : 0.551           
##     P-Value [Acc > NIR] : 0.0009468       
##                                           
##                   Kappa : 0.5405          
##                                           
##  Mcnemar's Test P-Value : 0.5464936       
##                                           
##             Sensitivity : 0.6818          
##             Specificity : 0.8519          
##          Pos Pred Value : 0.7895          
##          Neg Pred Value : 0.7667          
##              Prevalence : 0.4490          
##          Detection Rate : 0.3061          
##    Detection Prevalence : 0.3878          
##       Balanced Accuracy : 0.7668          
##                                           
##        'Positive' Class : FALSE           
## 

plot(pruneFitFinal, show.node.label=TRUE)
text(pruneFitFinal,pretty=0,cex=0.7)

Pruning

#cv.tree will perform cross-fold validation
pruneFit2 <- cv.tree(classTreeFit2, FUN = prune.misclass)

#plot the accuracy of the predictions based on number of branches
plot(pruneFit2$size, pruneFit2$dev, type = "b")

#pruneFit has two columns that indicate size of tree and deviation
#Here we combine the two, and order them such that the best size count is at the top
#We then take that top row value for size and set it to a variable to be used later to create the 
#    final pruned tree
dfPruneFit2 <- cbind(size=pruneFit2$size,dev=pruneFit2$dev)
dfPruneFit2 <- data.frame(dfPruneFit2)
dfPruneFit2 <- dfPruneFit2 %>% group_by(size)%>%arrange(size)%>%arrange(dev)
dfPruneFit2

## # A tibble: 6 × 2
## # Groups:   size [6]
##    size   dev
##   <dbl> <dbl>
## 1     5    26
## 2     7    26
## 3    11    26
## 4     3    33
## 5     2    36
## 6     1    67

#alternative method of choosing best method 
#dfPruneFit2$size[which.min(dfPruneFit2$dev)]

bestVal2 <- dfPruneFit2$size[1]

Predictions and Tree #2

#prune.misclass is a abbreviated form of prune.tree(method = "misclass") that takes a value 'k' 
#    to create the tree with that number of branches
pruneFitFinal2 <- prune.misclass(classTreeFit2, best = bestVal2)
summary(pruneFitFinal2)

## 
## Classification tree:
## snip.tree(tree = classTreeFit2, nodes = c(6L, 14L, 5L))
## Variables actually used in tree construction:
## [1] "tower_damage" "duration"    
## Number of terminal nodes:  5 
## Residual mean deviance:  0.5903 = 85 / 144 
## Misclassification error rate: 0.1074 = 16 / 149

prunePred2 <- predict(pruneFitFinal2, dplyr::select(dataTest2, -"win"), type = "class")

prunePred2

##  [1] FALSE FALSE TRUE  FALSE FALSE TRUE  TRUE  TRUE  TRUE  TRUE  FALSE TRUE 
## [13] FALSE TRUE  FALSE FALSE TRUE  TRUE  FALSE FALSE TRUE  FALSE FALSE TRUE 
## [25] FALSE TRUE  TRUE  FALSE TRUE  FALSE FALSE FALSE FALSE TRUE  TRUE  FALSE
## [37] TRUE  FALSE FALSE FALSE FALSE FALSE TRUE  TRUE  FALSE TRUE  FALSE TRUE 
## [49] TRUE 
## Levels: FALSE TRUE

cm2 <- confusionMatrix(prunePred2,dataTest2$win)

cm2

## Confusion Matrix and Statistics
## 
##           Reference
## Prediction FALSE TRUE
##      FALSE    19    7
##      TRUE      3   20
##                                           
##                Accuracy : 0.7959          
##                  95% CI : (0.6566, 0.8976)
##     No Information Rate : 0.551           
##     P-Value [Acc > NIR] : 0.000311        
##                                           
##                   Kappa : 0.5944          
##                                           
##  Mcnemar's Test P-Value : 0.342782        
##                                           
##             Sensitivity : 0.8636          
##             Specificity : 0.7407          
##          Pos Pred Value : 0.7308          
##          Neg Pred Value : 0.8696          
##              Prevalence : 0.4490          
##          Detection Rate : 0.3878          
##    Detection Prevalence : 0.5306          
##       Balanced Accuracy : 0.8022          
##                                           
##        'Positive' Class : FALSE           
## 

plot(pruneFitFinal2, show.node.label=TRUE)
text(pruneFitFinal2,pretty=0,cex=0.7)

Changes

When trying to represent a complicated topic with a decision tree, you might find that it becomes large and difficult to maintain. You should look for tools that allow you to version control your decision trees.

Subjectivity

A decision tree is best used in situations where the decision criteria / choices are fairly constant and the thought process is generally applicable in most cases. In cases where the choices depend on emotions or other subjective factors, creating a decision tree might be a challenge. And then there may be use cases where creating a decision tree just does not make sense.

Evolve

Your decision tree may evolve in its logic over time as you get more familiar with the topic and as you get more feedback from its audience. Unless you have tools designed to facilitate continuous improvement, your decision tree may lose its effectiveness over time.

Repeat

Often you find that the same logical pattern repeats in multiple branches of the same decision tree. This might mean that you are repeating the same logic at multiple places in your decision tree. In this situation, if you need to change the repeating logic, it can be a nightmare to make sure you made that change in all the places that pattern repeats. You need tools that allow you to clone your logic and easily manage cloned elements. “Life is too complicated not to be orderly.” – Martha Stewart

Complexity

Any decision tree created with detailed thought process might look complicated when you see it in a diagram. Most people do not like complicated things and have a strong preference for simpler alternatives. You might be better off using decision treeing platforms that have a clean and simple interface with an awesome user experience.

Familiarity

At times people use obscure elements/features inside their decision trees. This creates a problem for the audience that is not familiar with those elements/features. You need a standardized approach at creating and accessing decision trees.

Usability

A good decision tree might take a lot of work to create. Last thing you want is that it becomes this big diagram poster on a wall, frozen in time. Instead you want a solution that makes it easy for everyone to collaborate, share feedback and continuously improve the decision tree over time.

Mobile

Conventional diagramming tools allow you to create a big diagram and then share that diagram document with others. But most people are not comfortable following complex diagrams. On the contrary, a well thought out decision tree can almost behave like a dynamic application that can guide its audience to the most appropriate recommendation. Look for solutions that allow you to easily create a simple decision tree and then allow your audience to access that decision tree from any mobile browser, any time and from anywhere in the world.

Everywhere

Most conventional solutions need a laptop/desktop size screen to create or modify decision trees. But today we do everything on our mobile phones and so it naturally makes sense in looking for solutions that allow you to create/edit decision trees on any mobile browser.

Coding

Decision trees form the basic foundation for building the ever so popular chatbots today. But to create a chatbot today you almost always need a developer with advanced coding skills which makes it a big challenge. That is why you should look for a solution that allows anyone without coding skills to just create a decision tree which then becomes a dynamic conversational application serving their audience using the right tone and language. The fact that no coding skills are required in creating or modifying these decision trees means that it is easy for anyone to create and maintain these decision trees for the long term.

Features

Conventional tools to create decision trees are very limited in scope when it comes to the interactive and multimedia features. You are so much better of with a solution that allows you to attach files, show images, upload files, attach links and share feedback specific to each and every node in your decision tree.

Measure

Most decision tree solutions today cannot give you meaningful metrics and reports about how your decision tree is being used. Look for a solution that allows you to download raw data between the start and end dates of your choice so that you can analyze and find opportunities for improvement.

Integrate

Today most tools need to talk to other tools in the ecosystem. Your decision tree solution better be capable to integrate with popular CRM, Ticketing, ERP and other platforms. | As addressed earlier, it is unlikely that the data provided to a machine learning decision tree did not come available through a particular pipeline and also may not be necessary even if it did not. This is a criticism of how integrated an analyst’s environment is, not a criticism of the performance that a decision tree has as a model relative to other statistical models.