Chapter 8
Bruce Kim
October 4, 2016
Problem - Conceptual
1
Draw an example (of your own invention) of a partition of two-dimensional feature space that could result from recursive binary splitting. Your example should contain at least six regions. Draw a decision tree corresponding to this partition. Be sure to label all as- pects of your figures, including the regions R1, R2 ,…, the cutpoints t1, t2 , …, and so forth.
First, Partition the data. For the practice, I would use the training set.
Data <- OJ
iTrain <- createDataPartition(y = OJ$Purchase, p = 0.8, list = FALSE)
TrainOJ <- Data[iTrain, ]
TestOJ <- Data[-iTrain, ] 1. Tree Plotplot(Data$LoyalCH, Data$PriceDiff)
tree.carseats <- tree(Purchase~ LoyalCH + PriceDiff, Data)
Prune.Tree.OJ <- prune.misclass(tree.carseats, best = 4)
plot(Prune.Tree.OJ)
text(Prune.Tree.OJ,pretty = 1)
2. ggplot
G <- ggplot(data = Data, aes(x = Data$LoyalCH, y = Data$PriceDiff))
G + geom_hline(aes(yintercept = 0.276142)) +
geom_hline(aes(yintercept = 0.5036)) +
geom_vline(aes(xintercept = 0.05)) 
Applied
In the lab, we applied random forests to the Boston data using mtry=6 and using ntree=25 and ntree=500. Create a plot displaying the test error resulting from random forests on this data set for a more com- prehensive range of values for mtry and ntree. You can model your plot after Figure 8.10. Describe the results obtained.
Plot of Error rate - Variable up to 13, Number of trees up to 25
library(randomForest)## randomForest 4.6-12## Type rfNews() to see new features/changes/bug fixes.##
## Attaching package: 'randomForest'## The following object is masked from 'package:ggplot2':
##
## margin## The following object is masked from 'package:dplyr':
##
## combineMat <- matrix(NA, nrow = 13, 25)
for(i in 1:13) {
for(j in 1:25) {
bag.boston <- randomForest(medv~., data = TrainBot, mtry = i, ntree = j, importance =TRUE)
yhat.bag <- predict(bag.boston, newdata = TestBot)
Mat[i,j] <- sqrt(mean((yhat.bag-boston.test)^2))}
}
plot(1:13, Mat[,1], type = 'l', col = 1, xlim = c(1,13), ylim = c(1,7))
for(i in 2:25) {lines(1:13, Mat[,i], col = i)}
which.min(Mat)## [1] 160min(Mat)## [1] 2.981105150/25## [1] 6Tree With Lowest Error Rate
bag.boston.6 <- randomForest(medv~., data = TrainBot, mtry = 6, ntree = 25, importance =TRUE)
bag.boston.500 <- randomForest(medv~., data = TrainBot, mtry = 6, ntree = 500, importance =TRUE)
yhat.bag.6 <- predict(bag.boston.6, newdata = TestBot)
yhat.bag.500 <- predict(bag.boston.500, newdata = TestBot)
plot(bag.boston.500)
plot(yhat.bag.6, boston.test, col = 1)
points(yhat.bag.500, boston.test, col = 2)
bag.6 <- mean((yhat.bag.6-boston.test)^2); bag.500 <-mean((yhat.bag.500-boston.test)^2)
d <- data.frame(cbind(bag.6, bag.500), row.names = "Error")
colnames(d) <- c("ntree25","ntree500")
kable(d)| ntree25 | ntree500 | |
|---|---|---|
| Error | 10.35014 | 10.02425 |
a <- importance(bag.boston.500, type = 1); b <- importance(bag.boston.6, type = 1)
kable(cbind(a,b))| %IncMSE | %IncMSE | |
|---|---|---|
| crim | 14.183347 | 3.6351442 |
| zn | 3.150263 | 0.0908408 |
| indus | 9.349457 | 2.3866579 |
| chas | 2.834440 | -0.6183269 |
| nox | 18.028836 | 3.9316876 |
| rm | 37.020413 | 7.9408723 |
| age | 13.728558 | 3.6130493 |
| dis | 17.367296 | 4.6534036 |
| rad | 6.388366 | 2.2772102 |
| tax | 11.938116 | 2.1062521 |
| ptratio | 15.484746 | 3.1808428 |
| black | 8.792726 | 2.6755077 |
| lstat | 32.340405 | 7.7420715 |
par(mar = c(3,5,3,4), mfrow = c(1,2))
varImpPlot(bag.boston.6,type = 1)
varImpPlot(bag.boston.500,type = 1)
In the lab, a classification tree was applied to the Carseats data set after converting Sales into a qualitative response variable. Now we will seek to predict Sales using regression trees and related approaches, treating the response as a quantitative variable.
Car Seat Data
Car <- Carseats
Car <- Car %>%
mutate(High = ifelse(Sales <= median(Car$Sales),"No","Yes"))
Car$High <- as.factor(Car$High)
set.seed(4)
iTrain <- createDataPartition(y = Car$High, p = 0.75, list = F)
TrainCar <- Car[iTrain, ]
TestCar <- Car[-iTrain, ]tree.carseats <- tree(Sales ~. -High, TrainCar)
Pred.tree.Carseats <- predict(tree.carseats, TestCar)Error <- rep(0,500)
for(i in 1:500) {
tree.carseats <- tree(Sales ~. -High, TrainCar)
prune.carseats <- prune.tree(tree.carseats, best = i+1)
tree.pred <- predict(prune.carseats, TestCar)
Error[i] <- mean((tree.pred-TestCar$Sales)^2)
}## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree size
## Warning in prune.tree(tree.carseats, best = i + 1): best is bigger than
## tree sizepar(mfrow = c(1,1))
plot(1:500, Error, type = 'l', col = "red")
Tree <- prune.tree(tree.carseats, best = which.min(Error))
plot(Tree)
text(Tree, pretty = 0)
which.min(Error)## [1] 10cv.carseats <- cv.tree(tree.carseats, FUN = prune.tree)
data <- list(cv.carseats$size, cv.carseats$dev)
data <- data.frame(do.call(rbind,data), row.names = c("Size", "Dev"))
which.min(cv.carseats$dev)## [1] 9kable(data)| X1 | X2 | X3 | X4 | X5 | X6 | X7 | X8 | X9 | X10 | X11 | X12 | X13 | X14 | X15 | X16 | X17 | X18 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Size | 19.000 | 18.000 | 17.000 | 15.000 | 14.000 | 13.000 | 12.000 | 11.000 | 10.000 | 9.000 | 8.000 | 7.00 | 6.000 | 5.000 | 4.000 | 3.000 | 2.000 | 1.000 |
| Dev | 1401.307 | 1432.539 | 1439.545 | 1458.778 | 1453.513 | 1457.102 | 1452.613 | 1391.776 | 1376.365 | 1393.586 | 1411.681 | 1446.44 | 1521.482 | 1568.318 | 1530.353 | 1557.806 | 1805.747 | 2250.066 |
prune.carseats.19 <- prune.tree(tree.carseats, best = 19)
tree.pred.19 <- predict(prune.carseats.19, TestCar)
Error.19 <- mean((tree.pred.19-TestCar$Sales)^2)
prune.carseats.10 <- prune.tree(tree.carseats, best = 10)
tree.pred.10 <- predict(prune.carseats.10, TestCar)
Error.10 <- mean((tree.pred.10-TestCar$Sales)^2)
d <- do.call(rbind, list(Error.10, Error.19))
d <- data.frame(d, row.names = c("10 Nodes", "19 Nodes"))
kable(d)| d | |
|---|---|
| 10 Nodes | 4.515091 |
| 19 Nodes | 4.697121 |
plot(prune.carseats.19)
text(prune.carseats.19)
title("Ntree 19")
Error.10## [1] 4.515091
library(randomForest)
set.seed(2)
bag.Car <- randomForest(Sales~. - High, data = TrainCar, mtry = dim(TrainCar)[2]-2, importance =TRUE)
importance(bag.Car)## %IncMSE IncNodePurity
## CompPrice 31.2278276 219.80159
## Income 9.5398440 124.53352
## Advertising 21.6789648 180.17508
## Population 0.9407570 84.75201
## Price 66.1478659 655.98297
## ShelveLoc 71.4930873 581.43789
## Age 23.8121025 250.46009
## Education 0.4617071 62.21422
## Urban -2.5223435 10.62443
## US 1.6664370 9.69094varImpPlot(bag.Car)
str(TrainCar)## 'data.frame': 301 obs. of 12 variables:
## $ Sales : num 9.5 10.06 7.4 4.15 10.81 ...
## $ CompPrice : num 138 113 117 141 124 132 121 117 122 115 ...
## $ Income : num 73 35 100 64 113 110 78 94 35 28 ...
## $ Advertising: num 11 10 4 3 13 0 9 4 2 11 ...
## $ Population : num 276 269 466 340 501 108 150 503 393 29 ...
## $ Price : num 120 80 97 128 72 124 100 94 136 86 ...
## $ ShelveLoc : Factor w/ 3 levels "Bad","Good","Medium": 1 3 3 1 1 3 1 2 3 2 ...
## $ Age : num 42 59 55 38 78 76 26 50 62 53 ...
## $ Education : num 17 12 14 13 16 10 10 13 18 18 ...
## $ Urban : Factor w/ 2 levels "No","Yes": 2 2 2 2 1 1 1 2 2 2 ...
## $ US : Factor w/ 2 levels "No","Yes": 2 2 2 1 2 1 2 2 1 2 ...
## $ High : Factor w/ 2 levels "No","Yes": 2 2 1 1 2 1 2 2 1 2 ...This problem involves the OJ data set which is part of the ISLR package.
- Create a training set containing a random sample of 800 observations, and a test set containing the remaining observations.
Orange <- OJ
iTrain.OJ <- createDataPartition(OJ$Purchase, p = 800/1070, list = FALSE)
Train.OJ <- Orange[iTrain.OJ, ]
Test.OJ <- Orange[-iTrain.OJ, ]- Fit a tree to the training data, with Purchase as the response and the other variables except for Buy as predictors. Use the summary() function to produce summary statistics about the tree, and describe the results obtained. What is the training error rate? How many terminal nodes does the tree have?
tree.orange <- tree(formula = Purchase ~., data = Train.OJ)
summary(tree.orange)##
## Classification tree:
## tree(formula = Purchase ~ ., data = Train.OJ)
## Variables actually used in tree construction:
## [1] "LoyalCH" "PriceDiff" "ListPriceDiff" "PctDiscMM"
## Number of terminal nodes: 8
## Residual mean deviance: 0.7636 = 605.5 / 793
## Misclassification error rate: 0.1685 = 135 / 801pred.tree.orange <- predict(tree.orange, newdata = Train.OJ, type = 'class')
Tb.OJ <- table(Train.OJ$Purchase,pred.tree.orange)
Error.OJ <- (Tb.OJ[1,1]+Tb.OJ[2,2])/sum(Tb.OJ)
Tb.OJ;Error.OJ## pred.tree.orange
## CH MM
## CH 452 37
## MM 98 214## [1] 0.8314607- Create a plot of the tree, and interpret the results.
par(mfrow = c(1,1))
plot(tree.orange)
text(tree.orange)
- Predict the response on the test data, and produce a confusion matrix comparing the test labels to the predicted test labels. What is the test error rate?
pred.tree.orange.test <- predict(tree.orange, newdata = Test.OJ, type = 'class')
Tb.OJ.Test <- table(Test.OJ$Purchase,pred.tree.orange.test)
Error.OJ <- (Tb.OJ.Test[1,1]+Tb.OJ.Test[2,2])/sum(Tb.OJ.Test)
Tb.OJ;Error.OJ## pred.tree.orange
## CH MM
## CH 452 37
## MM 98 214## [1] 0.8550186- Apply the cv.tree() function to the training set in order to determine the optimal tree size.
cv.tree.orange <- cv.tree(object = tree.orange, FUN = prune.misclass,K = 10)
plot(cv.tree.orange$size,cv.tree.orange$dev, type = 'l')
title("Cross Validation Error per Size")
Produce a pruned tree corresponding to the optimal tree size obtained using cross-validation. If cross-validation does not lead to selection of a pruned tree, then create a pruned tree with five terminal nodes.
- Compare the training error rates between the pruned and unpruned trees. Which is higher?
Compare the test error rates between the pruned and unpruned trees. Which is higher?
Traing.prune.orange.9 <- prune.misclass(tree.orange, best = 9)## Warning in prune.tree(tree = tree.orange, best = 9, method = "misclass"):
## best is bigger than tree sizeTraing.pred.prune.orange.9 <- predict(Traing.prune.orange.9, newdata = Train.OJ, type = 'class')
Traing.Tb.prune.orange.9 <- table(Train.OJ$Purchase,Traing.pred.prune.orange.9)
Traing.Error.orange.9 <- (Traing.Tb.prune.orange.9[1,1] + Traing.Tb.prune.orange.9[2,2])/sum(Traing.Tb.prune.orange.9)
Traing.prune.orange.5 <- prune.misclass(tree.orange, best = 5)
Traing.pred.prune.orange.5 <- predict(Traing.prune.orange.5, newdata = Train.OJ, type = 'class')
Traing.Tb.prune.orange.5 <- table(Train.OJ$Purchase,Traing.pred.prune.orange.5)
Traing.Error.orange.5 <- (Traing.Tb.prune.orange.5[1,1] + Traing.Tb.prune.orange.5[2,2])/sum(Traing.Tb.prune.orange.5)
Traing.Error.orange.9## [1] 0.8314607Traing.Error.orange.5## [1] 0.8189763Test.prune.orange.9 <- prune.misclass(tree.orange, best = 9)## Warning in prune.tree(tree = tree.orange, best = 9, method = "misclass"):
## best is bigger than tree sizeTest.pred.prune.orange.9 <- predict(Test.prune.orange.9, newdata = Test.OJ, type = 'class')
Test.Tb.prune.orange.9 <- table(Test.OJ$Purchase,Test.pred.prune.orange.9)
Test.Error.orange.9 <- (Test.Tb.prune.orange.9[1,1] + Test.Tb.prune.orange.9[2,2])/sum(Test.Tb.prune.orange.9)
Test.prune.orange.5 <- prune.misclass(tree.orange, best = 5)
Test.pred.prune.orange.5 <- predict(Test.prune.orange.5, newdata = Test.OJ, type = 'class')
Test.Tb.prune.orange.5 <- table(Test.OJ$Purchase,Test.pred.prune.orange.5)
Test.Error.orange.5 <- (Test.Tb.prune.orange.5[1,1] + Test.Tb.prune.orange.5[2,2])/sum(Test.Tb.prune.orange.5)Error.Tb.Orange.Tr <- list(Traing.Error.orange.9, Traing.Error.orange.5)
Error.Tb.Orange.Ts <- list(Test.Error.orange.9, Test.Error.orange.5)
d <- data.frame(rbind(Error.Tb.Orange.Tr, Error.Tb.Orange.Ts), row.names = c("Tr Error", "Ts Error"))
colnames(d) <- c("Node 5", "Node 9")
kable(d)| Node 5 | Node 9 | |
|---|---|---|
| Tr Error | 0.8314607 | 0.8189763 |
| Ts Error | 0.8550186 | 0.8289963 |
- We now use boosting to predict Salary in the Hitters data set.
- Remove the observations for whom the salary information is unknown, and then log-transform the salaries.
Hit <- Hitters
Hit <- Hit[is.na(Hit$Salary) == FALSE,]- Create a training set consisting of the first 200 observations, and a test set consisting of the remaining observations.
set.seed(256)
iTrain.Hit <- createDataPartition(y = Hit$Salary, p = 200/nrow(Hit), list = FALSE)
Train.Hit <- Hit[iTrain.Hit, ]
Test.Hit <- Hit[-iTrain.Hit, ]Perform boosting on the training set with 1,000 trees for a range of values of the shrinkage parameter λ. Produce a plot with different shrinkage values on the x-axis and the corresponding training set MSE on the y-axis.
Produce a plot with different shrinkage values on the x-axis and the corresponding test set MSE on the y-axis.
library(gbm)## Loading required package: survival##
## Attaching package: 'survival'## The following object is masked from 'package:caret':
##
## cluster## Loading required package: splines## Loading required package: parallel## Loaded gbm 2.1.1Train.Error.Hit.Boosting <- rep(0,100)
Test.Error.Hit.Boosting <- rep(0,100)
Seq <- seq(1,100) * 0.01
for(i in Seq) {
boost.Hit <- gbm(Salary~., data = Train.Hit, distribution = "gaussian",
n.trees = 1000, interaction.depth = 1,
shrinkage = i, verbose = F)
Train.yhat.boost.Hit <- predict(boost.Hit, newdata = Train.Hit, n.trees = 1000)
Test.yhat.boost.Hit <- predict(boost.Hit, newdata = Test.Hit, n.trees = 1000)
Train.Error.Hit.Boosting[i*100] <- mean((Train.yhat.boost.Hit - Train.Hit$Salary)^2)
Test.Error.Hit.Boosting[i*100] <- mean((Test.yhat.boost.Hit - Test.Hit$Salary)^2)
}
par(mfrow = c(1,2), mar = c(4.1,4.1,2,4.1))
plot(Seq, Train.Error.Hit.Boosting, type = 'l', xlab = "Lambda", ylab = "Train Error MSE")
points(which.min(Train.Error.Hit.Boosting)*0.01, min(Train.Error.Hit.Boosting), col = "red", pch = 19)
title(main = "MSE - Train")
plot(Seq, Test.Error.Hit.Boosting, type = 'l', xlab = "Lambda", ylab = "Test Error MSE")
title("MSE - Test")
points(which.min(Test.Error.Hit.Boosting)*0.01, min(Test.Error.Hit.Boosting), col = "blue", pch = 19)
boost.Hit.Best <- gbm(Salary~., data = Train.Hit, distribution = "gaussian",
n.trees = 1000, interaction.depth = 1,
shrinkage = which.min(Train.Error.Hit.Boosting)*0.01, verbose = F)
pretty.gbm.tree(boost.Hit.Best)## SplitVar SplitCodePred LeftNode RightNode MissingNode ErrorReduction
## 0 11 301.000000 1 2 3 7120980
## 1 -1 -66.381987 -1 -1 -1 0
## 2 -1 92.279188 -1 -1 -1 0
## 3 -1 -4.504129 -1 -1 -1 0
## Weight Prediction
## 0 100 -4.504129
## 1 61 -66.381987
## 2 39 92.279188
## 3 100 -4.504129summary(boost.Hit.Best)
## var rel.inf
## PutOuts PutOuts 16.3040548
## CRBI CRBI 9.1115536
## Walks Walks 7.4427160
## CAtBat CAtBat 6.4187922
## CRuns CRuns 6.2650116
## AtBat AtBat 6.2455559
## CHmRun CHmRun 6.1317459
## CWalks CWalks 6.1275281
## RBI RBI 6.0743008
## Years Years 4.7229021
## Assists Assists 4.6936095
## Hits Hits 4.0871733
## CHits CHits 3.9041704
## Errors Errors 3.8997047
## HmRun HmRun 3.8476555
## Runs Runs 2.8990861
## NewLeague NewLeague 0.8529343
## Division Division 0.7257205
## League League 0.2457846par(mfrow = c(1,2))
plot(boost.Hit.Best, i = "Walks")
plot(boost.Hit.Best, i = "PutOuts")
Bagging.Hit <- randomForest(Salary~. , data = Train.Hit, mtry = dim(Train.Hit)[2]-1, importance = TRUE)
pred.Bagging.Hit <- predict(Bagging.Hit, newdata = Test.Hit)
importance(Bagging.Hit)## %IncMSE IncNodePurity
## AtBat 8.0944435 1744429.05
## Hits 3.7784518 1964888.35
## HmRun 4.9508669 614021.52
## Runs 8.7224893 2141299.44
## RBI 3.6364361 2046942.58
## Walks 2.5839704 2654949.06
## Years 7.1880876 626642.80
## CAtBat 11.4352075 1717391.95
## CHits 19.3887141 7212786.69
## CHmRun 7.2695405 2854245.93
## CRuns 10.9640116 2889256.97
## CRBI 11.6827649 4220094.18
## CWalks 4.8792321 1313376.46
## League -0.8535780 65399.13
## Division -2.1251482 40326.74
## PutOuts 5.9370363 2881512.66
## Assists 0.7219035 387572.61
## Errors 0.3059006 383018.71
## NewLeague 1.5139436 70309.77varImpPlot(Bagging.Hit)
Test.Error.Hit.Bagging <- mean((pred.Bagging.Hit-Test.Hit$Salary)^2)
Test.Error.Hit.Bagging## [1] 89596.78This question uses the Caravan data set.
- Create a training set consisting of the first 1,000 observations, and a test set consisting of the remaining observations.
Cvan <- Caravan
Cvan$Purchase## [1] No No No No No No No No No No No No No No No No No
## [18] No No No No No No No No No No No No No No No No No
## [35] No No No No No No No Yes No No No Yes No No No No No
## [52] No No No No No No Yes No No No No No No No No No No
## [69] No No No No No No No No No No No No No No No No No
## [86] No No No No No No No No No No No No Yes Yes No No No
## [103] No No No No No No No No No No No No No No No No No
## [120] No No No No No No No No Yes No No No No No No No No
## [137] No No No No No No No No No No No No No Yes Yes No No
## [154] No No No No No No No No No No No No No No No No No
## [171] No No No No Yes No No No No Yes No No No No No No No
## [188] No No No No No No No Yes No No No No No No No No Yes
## [205] No No No No No No No No No No No No No No No No No
## [222] No No No No No Yes No No No No No No No Yes Yes No No
## [239] No No No No No No No No Yes No Yes No No No No Yes No
## [256] No No No No No No No No No No No No No No No No No
## [273] No No No No No No No No No Yes No No No No No No No
## [290] No No No No No No No No Yes No No No No No No No No
## [307] No No No No No No No Yes No No No No Yes No No No No
## [324] No No No No No No No No No No No No Yes Yes No No No
## [341] No No No No No No No No No No No No No No No No Yes
## [358] No No No No No No No No No No No No No No No No No
## [375] No No No No No No No No No No No No No No No No No
## [392] No No No No No No No No No Yes No No Yes No No No No
## [409] No No No No No No No No No No No No No No No No No
## [426] No Yes No No No No No No No No No No No No Yes No No
## [443] No No Yes No No No No No No No No No No No No No No
## [460] No No No No No No No No No No No No No No No No No
## [477] No No No No Yes No No No No No No No No No No No No
## [494] No No No No No No No No No No No Yes No No No No No
## [511] Yes No No No No No No No No No No No No No No No No
## [528] No No No No No No No No No No No No No No No No No
## [545] No No No No No No No No Yes No No No No No No No Yes
## [562] No No No No No No No No No No No No No No No No No
## [579] No No No No No No No No No No No No No No No No No
## [596] No No No No No No No No No No No No No No No No No
## [613] No No No No No No No No Yes No No No No No Yes No No
## [630] No No No No No No No No No No No No No No No No No
## [647] No No No No No No No No No No No No No No No Yes No
## [664] No No No No No No Yes Yes No No No No No No No No No
## [681] No No No No Yes No No No No No No No No No No No No
## [698] No No No No No No No Yes No No No No No No No No No
## [715] No No No No No No No No No No No No No No No No No
## [732] No No No No No No No No No No No No No No No No No
## [749] No No No No No No No No No No No No No No Yes No No
## [766] No No No No No Yes No No Yes No No No No No No No No
## [783] No No No No No No No No No No No No No No No No No
## [800] Yes No No No No No No No No No No No Yes No No No Yes
## [817] No No No No No No No No No No No No No No No No No
## [834] No Yes No No Yes No No No No No No No No No No No No
## [851] No No No No No No Yes Yes No No No No No No No No No
## [868] No No Yes No Yes No No No No No No No No No No No No
## [885] No No No No No No No No No No No No No No No No No
## [902] No No No No No No No No No No No No No No No No No
## [919] No No No No No No Yes No No No No No No No No No No
## [936] No No Yes No No No No Yes No No No No No No No No No
## [953] No No No No No Yes No No No No No No No No No No No
## [970] No No No No No No No No No No No Yes No No No No No
## [987] No No No No No No No No No No No No No No No Yes No
## [1004] No No Yes No No No No No No No No No No No No No No
## [1021] No No No No No No No No No Yes No No No No No No No
## [1038] No No No No No No No No No No No No No No No No No
## [1055] No No No No No No Yes No No No No No No No No No No
## [1072] No No No Yes No No No No No No Yes No No No No No No
## [1089] No No No No No No No No No No No No No No Yes No No
## [1106] No No No No No No No No No No No No No Yes No No No
## [1123] No No Yes No No No No No No No No No No No No No No
## [1140] No No Yes Yes No No No No No No No No No No No No No
## [1157] No No No No No No No No No No No No No No No No No
## [1174] No No No No No No No No No No No No No No No No No
## [1191] No No No No No No No Yes No No No No No No No Yes No
## [1208] No No No No No No No No No Yes No No No No No No No
## [1225] No No No No No No No No No No No No No No No No No
## [1242] No No No No No Yes No No No No No No No No Yes No No
## [1259] No No No No No No No No No No Yes Yes No No No No No
## [1276] No No No No No No No No No No No No No No No No No
## [1293] No No No No No No No No No Yes No No No No No No No
## [1310] No No No No No No Yes No No No Yes No No No No No No
## [1327] No No Yes No No No No No No No No No No No Yes No No
## [1344] No No No No No No No Yes No No No No No No No No No
## [1361] No No No No Yes No No No No Yes No No No No No No No
## [1378] No No No No No No No No No Yes Yes No No No No No Yes
## [1395] No No No No No No No No No Yes No No No No No No No
## [1412] Yes No No No No No No No No No No No No No No No No
## [1429] No No No No No No No No No No No No No No No No No
## [1446] No No No No No No No No No No No No No No No No No
## [1463] No No No No No No No No No No No No No No No No No
## [1480] Yes No No No No No No No No No Yes No No No No No No
## [1497] No No No No No Yes No No No No No No No No No No No
## [1514] No No No No No No No No No No No No No No No No No
## [1531] No No No No No No No No No No No No No No No No No
## [1548] No No Yes No No No Yes No No No Yes No No No No No No
## [1565] No No No No No No No No No No No No No No No No No
## [1582] No No No No No No No No No No No Yes No No No No No
## [1599] No No No No No No No No No No No No No No No No No
## [1616] No No Yes Yes No No No Yes Yes No No No Yes No No No No
## [1633] No No Yes No No No No No No No No No No No No No No
## [1650] No No No No No No No No Yes No No No No Yes No No No
## [1667] No Yes No No No Yes No No No No Yes No No No No No No
## [1684] No No No No No No No No No No No No No No No No Yes
## [1701] No No No No No No No No No No No No No No No No No
## [1718] No No No No No No No No No No No No No No No No No
## [1735] No No No No No No Yes No No No No No No No No No No
## [1752] No No No No No No No No No No No No Yes No No No No
## [1769] No No No No No No No No No No No No No No No No No
## [1786] No No No No No No No No No No No No No No No No No
## [1803] No No Yes No No No No No No No No No No No No No No
## [1820] No No No No No No No No No No No No No No No No No
## [1837] No No No Yes No No No No No No No No No No No No No
## [1854] No No No No No No No No No No No No No No No No No
## [1871] No No Yes No No No No Yes No No No No No No No No No
## [1888] No No No No Yes No No No No No No Yes No No No No No
## [1905] No No No No No No No No No No No No No No No Yes No
## [1922] No No Yes No No No No No No No No No No No Yes No No
## [1939] No No No No No No No No No No No No No No No No No
## [1956] No No No No No No No No No No No No No No No No No
## [1973] No No No No No No No No No No No No No No No No No
## [1990] No No Yes No No No No No No No No No No No No No No
## [2007] No No No No No No No No Yes No No No No No No No No
## [2024] No No No No Yes No No No No No No No Yes No No No No
## [2041] No No No No No No No No No No No No No No No No No
## [2058] No No No No No No No Yes No No No No No No No No No
## [2075] No No No No No No No No No No No No No No No No No
## [2092] No No No No Yes No No Yes No No No No No No No No No
## [2109] No No No No No No No No No No Yes No No No No No No
## [2126] No No No No No No No No No No No No No No No No Yes
## [2143] No No No No No No No No No No Yes No No No No No No
## [2160] No No No No No No No No No No No No No No No No No
## [2177] No No No No No No No No No No No No No No No No No
## [2194] No No No No No No No No No No No No No No No No No
## [2211] No No No No No No No No No No No No Yes No No No No
## [2228] No No No No No No No No No No No No No No No No No
## [2245] No No No No No No No No No No No No No No No No No
## [2262] No No No No No No No No No No No No No No No Yes No
## [2279] No No No No No No No No Yes No No No No No Yes No No
## [2296] No No No No No No No No No No No No No No No No No
## [2313] No No No No No No No No Yes No No No No No No No No
## [2330] No No No No No No No No No No No No No No No No No
## [2347] No No No No No No No No Yes No No No No No No No No
## [2364] No No No No No No No No No No No No No No No No No
## [2381] No No No No No No No No No No No No No No No No No
## [2398] No No No No No No No No No No No No No No No No No
## [2415] No No No No No No No No No No No No No No No No No
## [2432] No No No No No No Yes No No No No No No No No No No
## [2449] Yes No Yes No No No No No Yes No No No No No No No No
## [2466] No No No No No No No No No No No No No No No No No
## [2483] No No No No No No No No No No No No Yes No No No No
## [2500] No No No No No No No No Yes No No No No No No No No
## [2517] No No No No No No No No No No No No Yes No No No No
## [2534] No No No No No No No No No No No No No No No No No
## [2551] No No No No No No No No No No No No No No No No No
## [2568] No No No No No No No No No No No No No Yes No No No
## [2585] No Yes No No No No No No No No No No No No No No No
## [2602] No No No No No No Yes No No No No No No No No No No
## [2619] No No No No No No No No No No No No Yes No No Yes Yes
## [2636] No No No No No No No No No No No No No No No No No
## [2653] No No No No No No No No No No Yes No Yes No Yes No No
## [2670] No No No No Yes No No No No No No No No No Yes No No
## [2687] No No No No No No No No No No No No No No No No No
## [2704] No No No No No No No No No No No No No No No No No
## [2721] No No No No No No No No No No No No No No No No No
## [2738] No No No No No No No Yes No No No No No No No No No
## [2755] No No No No No Yes No No Yes No No No No No No Yes No
## [2772] No No No No No No No No Yes No No No No Yes No No No
## [2789] No No No Yes No No No No No Yes No No No No Yes No No
## [2806] No No No No Yes No No No Yes No No No No No No No No
## [2823] No Yes No No No No No No No No No No No No No No No
## [2840] No No No No No No No No No No No No No No No No No
## [2857] Yes No Yes No No No No No No No No No No No No No No
## [2874] No No No No No Yes No No No No No No No No No No No
## [2891] No No No No Yes No Yes No No No No No No No No No No
## [2908] No No No No No No No No No No No No No No No No No
## [2925] No No No No No No No No No No Yes No No Yes No No No
## [2942] No No No No No No No No No No No No Yes No No No No
## [2959] No No No No No No No No No No No No No No No No No
## [2976] No No No No No No Yes No No No No No No No No No No
## [2993] No No No No No No No No No No No No No No No No No
## [3010] No No No No No No No No No No No No No No No No No
## [3027] No No Yes No No No No No No No No No No No No No No
## [3044] No No No No No No No No No No No No Yes No No No No
## [3061] No No No No No No No No No No No No No No No No No
## [3078] No No No No No No Yes No No No No Yes No No No No No
## [3095] No No No No No No No No No Yes No No No No No No No
## [3112] No No No No No No No No No No Yes No No No No No No
## [3129] No No No No No No No No No No No No No No No No No
## [3146] Yes No No No No No No No No No No No No No No No No
## [3163] No No No No No No No No No Yes No No No No No No No
## [3180] No No No No No No No Yes No No No No No No No No No
## [3197] No No No No No No No No No No No No Yes No No No No
## [3214] No No No No Yes No No No No No No No No No No No No
## [3231] No No No No No Yes Yes No No No No Yes No No Yes No No
## [3248] No No No No No No No No No No No No No No No No No
## [3265] No No No No No Yes No No No Yes Yes No No No No No No
## [3282] No No No No No No No No No No No No No No No Yes No
## [3299] No No No No No Yes No No No No No Yes No No No No No
## [3316] No No No No No No No No No No No No No No No No No
## [3333] No No No No No No No No No No No No No No No No No
## [3350] No No No No No No No No No No No Yes No No No No No
## [3367] No No No No No No No No No No No No No No No No No
## [3384] No No No No No Yes No No No No No No No No No No No
## [3401] No No No No No No No No No No No No No No No No No
## [3418] No No No No No No No No No No No No No No No No No
## [3435] No No No No No No No No No No No No No No No No No
## [3452] No No No No No No No No No No Yes No No No Yes No No
## [3469] Yes No No No No No No No Yes No No No No No No No No
## [3486] No No No Yes No No No No No No No No No No No No No
## [3503] No No No No Yes No Yes Yes No No No No No No No Yes No
## [3520] No No No No No No No No No No No No No No No No No
## [3537] No No No No No No No No Yes No No No No No No No No
## [3554] No No No Yes No No No No No No No No No No No No No
## [3571] No Yes No No No No No No No No No No No No No No No
## [3588] No No No No No No No No No No Yes No No No Yes No No
## [3605] No No No No No No No No No No No No No No No No No
## [3622] No No No No No Yes No No No No No No No No No No No
## [3639] No No Yes No No No No No No No No No No No No No No
## [3656] No No No No No No No No No No No Yes Yes No No No No
## [3673] No No No No No No No No No No No No No No No No No
## [3690] No No No No No No No No No No No No No No Yes No No
## [3707] No No No No No No No No No No No No No No No No No
## [3724] No No No No No No No Yes Yes No No No No No No No No
## [3741] No Yes No No No No No No No No No No No Yes No No No
## [3758] No No No No No Yes No No No No No No No No No No No
## [3775] No No No Yes No No No No Yes No No No No No No No Yes
## [3792] No No No No No No No No No No No No No No No No No
## [3809] No No No No Yes No No No No No No No No No Yes No No
## [3826] No No No No No No No No No Yes No No No No No No No
## [3843] No No No No No No No No No No No No No No No No No
## [3860] No No No No No No No Yes No No No No No No No No No
## [3877] No No No No No No No No No No No No No No No No No
## [3894] No No No No No No No No No No No No No No No No No
## [3911] No No No No No No No Yes No No No No No No No No No
## [3928] No No No No No No No No No No No No No No No No No
## [3945] No No No No No No Yes No No No No No No No No No No
## [3962] No No No No No No No No No No No No No Yes No No No
## [3979] No No No No No No No No No No No No No No No No No
## [3996] No No No No Yes No No Yes No No No No No No No No No
## [4013] No No No No No No No No No No No No No No No Yes No
## [4030] No No No No No No No No Yes No No No No No No No No
## [4047] No No No No No No No Yes No No No No No No No No No
## [4064] No No No No No Yes No No No Yes No No No No No No No
## [4081] No No No No No No No No No No No No No No No No No
## [4098] No No No No No No No No Yes No No No No No No No No
## [4115] No No No No No No No No No No No No No No No No No
## [4132] No No No No No No No No No No No No No No Yes No No
## [4149] No No No No No No Yes No No No No Yes No No No No No
## [4166] No No No No Yes No No No No No No No No Yes No No No
## [4183] No No No No No No No No No No No No Yes No No No No
## [4200] No No No No No No No No No No No No No Yes No No Yes
## [4217] No No No No No No No No No No No No No No No No No
## [4234] No Yes No No No No No No No No Yes Yes No No No No No
## [4251] No Yes No No No No No No No No No No No No No No No
## [4268] No No No No No No No No No No No No No No No No No
## [4285] No No No No No Yes No No No No No No Yes No No No No
## [4302] No No No No No No No No No No No Yes No No No No No
## [4319] No No No No No Yes No No No No No No No No No No No
## [4336] No No No No No No No No Yes No No No No No No No No
## [4353] No No No No No No No No No No Yes No Yes No No No No
## [4370] No No No No No No No No No No No No No No No No No
## [4387] No No No No No No No No No No No No No No No No No
## [4404] No No No No No No No No No No No No No No Yes No No
## [4421] No Yes No Yes No No No No No No No No No No No No No
## [4438] No No No No No No Yes No No No No No No No No No No
## [4455] No No No Yes No No No No No No No No No No No No No
## [4472] No No No No No No No No No No No No No No No No No
## [4489] No No No No No No No No No No No No No No No No No
## [4506] No No No No No No No No No No No No No No No No No
## [4523] No No No No No No No No No No No No No Yes Yes No No
## [4540] No No No No No No No No No No No No No No No No No
## [4557] No No No No No No No No No No No No No No No No No
## [4574] No No No No No No No No No No No Yes No No No No Yes
## [4591] No No No No No No No No No No No No No No No No Yes
## [4608] No No No No No No No No No Yes No No No No No Yes No
## [4625] No No No No No No No No No No No No No No No No No
## [4642] No No No No No No No No No No No No No No No No Yes
## [4659] No No No No No No No No No No No No No No No No No
## [4676] No No No No No No No No No No No No No No No No No
## [4693] No No No No No No Yes No No No No No No No No No No
## [4710] No No No No Yes Yes No No No No No No Yes No No No No
## [4727] No No No Yes No Yes No No No No No No No No No No No
## [4744] No No No No No No No No No No No No No No No No No
## [4761] No Yes No No No No No No No No No No No Yes No No No
## [4778] No No No No No No No No No No No No Yes No No No No
## [4795] No No No No No No No No No No No No No No No No No
## [4812] No No No No No No No No No No No No No No No No No
## [4829] No No No No No No No No No No No No No No No Yes No
## [4846] No No No No No No No No No No No No No No No No No
## [4863] No No No No No No No No No No No No No No No No No
## [4880] No No Yes No No No No No No No No No No No No No No
## [4897] Yes No No No No No No No No No No No No No No Yes No
## [4914] No No No No No No No No No No No No No No Yes No No
## [4931] No No No No No No No No No No No No Yes No No Yes No
## [4948] No No No No No No No No No No No No No No Yes No No
## [4965] No No No No No No No No No No No No No Yes Yes No No
## [4982] No No No No No No No No No No No No No Yes No No No
## [4999] No No No No No Yes No No No No No No No No No No Yes
## [5016] No No No No No No No No No No No No No Yes No No No
## [5033] Yes No No No No No No Yes No No No No No No No No No
## [5050] No No No No No No No No No Yes No No No No No No No
## [5067] No Yes No No No No No No No No No No No No No No No
## [5084] No No No No No No No No No No No No No No No No No
## [5101] No No No No No No No No No No No No No No No No No
## [5118] No No No No No No No No No No No No No No No No No
## [5135] No No Yes Yes No No No No No No No No No No No No No
## [5152] No Yes No No No No No No No No No No Yes No No No No
## [5169] Yes No Yes No No No No Yes No No No No No No No No Yes
## [5186] No No No No No No No No No No Yes No No No No No No
## [5203] No No No No No No No No No No No No No No Yes No No
## [5220] No No No Yes No Yes No No No No No Yes No Yes No No Yes
## [5237] No No No No No No No No No No No No No No No No No
## [5254] No No No No No No No No No No No No No No No No No
## [5271] No No No No No Yes No No No No No No No No No No No
## [5288] No No No No No No No No No No No No No No No No No
## [5305] No No No Yes No No Yes No No No No Yes No No No No Yes
## [5322] No No No No No No No No No No No No No No Yes No No
## [5339] No No No No Yes No No No No No No No No No No No No
## [5356] No No No No No No Yes No No No No No No No Yes No No
## [5373] No No No No No No Yes No No No No No No No No No No
## [5390] No No No No No No Yes No No No No No No No No No No
## [5407] No No No Yes No No No No No No No No No No No No No
## [5424] No No No No Yes No No No No No No No No No No No No
## [5441] Yes No Yes No No No No No No No No No No No No No Yes
## [5458] No No No No No No No No No No No No No No No No No
## [5475] No No No No No No No Yes No No No No No No No No No
## [5492] No No No No No No No No No No No No No No No No No
## [5509] Yes No No No No No No No No No No No No No No Yes No
## [5526] No No No No No No Yes No No No No No No No No No No
## [5543] No No No No No No No No No No No No No No No No Yes
## [5560] No No No No No No No No No No No No No No No No No
## [5577] No No No Yes No No No No No No No No No No No No No
## [5594] No No No No No No No No No No No No No No No No No
## [5611] No No No No No No No No No No No No No No No No No
## [5628] No No No No Yes No No No No No No No No No No No No
## [5645] No No No No No No No No No No No No No No No No No
## [5662] No No No No No No Yes No No No No No No No No No No
## [5679] No No No No No No No No No No No No No No No No No
## [5696] No No No No No No No Yes No No No No No No No No Yes
## [5713] No No No No No No No No Yes No No No No No No No No
## [5730] No No No No No No No No No Yes No Yes No No No No No
## [5747] No No No No No No No No No No Yes No No No No No No
## [5764] No No No No Yes No No No No No No No No No No No No
## [5781] No No No No No No No Yes No No No No No No No No No
## [5798] Yes No No No No No No No No No No No No No No No No
## [5815] No No No No No Yes No No
## Levels: No YesCvan <- Cvan %>%
mutate(Purchase = ifelse(Purchase == "Yes", 1,0))
head(Cvan)## MOSTYPE MAANTHUI MGEMOMV MGEMLEEF MOSHOOFD MGODRK MGODPR MGODOV MGODGE
## 1 33 1 3 2 8 0 5 1 3
## 2 37 1 2 2 8 1 4 1 4
## 3 37 1 2 2 8 0 4 2 4
## 4 9 1 3 3 3 2 3 2 4
## 5 40 1 4 2 10 1 4 1 4
## 6 23 1 2 1 5 0 5 0 5
## MRELGE MRELSA MRELOV MFALLEEN MFGEKIND MFWEKIND MOPLHOOG MOPLMIDD
## 1 7 0 2 1 2 6 1 2
## 2 6 2 2 0 4 5 0 5
## 3 3 2 4 4 4 2 0 5
## 4 5 2 2 2 3 4 3 4
## 5 7 1 2 2 4 4 5 4
## 6 0 6 3 3 5 2 0 5
## MOPLLAAG MBERHOOG MBERZELF MBERBOER MBERMIDD MBERARBG MBERARBO MSKA
## 1 7 1 0 1 2 5 2 1
## 2 4 0 0 0 5 0 4 0
## 3 4 0 0 0 7 0 2 0
## 4 2 4 0 0 3 1 2 3
## 5 0 0 5 4 0 0 0 9
## 6 4 2 0 0 4 2 2 2
## MSKB1 MSKB2 MSKC MSKD MHHUUR MHKOOP MAUT1 MAUT2 MAUT0 MZFONDS MZPART
## 1 1 2 6 1 1 8 8 0 1 8 1
## 2 2 3 5 0 2 7 7 1 2 6 3
## 3 5 0 4 0 7 2 7 0 2 9 0
## 4 2 1 4 0 5 4 9 0 0 7 2
## 5 0 0 0 0 4 5 6 2 1 5 4
## 6 2 2 4 2 9 0 5 3 3 9 0
## MINKM30 MINK3045 MINK4575 MINK7512 MINK123M MINKGEM MKOOPKLA PWAPART
## 1 0 4 5 0 0 4 3 0
## 2 2 0 5 2 0 5 4 2
## 3 4 5 0 0 0 3 4 2
## 4 1 5 3 0 0 4 4 0
## 5 0 0 9 0 0 6 3 0
## 6 5 2 3 0 0 3 3 0
## PWABEDR PWALAND PPERSAUT PBESAUT PMOTSCO PVRAAUT PAANHANG PTRACTOR
## 1 0 0 6 0 0 0 0 0
## 2 0 0 0 0 0 0 0 0
## 3 0 0 6 0 0 0 0 0
## 4 0 0 6 0 0 0 0 0
## 5 0 0 0 0 0 0 0 0
## 6 0 0 6 0 0 0 0 0
## PWERKT PBROM PLEVEN PPERSONG PGEZONG PWAOREG PBRAND PZEILPL PPLEZIER
## 1 0 0 0 0 0 0 5 0 0
## 2 0 0 0 0 0 0 2 0 0
## 3 0 0 0 0 0 0 2 0 0
## 4 0 0 0 0 0 0 2 0 0
## 5 0 0 0 0 0 0 6 0 0
## 6 0 0 0 0 0 0 0 0 0
## PFIETS PINBOED PBYSTAND AWAPART AWABEDR AWALAND APERSAUT ABESAUT AMOTSCO
## 1 0 0 0 0 0 0 1 0 0
## 2 0 0 0 2 0 0 0 0 0
## 3 0 0 0 1 0 0 1 0 0
## 4 0 0 0 0 0 0 1 0 0
## 5 0 0 0 0 0 0 0 0 0
## 6 0 0 0 0 0 0 1 0 0
## AVRAAUT AAANHANG ATRACTOR AWERKT ABROM ALEVEN APERSONG AGEZONG AWAOREG
## 1 0 0 0 0 0 0 0 0 0
## 2 0 0 0 0 0 0 0 0 0
## 3 0 0 0 0 0 0 0 0 0
## 4 0 0 0 0 0 0 0 0 0
## 5 0 0 0 0 0 0 0 0 0
## 6 0 0 0 0 0 0 0 0 0
## ABRAND AZEILPL APLEZIER AFIETS AINBOED ABYSTAND Purchase
## 1 1 0 0 0 0 0 0
## 2 1 0 0 0 0 0 0
## 3 1 0 0 0 0 0 0
## 4 1 0 0 0 0 0 0
## 5 1 0 0 0 0 0 0
## 6 0 0 0 0 0 0 0iTrain.Cvan <- createDataPartition(y = Cvan$Purchase, p = 1000/nrow(Cvan), list = FALSE)
Train.Cvan <- Cvan[iTrain.Cvan, ]
Test.Cvan <- Cvan[-iTrain.Cvan, ]- Fit a boosting model to the training set with Purchase as the response and the other variables as predictors. Use 1,000 trees, and a shrinkage value of 0.01. Which predictors appear to be the most important?
Boosting.Cvan <- gbm(Purchase ~. , distribution = "bernoulli", data = Train.Cvan, shrinkage = 0.01, n.trees = 1000)## Warning in gbm.fit(x, y, offset = offset, distribution = distribution, w =
## w, : variable 50: PVRAAUT has no variation.## Warning in gbm.fit(x, y, offset = offset, distribution = distribution, w =
## w, : variable 71: AVRAAUT has no variation.summary(Boosting.Cvan)
## var rel.inf
## PBRAND PBRAND 16.34497996
## PPERSAUT PPERSAUT 11.69863366
## MOPLMIDD MOPLMIDD 4.59932541
## MOSTYPE MOSTYPE 4.51737978
## MBERZELF MBERZELF 4.27435038
## MINK3045 MINK3045 4.12528842
## MAUT1 MAUT1 4.02006024
## MOPLHOOG MOPLHOOG 3.69022299
## PMOTSCO PMOTSCO 3.43270550
## MBERMIDD MBERMIDD 3.01411980
## MINK7512 MINK7512 2.97212055
## APERSAUT APERSAUT 2.95051731
## MINKGEM MINKGEM 2.88098170
## MSKC MSKC 2.68034002
## MSKA MSKA 2.59458701
## MRELGE MRELGE 2.56735798
## MBERARBG MBERARBG 2.16815748
## PWAPART PWAPART 2.06367389
## MGODRK MGODRK 1.56545663
## PLEVEN PLEVEN 1.48410817
## MKOOPKLA MKOOPKLA 1.48068239
## MOPLLAAG MOPLLAAG 1.23913142
## MBERHOOG MBERHOOG 1.06586733
## MINKM30 MINKM30 1.03010019
## MGEMOMV MGEMOMV 0.96931282
## MINK4575 MINK4575 0.93532620
## MSKB1 MSKB1 0.90857191
## MGODPR MGODPR 0.89218721
## MFALLEEN MFALLEEN 0.81706122
## MOSHOOFD MOSHOOFD 0.80260762
## MAUT0 MAUT0 0.66547876
## MRELOV MRELOV 0.52910300
## MAUT2 MAUT2 0.50573536
## MINK123M MINK123M 0.50245356
## MFGEKIND MFGEKIND 0.47403681
## MHKOOP MHKOOP 0.43929100
## MFWEKIND MFWEKIND 0.40582151
## MZFONDS MZFONDS 0.39498770
## ALEVEN ALEVEN 0.38200313
## MHHUUR MHHUUR 0.33183983
## MBERARBO MBERARBO 0.27971469
## MGODGE MGODGE 0.25312084
## MZPART MZPART 0.24953448
## MGODOV MGODOV 0.22001136
## MSKB2 MSKB2 0.18213667
## PBYSTAND PBYSTAND 0.12259418
## MGEMLEEF MGEMLEEF 0.07979921
## MBERBOER MBERBOER 0.07259433
## MSKD MSKD 0.06717401
## MRELSA MRELSA 0.05735437
## MAANTHUI MAANTHUI 0.00000000
## PWABEDR PWABEDR 0.00000000
## PWALAND PWALAND 0.00000000
## PBESAUT PBESAUT 0.00000000
## PVRAAUT PVRAAUT 0.00000000
## PAANHANG PAANHANG 0.00000000
## PTRACTOR PTRACTOR 0.00000000
## PWERKT PWERKT 0.00000000
## PBROM PBROM 0.00000000
## PPERSONG PPERSONG 0.00000000
## PGEZONG PGEZONG 0.00000000
## PWAOREG PWAOREG 0.00000000
## PZEILPL PZEILPL 0.00000000
## PPLEZIER PPLEZIER 0.00000000
## PFIETS PFIETS 0.00000000
## PINBOED PINBOED 0.00000000
## AWAPART AWAPART 0.00000000
## AWABEDR AWABEDR 0.00000000
## AWALAND AWALAND 0.00000000
## ABESAUT ABESAUT 0.00000000
## AMOTSCO AMOTSCO 0.00000000
## AVRAAUT AVRAAUT 0.00000000
## AAANHANG AAANHANG 0.00000000
## ATRACTOR ATRACTOR 0.00000000
## AWERKT AWERKT 0.00000000
## ABROM ABROM 0.00000000
## APERSONG APERSONG 0.00000000
## AGEZONG AGEZONG 0.00000000
## AWAOREG AWAOREG 0.00000000
## ABRAND ABRAND 0.00000000
## AZEILPL AZEILPL 0.00000000
## APLEZIER APLEZIER 0.00000000
## AFIETS AFIETS 0.00000000
## AINBOED AINBOED 0.00000000
## ABYSTAND ABYSTAND 0.00000000pred.Boosting.Cvan <- predict(Boosting.Cvan, newdata = Test.Cvan, n.trees = 1000, type = "response")
BN.pred.Boosting.Cvan <- rep("Yes",length(pred.Boosting.Cvan))
BN.pred.Boosting.Cvan[pred.Boosting.Cvan < .2] <- "No"
Test.Cvan$Purchase[Test.Cvan$Purchase == 1] <- "Yes"
Test.Cvan$Purchase[Test.Cvan$Purchase == 0] <- "No"
TB.Boosting.Cvan <- table(Test.Cvan$Purchase, BN.pred.Boosting.Cvan)
TB.Boosting.Cvan; (TB.Boosting.Cvan[1,1]+ TB.Boosting.Cvan[2,2])/sum(TB.Boosting.Cvan)## BN.pred.Boosting.Cvan
## No Yes
## No 4301 236
## Yes 238 47## [1] 0.9017005