AdaBoosting

Ada Boosting

The AdaBoosting is a tecnique to create a aggreagate model which takes accounts of all the errors of weaklearners and inturn create a model to prediction model for Classification.

The Main Ideas Behind AdaBoosting:

• Ada Boost combines lots of weak learners and forms a strong classifier. The Weak learners are always stumps.

• Some stumps (weak learners) have a lot od say in final classification than others.

• Each stump mistake is taken into account while creating next stump.

suppressWarnings(library(adabag))
suppressMessages(library(caret))
data("iris")

Creating Training and Test Data

set.seed(42)


index <- createDataPartition(iris$Species , p =0.7, list = FALSE)

train.iris <- iris[index,]
test.iris <- iris[-index,]

Create the Model without Cross Validation with the help of Boosting Function.

model = boosting(Species~., data = train.iris, boos = TRUE, mfinal = 50)

Prediction of the Test Data

pred = predict(model , test.iris)

pred$confusion

##                Observed Class
## Predicted Class setosa versicolor virginica
##      setosa         15          0         0
##      versicolor      0         14         2
##      virginica       0          1        13

The Prediction Error is 0.0666667

Training the Data with Boosting with Cross Validation

cvmodel = boosting.cv(Species ~., data = iris, boos = TRUE, mfinal = 10, v =5)

## i:  1 Sat May 02 14:12:55 2020 
## i:  2 Sat May 02 14:12:58 2020 
## i:  3 Sat May 02 14:13:02 2020 
## i:  4 Sat May 02 14:13:05 2020 
## i:  5 Sat May 02 14:13:09 2020

cvmodel$confusion

##                Observed Class
## Predicted Class setosa versicolor virginica
##      setosa         50          0         0
##      versicolor      0         45         4
##      virginica       0          5        46

The Boosting with Cross Validation error rate = 0.06