Ensemble Approaches

Algorithms which combine multiple ML techniques into one predictive model to:
1. Decrease Variance
2. Decrease Bias
3. Improve Predictions

---

Example:

Example

---

Popular Ensemble Methods

Bagging - Bootstrap Aggregation

Multiple Models of same learning algorithm trained with subsets of dataset randomly picked from the data set.

Note: The basic idea of Bootstrap is randomly drawing datasets with replacement from the training data, each sample the same size as the original training set

Bagging

Boosting

1. Little Variation on Bagging
   
2. Selecting points which give wrong predictions

Boosting

Summary

Rather than making one model and hoping this model is the best/most accurate predictor we can make, ensemble methods take a myriad of models into account, and average those models to produce one final model.