Bootstrapping Loess

Dmitriy Gorbachev
Thu Aug 14 16:09:14 2014

Introduction

The very basic idea of bagging technique in prediction analysis is[1]:

  1. Resample cases and recalculate predictions;
  2. Average the results by taking means or majority vote.

For demonstration 'ozone' dataset from 'ElemStatLearn' package was used.

  ozone radiation temperature wind
1    41       190          67  7.4
2    36       118          72  8.0

To relate ozone level to air temperature Loess regression with bagging was performed.

Study Design

Steps:

  1. The 'ozone' dataset rows were resampled with replacement.
  2. Loess regression models were calculated on bootstrap samples.
  3. Bootstrap Loess Models were applied to the entire dataset.
  4. Predicted air temperature values were stored in a matrix.
  5. The averages of the predicted values against the ozone level values were calculated.
  6. The original data, bootstrapped Loess curves and the averaged (bagged) prediction curve were superimposed on a single ozone-temperature plot.

Demonstration

Ozone-remperature plot with 10 bootstrap Loess predictions (grey) and their average (red):

plot of chunk unnamed-chunk-2

References

  1. Coursera, John Hopkins University “Practical Machine Learning” URL: https://class.coursera.org/predmachlearn-003/auth/auth_redirector?type=login&subtype=normal

  2. “The Elements of Statistical Learning” Page. URL: http://statweb.stanford.edu/~tibs/ElemStatLearn/ Accessed 8/14/2014

  3. Wikipedia, Local regression (LOESS) Page. URL: http://en.wikipedia.org/wiki/LOESS

  4. Wikipedia, Bootstrapping (statistics) Page. URL: http://en.wikipedia.org/wiki/Bootstrapping_%28statistics%29 Accessed 8/14/2014

  5. Shiny by RStudio.URL: http://shiny.rstudio.com