Polynomial Ridge Regression App
Jonathan Dorsey
June 7, 2018
What is the app?
- The polynomial ridge regession app is a teaching app.
- Shows how high degree polynomials can badly overfit.
- Shows how regularization can rein in overfitting.
- Interactive sliders give students a hands-on feel for what ridge regression does.
What is polynomial ridge regression?
- Given a reponse \( y \) and a predictor \( x \), fit \( y = \sum_0^n \beta_i x^i \) via least squares.
- Ridge regression “regularizes” least squares to penalize large \( \beta_i \)'s
- Shrinks the \( \beta_i \)'s and gives a smoother fit
- Ridge regression makes the curve fit the data at hand worse!
- But hopefully makes the curve fit unseen data better.
Example of overfitting
- Degree 10 polynomial with no regularization
- This is the best fitting degree 10 polynomial in the least squares sense
- Does the price really dip before 2.5 carats?
What regularization can do
- Ridge regression has smoothed out the curve
- This looks more sensible
