Assignment 6 - Predictive Modeling
Matthew Reyes
2026-07-17
Question 6
- Perform polynomial regression to predict wage using age. Use cross-validation to select the optimal degree d for the polynomial. What degree was chosen, and how does this compare to the results of hypothesis testing using ANOVA? Make a plot of the resulting polynomial fit to the data.
ANSWER: I ended up choose d = 6, as it was the lowest RMSE across 10 folds using CV. Interestingly, it looks like ANOVA favors d = 2 or 3, as those are the most statistically significant changes to the model. This tells us that polynomial = 6 gives us the best predictive accuracy, I think. However, ANOVA shows that polynomial = 2 or 3 explains the most variation in the model.
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 111.70361 0.7287409 153.283015 0.000000e+00
## poly(age, 4)1 447.06785 39.9147851 11.200558 1.484604e-28
## poly(age, 4)2 -478.31581 39.9147851 -11.983424 2.355831e-32
## poly(age, 4)3 125.52169 39.9147851 3.144742 1.678622e-03
## poly(age, 4)4 -77.91118 39.9147851 -1.951938 5.103865e-02
## [1] 3
## Analysis of Variance Table
##
## Model 1: wage ~ age
## Model 2: wage ~ poly(age, 2)
## Model 3: wage ~ poly(age, 3)
## Model 4: wage ~ poly(age, 4)
## Res.Df RSS Df Sum of Sq F Pr(>F)
## 1 2998 5022216
## 2 2997 4793430 1 228786 143.6025 < 2.2e-16 ***
## 3 2996 4777674 1 15756 9.8894 0.001679 **
## 4 2995 4771604 1 6070 3.8101 0.051039 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
- Fit a step function to predict wage using age, and perform cross- validation to choose the optimal number of cuts. Make a plot of the fit obtained.
ANSWER: The optimal number of cuts is 8. See graph below.
## The following objects are masked from Wage (pos = 3):
##
## age, education, health, health_ins, jobclass, logwage, maritl,
## race, region, wage, year##
## (17.9,33.5] (33.5,49] (49,64.5] (64.5,80.1]
## 750 1399 779 72## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 94.158392 1.476069 63.789970 0.000000e+00
## cut(age, 4)(33.5,49] 24.053491 1.829431 13.148074 1.982315e-38
## cut(age, 4)(49,64.5] 23.664559 2.067958 11.443444 1.040750e-29
## cut(age, 4)(64.5,80.1] 7.640592 4.987424 1.531972 1.256350e-01
##
## (17.9,25.8] (25.8,33.5] (33.5,41.2] (41.2,49] (49,56.8] (56.8,64.5]
## 231 519 671 728 503 276
## (64.5,72.2] (72.2,80.1]
## 54 18## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 76.28175 2.629812 29.006542 3.110596e-163
## cut(age, 8)(25.8,33.5] 25.83329 3.161343 8.171618 4.440913e-16
## cut(age, 8)(33.5,41.2] 40.22568 3.049065 13.192791 1.136044e-38
## cut(age, 8)(41.2,49] 43.50112 3.018341 14.412262 1.406253e-45
## cut(age, 8)(49,56.8] 40.13583 3.176792 12.634076 1.098741e-35
## cut(age, 8)(56.8,64.5] 44.10243 3.564299 12.373380 2.481643e-34
## cut(age, 8)(64.5,72.2] 28.94825 6.041576 4.791505 1.736008e-06
## cut(age, 8)(72.2,80.1] 15.22418 9.781110 1.556488 1.196978e-01Question 10
- Split the data into a training set and a test set. Using out-of-state tuition as the response and the other variables as the predictors, perform forward stepwise selection on the training set in order to identify a satisfactory model that uses just a subset of the predictors.
ANSWER: Analysis shows 6 variables should be used: private, room.board, phd, perc.alumni, expend, and grad.rate.
## [1] "(Intercept)" "PrivateYes" "Room.Board" "Terminal" "perc.alumni"
## [6] "Expend" "Grad.Rate"- Fit a GAM on the training data, using out-of-state tuition as the response and the features selected in the previous step as the predictors. Plot the results, and explain your findings.
ANSWER: Pretty obvious that private schools correspond to higher out-of-state tuition (OOST). We see relatively linear relationships with room/board, the PhD variable, the graduation rate, and the percent of alumni who donate. The expend variable is interesting, as it shows that the relationship between that and OOST is non-linear.
- Evaluate the model obtained on the test set, and explain the results obtained.
ANSWER: The R^2 of the GAM with 6 dependent variables is ~.74, meaning the model expains about 74% of the variation in the data. Not bad!
## [1] 3454988## [1] 0.7580254- For which variables, if any, is there evidence of a non-linear relationship with the response?
ANSWER: Only the expend variable is definitely non-linear. The other are slightly curved but I would say mostly linear.
##
## Call: gam(formula = Outstate ~ Private + s(Room.Board, df = 2) + s(PhD,
## df = 2) + s(perc.alumni, df = 2) + s(Expend, df = 5) + s(Grad.Rate,
## df = 2), data = College.train)
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -7071.40 -1098.00 -26.69 1245.70 7579.79
##
## (Dispersion Parameter for gaussian family taken to be 3491295)
##
## Null Deviance: 9817819951 on 584 degrees of freedom
## Residual Deviance: 1990039096 on 570.0003 degrees of freedom
## AIC: 10490.45
##
## Number of Local Scoring Iterations: NA
##
## Anova for Parametric Effects
## Df Sum Sq Mean Sq F value Pr(>F)
## Private 1 2529435661 2529435661 724.498 < 2.2e-16 ***
## s(Room.Board, df = 2) 1 1997695571 1997695571 572.193 < 2.2e-16 ***
## s(PhD, df = 2) 1 532704677 532704677 152.581 < 2.2e-16 ***
## s(perc.alumni, df = 2) 1 423091964 423091964 121.185 < 2.2e-16 ***
## s(Expend, df = 5) 1 810371312 810371312 232.112 < 2.2e-16 ***
## s(Grad.Rate, df = 2) 1 100732212 100732212 28.852 1.139e-07 ***
## Residuals 570 1990039096 3491295
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Anova for Nonparametric Effects
## Npar Df Npar F Pr(F)
## (Intercept)
## Private
## s(Room.Board, df = 2) 1 2.2190 0.13688
## s(PhD, df = 2) 1 3.6758 0.05571 .
## s(perc.alumni, df = 2) 1 1.3952 0.23802
## s(Expend, df = 5) 4 30.2226 < 2e-16 ***
## s(Grad.Rate, df = 2) 1 2.9153 0.08829 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1