1 Pendahuluan

Analisis dilakukan menggunakan data Hitters dari package ISLR2, dengan Salary sebagai peubah respon dan seluruh peubah lainnya sebagai prediktor. Tahapannya adalah: 1. Regresi linear penuh 2. Backward selection 3. Forward selection 4. Pemilihan berdasarkan BIC 5. Perbandingan model 6. Pemeriksaan diagnostik 7. Interpretasi.

2 1. Persiapan Data

library(ISLR2)
library(tidyverse)
library(broom)
library(car)
library(leaps)

data("Hitters")

dim(Hitters)
## [1] 322  20
names(Hitters)
##  [1] "AtBat"     "Hits"      "HmRun"     "Runs"      "RBI"       "Walks"    
##  [7] "Years"     "CAtBat"    "CHits"     "CHmRun"    "CRuns"     "CRBI"     
## [13] "CWalks"    "League"    "Division"  "PutOuts"   "Assists"   "Errors"   
## [19] "Salary"    "NewLeague"
head(Hitters)
str(Hitters)
## 'data.frame':    322 obs. of  20 variables:
##  $ AtBat    : int  293 315 479 496 321 594 185 298 323 401 ...
##  $ Hits     : int  66 81 130 141 87 169 37 73 81 92 ...
##  $ HmRun    : int  1 7 18 20 10 4 1 0 6 17 ...
##  $ Runs     : int  30 24 66 65 39 74 23 24 26 49 ...
##  $ RBI      : int  29 38 72 78 42 51 8 24 32 66 ...
##  $ Walks    : int  14 39 76 37 30 35 21 7 8 65 ...
##  $ Years    : int  1 14 3 11 2 11 2 3 2 13 ...
##  $ CAtBat   : int  293 3449 1624 5628 396 4408 214 509 341 5206 ...
##  $ CHits    : int  66 835 457 1575 101 1133 42 108 86 1332 ...
##  $ CHmRun   : int  1 69 63 225 12 19 1 0 6 253 ...
##  $ CRuns    : int  30 321 224 828 48 501 30 41 32 784 ...
##  $ CRBI     : int  29 414 266 838 46 336 9 37 34 890 ...
##  $ CWalks   : int  14 375 263 354 33 194 24 12 8 866 ...
##  $ League   : Factor w/ 2 levels "A","N": 1 2 1 2 2 1 2 1 2 1 ...
##  $ Division : Factor w/ 2 levels "E","W": 1 2 2 1 1 2 1 2 2 1 ...
##  $ PutOuts  : int  446 632 880 200 805 282 76 121 143 0 ...
##  $ Assists  : int  33 43 82 11 40 421 127 283 290 0 ...
##  $ Errors   : int  20 10 14 3 4 25 7 9 19 0 ...
##  $ Salary   : num  NA 475 480 500 91.5 750 70 100 75 1100 ...
##  $ NewLeague: Factor w/ 2 levels "A","N": 1 2 1 2 2 1 1 1 2 1 ...
sum(is.na(Hitters$Salary))
## [1] 59
hitters <- Hitters %>%
  drop_na()

dim(hitters)
## [1] 263  20
sum(is.na(hitters))
## [1] 0

Setelah baris yang mengandung NA dihapus, data yang digunakan adalah data lengkap. Salary digunakan sebagai peubah respon, sedangkan seluruh peubah lainnya digunakan sebagai prediktor.

3 2. Model Regresi Linear dengan Seluruh Prediktor

Model awal dibentuk menggunakan seluruh prediktor yang tersedia pada data hitters.

mod_full <- lm(
  Salary ~ .,
  data = hitters
)

summary(mod_full)
## 
## Call:
## lm(formula = Salary ~ ., data = hitters)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -907.62 -178.35  -31.11  139.09 1877.04 
## 
## Coefficients:
##               Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  163.10359   90.77854   1.797 0.073622 .  
## AtBat         -1.97987    0.63398  -3.123 0.002008 ** 
## Hits           7.50077    2.37753   3.155 0.001808 ** 
## HmRun          4.33088    6.20145   0.698 0.485616    
## Runs          -2.37621    2.98076  -0.797 0.426122    
## RBI           -1.04496    2.60088  -0.402 0.688204    
## Walks          6.23129    1.82850   3.408 0.000766 ***
## Years         -3.48905   12.41219  -0.281 0.778874    
## CAtBat        -0.17134    0.13524  -1.267 0.206380    
## CHits          0.13399    0.67455   0.199 0.842713    
## CHmRun        -0.17286    1.61724  -0.107 0.914967    
## CRuns          1.45430    0.75046   1.938 0.053795 .  
## CRBI           0.80771    0.69262   1.166 0.244691    
## CWalks        -0.81157    0.32808  -2.474 0.014057 *  
## LeagueN       62.59942   79.26140   0.790 0.430424    
## DivisionW   -116.84925   40.36695  -2.895 0.004141 ** 
## PutOuts        0.28189    0.07744   3.640 0.000333 ***
## Assists        0.37107    0.22120   1.678 0.094723 .  
## Errors        -3.36076    4.39163  -0.765 0.444857    
## NewLeagueN   -24.76233   79.00263  -0.313 0.754218    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 315.6 on 243 degrees of freedom
## Multiple R-squared:  0.5461, Adjusted R-squared:  0.5106 
## F-statistic: 15.39 on 19 and 243 DF,  p-value: < 2.2e-16
tidy(mod_full)
glance(mod_full)

3.1 Multikolinearitas

vif(mod_full)
##      AtBat       Hits      HmRun       Runs        RBI      Walks      Years 
##  22.944366  30.281255   7.758668  15.246418  11.921715   4.148712   9.313280 
##     CAtBat      CHits     CHmRun      CRuns       CRBI     CWalks     League 
## 251.561160 502.954289  46.488462 162.520810 131.965858  19.744105   4.134115 
##   Division    PutOuts    Assists     Errors  NewLeague 
##   1.075398   1.236317   2.709341   2.214543   4.099063

Nilai VIF digunakan untuk melihat adanya hubungan linear yang kuat antarpeubah prediktor. Nilai VIF yang besar menunjukkan bahwa suatu prediktor memiliki hubungan yang kuat dengan prediktor lainnya.

4 3. Backward Selection

Backward selection dimulai dari model penuh, kemudian prediktor dihapus secara bertahap berdasarkan kriteria AIC.

mod_backward <- step(
  mod_full,
  direction = "backward",
  trace = TRUE
)
## Start:  AIC=3046.02
## Salary ~ AtBat + Hits + HmRun + Runs + RBI + Walks + Years + 
##     CAtBat + CHits + CHmRun + CRuns + CRBI + CWalks + League + 
##     Division + PutOuts + Assists + Errors + NewLeague
## 
##             Df Sum of Sq      RSS    AIC
## - CHmRun     1      1138 24201837 3044.0
## - CHits      1      3930 24204629 3044.1
## - Years      1      7869 24208569 3044.1
## - NewLeague  1      9784 24210484 3044.1
## - RBI        1     16076 24216776 3044.2
## - HmRun      1     48572 24249272 3044.6
## - Errors     1     58324 24259023 3044.7
## - League     1     62121 24262821 3044.7
## - Runs       1     63291 24263990 3044.7
## - CRBI       1    135439 24336138 3045.5
## - CAtBat     1    159864 24360564 3045.8
## <none>                   24200700 3046.0
## - Assists    1    280263 24480963 3047.1
## - CRuns      1    374007 24574707 3048.1
## - CWalks     1    609408 24810108 3050.6
## - Division   1    834491 25035190 3052.9
## - AtBat      1    971288 25171987 3054.4
## - Hits       1    991242 25191941 3054.6
## - Walks      1   1156606 25357305 3056.3
## - PutOuts    1   1319628 25520328 3058.0
## 
## Step:  AIC=3044.03
## Salary ~ AtBat + Hits + HmRun + Runs + RBI + Walks + Years + 
##     CAtBat + CHits + CRuns + CRBI + CWalks + League + Division + 
##     PutOuts + Assists + Errors + NewLeague
## 
##             Df Sum of Sq      RSS    AIC
## - Years      1      7609 24209447 3042.1
## - NewLeague  1     10268 24212106 3042.2
## - CHits      1     14003 24215840 3042.2
## - RBI        1     14955 24216793 3042.2
## - HmRun      1     52777 24254614 3042.6
## - Errors     1     59530 24261367 3042.7
## - League     1     63407 24265244 3042.7
## - Runs       1     64860 24266698 3042.7
## - CAtBat     1    174992 24376830 3043.9
## <none>                   24201837 3044.0
## - Assists    1    285766 24487603 3045.1
## - CRuns      1    611358 24813196 3048.6
## - CWalks     1    645627 24847464 3049.0
## - Division   1    834637 25036474 3050.9
## - CRBI       1    864220 25066057 3051.3
## - AtBat      1    970861 25172699 3052.4
## - Hits       1   1025981 25227819 3052.9
## - Walks      1   1167378 25369216 3054.4
## - PutOuts    1   1325273 25527110 3056.1
## 
## Step:  AIC=3042.12
## Salary ~ AtBat + Hits + HmRun + Runs + RBI + Walks + CAtBat + 
##     CHits + CRuns + CRBI + CWalks + League + Division + PutOuts + 
##     Assists + Errors + NewLeague
## 
##             Df Sum of Sq      RSS    AIC
## - NewLeague  1      9931 24219377 3040.2
## - RBI        1     15989 24225436 3040.3
## - CHits      1     18291 24227738 3040.3
## - HmRun      1     54144 24263591 3040.7
## - Errors     1     57312 24266759 3040.7
## - Runs       1     63172 24272619 3040.8
## - League     1     65732 24275178 3040.8
## <none>                   24209447 3042.1
## - CAtBat     1    266205 24475652 3043.0
## - Assists    1    293479 24502926 3043.3
## - CRuns      1    646350 24855797 3047.1
## - CWalks     1    649269 24858716 3047.1
## - Division   1    827511 25036958 3049.0
## - CRBI       1    872121 25081568 3049.4
## - AtBat      1    968713 25178160 3050.4
## - Hits       1   1018379 25227825 3050.9
## - Walks      1   1164536 25373983 3052.5
## - PutOuts    1   1334525 25543972 3054.2
## 
## Step:  AIC=3040.22
## Salary ~ AtBat + Hits + HmRun + Runs + RBI + Walks + CAtBat + 
##     CHits + CRuns + CRBI + CWalks + League + Division + PutOuts + 
##     Assists + Errors
## 
##            Df Sum of Sq      RSS    AIC
## - RBI       1     15800 24235177 3038.4
## - CHits     1     15859 24235237 3038.4
## - Errors    1     54505 24273883 3038.8
## - HmRun     1     54938 24274316 3038.8
## - Runs      1     62294 24281671 3038.9
## - League    1    107479 24326856 3039.4
## <none>                  24219377 3040.2
## - CAtBat    1    261336 24480713 3041.1
## - Assists   1    295536 24514914 3041.4
## - CWalks    1    648860 24868237 3045.2
## - CRuns     1    661449 24880826 3045.3
## - Division  1    824672 25044049 3047.0
## - CRBI      1    880429 25099806 3047.6
## - AtBat     1    999057 25218434 3048.9
## - Hits      1   1034463 25253840 3049.2
## - Walks     1   1157205 25376583 3050.5
## - PutOuts   1   1335173 25554550 3052.3
## 
## Step:  AIC=3038.4
## Salary ~ AtBat + Hits + HmRun + Runs + Walks + CAtBat + CHits + 
##     CRuns + CRBI + CWalks + League + Division + PutOuts + Assists + 
##     Errors
## 
##            Df Sum of Sq      RSS    AIC
## - CHits     1     13483 24248660 3036.5
## - HmRun     1     44586 24279763 3036.9
## - Runs      1     54057 24289234 3037.0
## - Errors    1     57656 24292833 3037.0
## - League    1    108644 24343821 3037.6
## <none>                  24235177 3038.4
## - CAtBat    1    252756 24487934 3039.1
## - Assists   1    294674 24529851 3039.6
## - CWalks    1    639690 24874868 3043.2
## - CRuns     1    693535 24928712 3043.8
## - Division  1    808984 25044161 3045.0
## - CRBI      1    893830 25129008 3045.9
## - Hits      1   1034884 25270061 3047.4
## - AtBat     1   1042798 25277975 3047.5
## - Walks     1   1145013 25380191 3048.5
## - PutOuts   1   1340713 25575890 3050.6
## 
## Step:  AIC=3036.54
## Salary ~ AtBat + Hits + HmRun + Runs + Walks + CAtBat + CRuns + 
##     CRBI + CWalks + League + Division + PutOuts + Assists + Errors
## 
##            Df Sum of Sq      RSS    AIC
## - HmRun     1     40487 24289148 3035.0
## - Errors    1     51930 24300590 3035.1
## - Runs      1     79343 24328003 3035.4
## - League    1    114742 24363402 3035.8
## <none>                  24248660 3036.5
## - Assists   1    283442 24532103 3037.6
## - CAtBat    1    613356 24862016 3041.1
## - Division  1    801474 25050134 3043.1
## - CRBI      1    903248 25151908 3044.2
## - CWalks    1   1011953 25260613 3045.3
## - Walks     1   1246164 25494824 3047.7
## - AtBat     1   1339620 25588280 3048.7
## - CRuns     1   1390808 25639469 3049.2
## - PutOuts   1   1406023 25654684 3049.4
## - Hits      1   1607990 25856650 3051.4
## 
## Step:  AIC=3034.98
## Salary ~ AtBat + Hits + Runs + Walks + CAtBat + CRuns + CRBI + 
##     CWalks + League + Division + PutOuts + Assists + Errors
## 
##            Df Sum of Sq      RSS    AIC
## - Errors    1     44085 24333232 3033.5
## - Runs      1     49068 24338215 3033.5
## - League    1    103837 24392985 3034.1
## <none>                  24289148 3035.0
## - Assists   1    247002 24536150 3035.6
## - CAtBat    1    652746 24941894 3040.0
## - Division  1    795643 25084791 3041.5
## - CWalks    1    982896 25272044 3043.4
## - Walks     1   1205823 25494971 3045.7
## - AtBat     1   1300972 25590120 3046.7
## - CRuns     1   1351200 25640348 3047.2
## - CRBI      1   1353507 25642655 3047.2
## - PutOuts   1   1429006 25718154 3048.0
## - Hits      1   1574140 25863288 3049.5
## 
## Step:  AIC=3033.46
## Salary ~ AtBat + Hits + Runs + Walks + CAtBat + CRuns + CRBI + 
##     CWalks + League + Division + PutOuts + Assists
## 
##            Df Sum of Sq      RSS    AIC
## - Runs      1     54113 24387345 3032.0
## - League    1     91269 24424501 3032.4
## <none>                  24333232 3033.5
## - Assists   1    220010 24553242 3033.8
## - CAtBat    1    650513 24983746 3038.4
## - Division  1    799455 25132687 3040.0
## - CWalks    1    971260 25304493 3041.8
## - Walks     1   1239533 25572765 3044.5
## - CRBI      1   1331672 25664904 3045.5
## - CRuns     1   1361070 25694302 3045.8
## - AtBat     1   1378592 25711824 3045.9
## - PutOuts   1   1391660 25724892 3046.1
## - Hits      1   1649291 25982523 3048.7
## 
## Step:  AIC=3032.04
## Salary ~ AtBat + Hits + Walks + CAtBat + CRuns + CRBI + CWalks + 
##     League + Division + PutOuts + Assists
## 
##            Df Sum of Sq      RSS    AIC
## - League    1    113056 24500402 3031.3
## <none>                  24387345 3032.0
## - Assists   1    280689 24668034 3033.1
## - CAtBat    1    596622 24983967 3036.4
## - Division  1    780369 25167714 3038.3
## - CWalks    1    946687 25334032 3040.1
## - Walks     1   1212997 25600342 3042.8
## - CRuns     1   1334397 25721742 3044.1
## - CRBI      1   1361339 25748684 3044.3
## - PutOuts   1   1455210 25842555 3045.3
## - AtBat     1   1522760 25910105 3046.0
## - Hits      1   1718870 26106215 3047.9
## 
## Step:  AIC=3031.26
## Salary ~ AtBat + Hits + Walks + CAtBat + CRuns + CRBI + CWalks + 
##     Division + PutOuts + Assists
## 
##            Df Sum of Sq      RSS    AIC
## <none>                  24500402 3031.3
## - Assists   1    313650 24814051 3032.6
## - CAtBat    1    534156 25034558 3034.9
## - Division  1    798473 25298875 3037.7
## - CWalks    1    965875 25466276 3039.4
## - CRuns     1   1265082 25765484 3042.5
## - Walks     1   1290168 25790569 3042.8
## - CRBI      1   1326770 25827172 3043.1
## - PutOuts   1   1551523 26051925 3045.4
## - AtBat     1   1589780 26090181 3045.8
## - Hits      1   1716068 26216469 3047.1
summary(mod_backward)
## 
## Call:
## lm(formula = Salary ~ AtBat + Hits + Walks + CAtBat + CRuns + 
##     CRBI + CWalks + Division + PutOuts + Assists, data = hitters)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -939.11 -176.87  -34.08  130.90 1910.55 
## 
## Coefficients:
##               Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  162.53544   66.90784   2.429 0.015830 *  
## AtBat         -2.16865    0.53630  -4.044 7.00e-05 ***
## Hits           6.91802    1.64665   4.201 3.69e-05 ***
## Walks          5.77322    1.58483   3.643 0.000327 ***
## CAtBat        -0.13008    0.05550  -2.344 0.019858 *  
## CRuns          1.40825    0.39040   3.607 0.000373 ***
## CRBI           0.77431    0.20961   3.694 0.000271 ***
## CWalks        -0.83083    0.26359  -3.152 0.001818 ** 
## DivisionW   -112.38006   39.21438  -2.866 0.004511 ** 
## PutOuts        0.29737    0.07444   3.995 8.50e-05 ***
## Assists        0.28317    0.15766   1.796 0.073673 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 311.8 on 252 degrees of freedom
## Multiple R-squared:  0.5405, Adjusted R-squared:  0.5223 
## F-statistic: 29.64 on 10 and 252 DF,  p-value: < 2.2e-16
formula(mod_backward)
## Salary ~ AtBat + Hits + Walks + CAtBat + CRuns + CRBI + CWalks + 
##     Division + PutOuts + Assists
AIC(mod_backward)
## [1] 3779.62

Prediktor yang masih terdapat pada mod_backward merupakan prediktor yang dipilih oleh prosedur backward selection berdasarkan AIC.

coef(mod_backward)
##  (Intercept)        AtBat         Hits        Walks       CAtBat        CRuns 
##  162.5354420   -2.1686501    6.9180175    5.7732246   -0.1300798    1.4082490 
##         CRBI       CWalks    DivisionW      PutOuts      Assists 
##    0.7743122   -0.8308264 -112.3800575    0.2973726    0.2831680
tidy(mod_backward)

5 4. Forward Selection

Forward selection dimulai dari model kosong, kemudian prediktor ditambahkan secara bertahap sampai diperoleh model berdasarkan AIC.

mod_null <- lm(
  Salary ~ 1,
  data = hitters
)

summary(mod_null)
## 
## Call:
## lm(formula = Salary ~ 1, data = hitters)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -468.4 -345.9 -110.9  214.1 1924.1 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   535.93      27.82   19.27   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 451.1 on 262 degrees of freedom
mod_forward <- step(
  mod_null,
  scope = list(
    lower = mod_null,
    upper = mod_full
  ),
  direction = "forward",
  trace = TRUE
)
## Start:  AIC=3215.77
## Salary ~ 1
## 
##             Df Sum of Sq      RSS    AIC
## + CRBI       1  17139434 36179679 3115.8
## + CRuns      1  16881162 36437951 3117.6
## + CHits      1  16065140 37253973 3123.5
## + CAtBat     1  14759710 38559403 3132.5
## + CHmRun     1  14692193 38626920 3133.0
## + CWalks     1  12792622 40526491 3145.6
## + RBI        1  10771083 42548030 3158.4
## + Walks      1  10504833 42814280 3160.1
## + Hits       1  10260491 43058621 3161.6
## + Runs       1   9399158 43919955 3166.8
## + Years      1   8559105 44760007 3171.7
## + AtBat      1   8309469 45009644 3173.2
## + HmRun      1   6273967 47045145 3184.8
## + PutOuts    1   4814100 48505013 3192.9
## + Division   1   1976102 51343011 3207.8
## <none>                   53319113 3215.8
## + Assists    1     34497 53284615 3217.6
## + League     1     10876 53308237 3217.7
## + Errors     1      1555 53317558 3217.8
## + NewLeague  1       428 53318684 3217.8
## 
## Step:  AIC=3115.78
## Salary ~ CRBI
## 
##             Df Sum of Sq      RSS    AIC
## + Hits       1   5533119 30646560 3074.1
## + Runs       1   5176532 31003147 3077.2
## + Walks      1   4199733 31979946 3085.3
## + AtBat      1   4064585 32115095 3086.4
## + RBI        1   3308272 32871407 3092.6
## + PutOuts    1   3267035 32912644 3092.9
## + Division   1   1733887 34445793 3104.9
## + Years      1   1667339 34512340 3105.4
## + HmRun      1   1271587 34908092 3108.4
## + CRuns      1    354561 35825119 3115.2
## + Assists    1    346020 35833659 3115.2
## <none>                   36179679 3115.8
## + Errors     1    194403 35985276 3116.4
## + CAtBat     1     92261 36087418 3117.1
## + CHits      1     75469 36104210 3117.2
## + CWalks     1     51974 36127705 3117.4
## + NewLeague  1     17778 36161901 3117.7
## + League     1     11825 36167855 3117.7
## + CHmRun     1       515 36179165 3117.8
## 
## Step:  AIC=3074.13
## Salary ~ CRBI + Hits
## 
##             Df Sum of Sq      RSS    AIC
## + PutOuts    1   1397263 29249297 3063.8
## + Division   1   1279275 29367285 3064.9
## + AtBat      1    821767 29824793 3069.0
## + Walks      1    781767 29864793 3069.3
## + Years      1    254910 30391650 3073.9
## <none>                   30646560 3074.1
## + League     1    208880 30437680 3074.3
## + CRuns      1    132614 30513946 3075.0
## + NewLeague  1    118474 30528086 3075.1
## + Runs       1    114198 30532362 3075.1
## + Errors     1     99776 30546784 3075.3
## + CAtBat     1     83517 30563043 3075.4
## + Assists    1     44781 30601779 3075.7
## + CWalks     1     23668 30622892 3075.9
## + CHmRun     1      4790 30641769 3076.1
## + CHits      1      4358 30642202 3076.1
## + HmRun      1      2173 30644387 3076.1
## + RBI        1      1137 30645423 3076.1
## 
## Step:  AIC=3063.85
## Salary ~ CRBI + Hits + PutOuts
## 
##             Df Sum of Sq      RSS    AIC
## + Division   1   1278445 27970852 3054.1
## + AtBat      1   1009933 28239364 3056.6
## + Walks      1    539490 28709807 3061.0
## + CRuns      1    273649 28975648 3063.4
## <none>                   29249297 3063.8
## + Years      1    136906 29112391 3064.6
## + League     1    122841 29126456 3064.8
## + Runs       1    117930 29131367 3064.8
## + Errors     1     97244 29152053 3065.0
## + NewLeague  1     57839 29191458 3065.3
## + CHits      1     35096 29214201 3065.5
## + RBI        1     33965 29215331 3065.6
## + HmRun      1     31227 29218070 3065.6
## + CWalks     1     28572 29220725 3065.6
## + CAtBat     1     20518 29228779 3065.7
## + Assists    1      1681 29247616 3065.8
## + CHmRun     1      1419 29247878 3065.8
## 
## Step:  AIC=3054.1
## Salary ~ CRBI + Hits + PutOuts + Division
## 
##             Df Sum of Sq      RSS    AIC
## + AtBat      1    820952 27149899 3048.3
## + Walks      1    491584 27479268 3051.4
## <none>                   27970852 3054.1
## + CRuns      1    193604 27777248 3054.3
## + Years      1    166845 27804007 3054.5
## + League     1    110628 27860224 3055.1
## + Errors     1     81385 27889467 3055.3
## + Runs       1     65921 27904931 3055.5
## + RBI        1     53719 27917133 3055.6
## + NewLeague  1     52275 27918577 3055.6
## + CHits      1     33863 27936989 3055.8
## + HmRun      1     26390 27944462 3055.8
## + CAtBat     1     18751 27952101 3055.9
## + CWalks     1      5723 27965129 3056.0
## + Assists    1      1036 27969816 3056.1
## + CHmRun     1       165 27970687 3056.1
## 
## Step:  AIC=3048.26
## Salary ~ CRBI + Hits + PutOuts + Division + AtBat
## 
##             Df Sum of Sq      RSS    AIC
## + Walks      1    954996 26194904 3040.8
## + Years      1    253362 26896537 3047.8
## + Runs       1    208743 26941157 3048.2
## <none>                   27149899 3048.3
## + CRuns      1    185825 26964075 3048.5
## + League     1     95986 27053913 3049.3
## + NewLeague  1     52693 27097206 3049.8
## + CHmRun     1     43173 27106726 3049.8
## + Assists    1     28898 27121001 3050.0
## + CAtBat     1     20989 27128910 3050.1
## + CWalks     1     15599 27134301 3050.1
## + Errors     1      6265 27143634 3050.2
## + CHits      1      5305 27144594 3050.2
## + RBI        1      1236 27148663 3050.2
## + HmRun      1        11 27149888 3050.3
## 
## Step:  AIC=3040.85
## Salary ~ CRBI + Hits + PutOuts + Division + AtBat + Walks
## 
##             Df Sum of Sq      RSS    AIC
## + CWalks     1    240687 25954217 3040.4
## <none>                   26194904 3040.8
## + Years      1    184508 26010396 3041.0
## + CRuns      1    110695 26084209 3041.7
## + League     1     77974 26116930 3042.1
## + Assists    1     75782 26119122 3042.1
## + NewLeague  1     40909 26153995 3042.4
## + CHits      1     37304 26157599 3042.5
## + RBI        1     11728 26183176 3042.7
## + HmRun      1      4747 26190157 3042.8
## + Errors     1      2727 26192177 3042.8
## + CAtBat     1      2630 26192274 3042.8
## + CHmRun     1       943 26193961 3042.8
## + Runs       1        37 26194867 3042.8
## 
## Step:  AIC=3040.42
## Salary ~ CRBI + Hits + PutOuts + Division + AtBat + Walks + CWalks
## 
##             Df Sum of Sq      RSS    AIC
## + CRuns      1    794983 25159234 3034.2
## + CHits      1    273728 25680489 3039.6
## <none>                   25954217 3040.4
## + Assists    1    138506 25815711 3041.0
## + CAtBat     1     89289 25864929 3041.5
## + RBI        1     86941 25867276 3041.5
## + League     1     77159 25877058 3041.6
## + Years      1     70126 25884091 3041.7
## + NewLeague  1     37807 25916410 3042.0
## + HmRun      1     33601 25920616 3042.1
## + CHmRun     1      9034 25945183 3042.3
## + Errors     1      6928 25947289 3042.3
## + Runs       1        82 25954135 3042.4
## 
## Step:  AIC=3034.24
## Salary ~ CRBI + Hits + PutOuts + Division + AtBat + Walks + CWalks + 
##     CRuns
## 
##             Df Sum of Sq      RSS    AIC
## + CAtBat     1    345182 24814051 3032.6
## + Years      1    265068 24894165 3033.4
## <none>                   25159234 3034.2
## + CHits      1    173215 24986019 3034.4
## + Assists    1    124676 25034558 3034.9
## + CHmRun     1     91728 25067505 3035.3
## + League     1     72985 25086248 3035.5
## + NewLeague  1     25244 25133990 3036.0
## + Runs       1     17700 25141534 3036.1
## + Errors     1      8531 25150703 3036.2
## + HmRun      1      2390 25156844 3036.2
## + RBI        1         0 25159234 3036.2
## 
## Step:  AIC=3032.6
## Salary ~ CRBI + Hits + PutOuts + Division + AtBat + Walks + CWalks + 
##     CRuns + CAtBat
## 
##             Df Sum of Sq      RSS    AIC
## + Assists    1    313650 24500402 3031.3
## <none>                   24814051 3032.6
## + Runs       1    153499 24660553 3033.0
## + League     1    146017 24668034 3033.1
## + NewLeague  1     77209 24736842 3033.8
## + Years      1     47431 24766620 3034.1
## + CHits      1     47127 24766925 3034.1
## + Errors     1     43926 24770126 3034.1
## + HmRun      1     23132 24790919 3034.4
## + CHmRun     1     19917 24794134 3034.4
## + RBI        1     19027 24795024 3034.4
## 
## Step:  AIC=3031.26
## Salary ~ CRBI + Hits + PutOuts + Division + AtBat + Walks + CWalks + 
##     CRuns + CAtBat + Assists
## 
##             Df Sum of Sq      RSS    AIC
## <none>                   24500402 3031.3
## + League     1    113056 24387345 3032.0
## + Runs       1     75900 24424501 3032.4
## + NewLeague  1     64712 24435690 3032.6
## + CHits      1     37564 24462838 3032.8
## + Errors     1     35264 24465138 3032.9
## + Years      1     19883 24480519 3033.0
## + CHmRun     1      4356 24496046 3033.2
## + HmRun      1      1189 24499212 3033.2
## + RBI        1       359 24500043 3033.2
summary(mod_forward)
## 
## Call:
## lm(formula = Salary ~ CRBI + Hits + PutOuts + Division + AtBat + 
##     Walks + CWalks + CRuns + CAtBat + Assists, data = hitters)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -939.11 -176.87  -34.08  130.90 1910.55 
## 
## Coefficients:
##               Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  162.53544   66.90784   2.429 0.015830 *  
## CRBI           0.77431    0.20961   3.694 0.000271 ***
## Hits           6.91802    1.64665   4.201 3.69e-05 ***
## PutOuts        0.29737    0.07444   3.995 8.50e-05 ***
## DivisionW   -112.38006   39.21438  -2.866 0.004511 ** 
## AtBat         -2.16865    0.53630  -4.044 7.00e-05 ***
## Walks          5.77322    1.58483   3.643 0.000327 ***
## CWalks        -0.83083    0.26359  -3.152 0.001818 ** 
## CRuns          1.40825    0.39040   3.607 0.000373 ***
## CAtBat        -0.13008    0.05550  -2.344 0.019858 *  
## Assists        0.28317    0.15766   1.796 0.073673 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 311.8 on 252 degrees of freedom
## Multiple R-squared:  0.5405, Adjusted R-squared:  0.5223 
## F-statistic: 29.64 on 10 and 252 DF,  p-value: < 2.2e-16
formula(mod_forward)
## Salary ~ CRBI + Hits + PutOuts + Division + AtBat + Walks + CWalks + 
##     CRuns + CAtBat + Assists
AIC(mod_forward)
## [1] 3779.62

Prediktor yang masih terdapat pada mod_forward merupakan prediktor yang dipilih oleh prosedur forward selection berdasarkan AIC.

coef(mod_forward)
##  (Intercept)         CRBI         Hits      PutOuts    DivisionW        AtBat 
##  162.5354420    0.7743122    6.9180175    0.2973726 -112.3800575   -2.1686501 
##        Walks       CWalks        CRuns       CAtBat      Assists 
##    5.7732246   -0.8308264    1.4082490   -0.1300798    0.2831680
tidy(mod_forward)

6 5. Pemilihan Model Berdasarkan BIC

Pemilihan model berdasarkan BIC dilakukan dengan regsubsets() dari package leaps. Karena seluruh prediktor digunakan, nilai nvmax disesuaikan dengan jumlah prediktor pada data.

p <- ncol(hitters) - 1

reg_forward <- regsubsets(
  Salary ~ .,
  data = hitters,
  method = "forward",
  nvmax = p
)

summary_forward <- summary(reg_forward)

summary_forward$which
##    (Intercept) AtBat  Hits HmRun  Runs   RBI Walks Years CAtBat CHits CHmRun
## 1         TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE  FALSE FALSE  FALSE
## 2         TRUE FALSE  TRUE FALSE FALSE FALSE FALSE FALSE  FALSE FALSE  FALSE
## 3         TRUE FALSE  TRUE FALSE FALSE FALSE FALSE FALSE  FALSE FALSE  FALSE
## 4         TRUE FALSE  TRUE FALSE FALSE FALSE FALSE FALSE  FALSE FALSE  FALSE
## 5         TRUE  TRUE  TRUE FALSE FALSE FALSE FALSE FALSE  FALSE FALSE  FALSE
## 6         TRUE  TRUE  TRUE FALSE FALSE FALSE  TRUE FALSE  FALSE FALSE  FALSE
## 7         TRUE  TRUE  TRUE FALSE FALSE FALSE  TRUE FALSE  FALSE FALSE  FALSE
## 8         TRUE  TRUE  TRUE FALSE FALSE FALSE  TRUE FALSE  FALSE FALSE  FALSE
## 9         TRUE  TRUE  TRUE FALSE FALSE FALSE  TRUE FALSE   TRUE FALSE  FALSE
## 10        TRUE  TRUE  TRUE FALSE FALSE FALSE  TRUE FALSE   TRUE FALSE  FALSE
## 11        TRUE  TRUE  TRUE FALSE FALSE FALSE  TRUE FALSE   TRUE FALSE  FALSE
## 12        TRUE  TRUE  TRUE FALSE  TRUE FALSE  TRUE FALSE   TRUE FALSE  FALSE
## 13        TRUE  TRUE  TRUE FALSE  TRUE FALSE  TRUE FALSE   TRUE FALSE  FALSE
## 14        TRUE  TRUE  TRUE  TRUE  TRUE FALSE  TRUE FALSE   TRUE FALSE  FALSE
## 15        TRUE  TRUE  TRUE  TRUE  TRUE FALSE  TRUE FALSE   TRUE  TRUE  FALSE
## 16        TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE   TRUE  TRUE  FALSE
## 17        TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE   TRUE  TRUE  FALSE
## 18        TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE   TRUE  TRUE  FALSE
## 19        TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE   TRUE  TRUE   TRUE
##    CRuns CRBI CWalks LeagueN DivisionW PutOuts Assists Errors NewLeagueN
## 1  FALSE TRUE  FALSE   FALSE     FALSE   FALSE   FALSE  FALSE      FALSE
## 2  FALSE TRUE  FALSE   FALSE     FALSE   FALSE   FALSE  FALSE      FALSE
## 3  FALSE TRUE  FALSE   FALSE     FALSE    TRUE   FALSE  FALSE      FALSE
## 4  FALSE TRUE  FALSE   FALSE      TRUE    TRUE   FALSE  FALSE      FALSE
## 5  FALSE TRUE  FALSE   FALSE      TRUE    TRUE   FALSE  FALSE      FALSE
## 6  FALSE TRUE  FALSE   FALSE      TRUE    TRUE   FALSE  FALSE      FALSE
## 7  FALSE TRUE   TRUE   FALSE      TRUE    TRUE   FALSE  FALSE      FALSE
## 8   TRUE TRUE   TRUE   FALSE      TRUE    TRUE   FALSE  FALSE      FALSE
## 9   TRUE TRUE   TRUE   FALSE      TRUE    TRUE   FALSE  FALSE      FALSE
## 10  TRUE TRUE   TRUE   FALSE      TRUE    TRUE    TRUE  FALSE      FALSE
## 11  TRUE TRUE   TRUE    TRUE      TRUE    TRUE    TRUE  FALSE      FALSE
## 12  TRUE TRUE   TRUE    TRUE      TRUE    TRUE    TRUE  FALSE      FALSE
## 13  TRUE TRUE   TRUE    TRUE      TRUE    TRUE    TRUE   TRUE      FALSE
## 14  TRUE TRUE   TRUE    TRUE      TRUE    TRUE    TRUE   TRUE      FALSE
## 15  TRUE TRUE   TRUE    TRUE      TRUE    TRUE    TRUE   TRUE      FALSE
## 16  TRUE TRUE   TRUE    TRUE      TRUE    TRUE    TRUE   TRUE      FALSE
## 17  TRUE TRUE   TRUE    TRUE      TRUE    TRUE    TRUE   TRUE       TRUE
## 18  TRUE TRUE   TRUE    TRUE      TRUE    TRUE    TRUE   TRUE       TRUE
## 19  TRUE TRUE   TRUE    TRUE      TRUE    TRUE    TRUE   TRUE       TRUE

Membentuk tabel BIC:

forward_metric <- tibble(
  nvar = 1:length(summary_forward$bic),
  rss = summary_forward$rss,
  adjr2 = summary_forward$adjr2,
  cp = summary_forward$cp,
  bic = summary_forward$bic
)

forward_metric

Model dengan BIC terkecil:

best_forward_bic <- which.min(summary_forward$bic)

best_forward_bic
## [1] 6
coef(
  reg_forward,
  id = best_forward_bic
)
##  (Intercept)        AtBat         Hits        Walks         CRBI    DivisionW 
##   91.5117981   -1.8685892    7.6043976    3.6976468    0.6430169 -122.9515338 
##      PutOuts 
##    0.2643076

Visualisasi BIC:

forward_metric %>%
  ggplot(aes(x = nvar, y = bic)) +
  geom_line() +
  geom_point() +
  labs(
    title = "BIC Forward Selection",
    x = "Jumlah Prediktor",
    y = "BIC"
  ) +
  theme_bw()

6.1 BIC dengan Backward Selection

reg_backward <- regsubsets(
  Salary ~ .,
  data = hitters,
  method = "backward",
  nvmax = p
)

summary_backward <- summary(reg_backward)

summary_backward$which
##    (Intercept) AtBat  Hits HmRun  Runs   RBI Walks Years CAtBat CHits CHmRun
## 1         TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE  FALSE FALSE  FALSE
## 2         TRUE FALSE  TRUE FALSE FALSE FALSE FALSE FALSE  FALSE FALSE  FALSE
## 3         TRUE FALSE  TRUE FALSE FALSE FALSE FALSE FALSE  FALSE FALSE  FALSE
## 4         TRUE  TRUE  TRUE FALSE FALSE FALSE FALSE FALSE  FALSE FALSE  FALSE
## 5         TRUE  TRUE  TRUE FALSE FALSE FALSE  TRUE FALSE  FALSE FALSE  FALSE
## 6         TRUE  TRUE  TRUE FALSE FALSE FALSE  TRUE FALSE  FALSE FALSE  FALSE
## 7         TRUE  TRUE  TRUE FALSE FALSE FALSE  TRUE FALSE  FALSE FALSE  FALSE
## 8         TRUE  TRUE  TRUE FALSE FALSE FALSE  TRUE FALSE  FALSE FALSE  FALSE
## 9         TRUE  TRUE  TRUE FALSE FALSE FALSE  TRUE FALSE   TRUE FALSE  FALSE
## 10        TRUE  TRUE  TRUE FALSE FALSE FALSE  TRUE FALSE   TRUE FALSE  FALSE
## 11        TRUE  TRUE  TRUE FALSE FALSE FALSE  TRUE FALSE   TRUE FALSE  FALSE
## 12        TRUE  TRUE  TRUE FALSE  TRUE FALSE  TRUE FALSE   TRUE FALSE  FALSE
## 13        TRUE  TRUE  TRUE FALSE  TRUE FALSE  TRUE FALSE   TRUE FALSE  FALSE
## 14        TRUE  TRUE  TRUE  TRUE  TRUE FALSE  TRUE FALSE   TRUE FALSE  FALSE
## 15        TRUE  TRUE  TRUE  TRUE  TRUE FALSE  TRUE FALSE   TRUE  TRUE  FALSE
## 16        TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE   TRUE  TRUE  FALSE
## 17        TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE   TRUE  TRUE  FALSE
## 18        TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE   TRUE  TRUE  FALSE
## 19        TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE   TRUE  TRUE   TRUE
##    CRuns  CRBI CWalks LeagueN DivisionW PutOuts Assists Errors NewLeagueN
## 1   TRUE FALSE  FALSE   FALSE     FALSE   FALSE   FALSE  FALSE      FALSE
## 2   TRUE FALSE  FALSE   FALSE     FALSE   FALSE   FALSE  FALSE      FALSE
## 3   TRUE FALSE  FALSE   FALSE     FALSE    TRUE   FALSE  FALSE      FALSE
## 4   TRUE FALSE  FALSE   FALSE     FALSE    TRUE   FALSE  FALSE      FALSE
## 5   TRUE FALSE  FALSE   FALSE     FALSE    TRUE   FALSE  FALSE      FALSE
## 6   TRUE FALSE  FALSE   FALSE      TRUE    TRUE   FALSE  FALSE      FALSE
## 7   TRUE FALSE   TRUE   FALSE      TRUE    TRUE   FALSE  FALSE      FALSE
## 8   TRUE  TRUE   TRUE   FALSE      TRUE    TRUE   FALSE  FALSE      FALSE
## 9   TRUE  TRUE   TRUE   FALSE      TRUE    TRUE   FALSE  FALSE      FALSE
## 10  TRUE  TRUE   TRUE   FALSE      TRUE    TRUE    TRUE  FALSE      FALSE
## 11  TRUE  TRUE   TRUE    TRUE      TRUE    TRUE    TRUE  FALSE      FALSE
## 12  TRUE  TRUE   TRUE    TRUE      TRUE    TRUE    TRUE  FALSE      FALSE
## 13  TRUE  TRUE   TRUE    TRUE      TRUE    TRUE    TRUE   TRUE      FALSE
## 14  TRUE  TRUE   TRUE    TRUE      TRUE    TRUE    TRUE   TRUE      FALSE
## 15  TRUE  TRUE   TRUE    TRUE      TRUE    TRUE    TRUE   TRUE      FALSE
## 16  TRUE  TRUE   TRUE    TRUE      TRUE    TRUE    TRUE   TRUE      FALSE
## 17  TRUE  TRUE   TRUE    TRUE      TRUE    TRUE    TRUE   TRUE       TRUE
## 18  TRUE  TRUE   TRUE    TRUE      TRUE    TRUE    TRUE   TRUE       TRUE
## 19  TRUE  TRUE   TRUE    TRUE      TRUE    TRUE    TRUE   TRUE       TRUE

Membentuk tabel BIC:

backward_metric <- tibble(
  nvar = 1:length(summary_backward$bic),
  rss = summary_backward$rss,
  adjr2 = summary_backward$adjr2,
  cp = summary_backward$cp,
  bic = summary_backward$bic
)

backward_metric

Model dengan BIC terkecil:

best_backward_bic <- which.min(summary_backward$bic)

best_backward_bic
## [1] 8
coef(
  reg_backward,
  id = best_backward_bic
)
##  (Intercept)        AtBat         Hits        Walks        CRuns         CRBI 
##  117.1520434   -2.0339209    6.8549136    6.4406642    0.7045391    0.5273238 
##       CWalks    DivisionW      PutOuts 
##   -0.8066062 -123.7798366    0.2753892

Visualisasi BIC:

backward_metric %>%
  ggplot(aes(x = nvar, y = bic)) +
  geom_line() +
  geom_point() +
  labs(
    title = "BIC Backward Selection",
    x = "Jumlah Prediktor",
    y = "BIC"
  ) +
  theme_bw()

7 6. Perbandingan Prediktor Forward dan Backward

formula(mod_forward)
## Salary ~ CRBI + Hits + PutOuts + Division + AtBat + Walks + CWalks + 
##     CRuns + CAtBat + Assists
formula(mod_backward)
## Salary ~ AtBat + Hits + Walks + CAtBat + CRuns + CRBI + CWalks + 
##     Division + PutOuts + Assists
vars_forward <- attr(
  terms(mod_forward),
  "term.labels"
)

vars_backward <- attr(
  terms(mod_backward),
  "term.labels"
)

comparison <- tibble(
  Prediktor = sort(unique(c(vars_forward, vars_backward))),
  Forward = sort(unique(c(vars_forward, vars_backward))) %in% vars_forward,
  Backward = sort(unique(c(vars_forward, vars_backward))) %in% vars_backward
)

comparison

Prediktor yang sama-sama terpilih:

intersect(vars_forward, vars_backward)
##  [1] "CRBI"     "Hits"     "PutOuts"  "Division" "AtBat"    "Walks"   
##  [7] "CWalks"   "CRuns"    "CAtBat"   "Assists"

Prediktor yang hanya dipilih forward:

setdiff(vars_forward, vars_backward)
## character(0)

Prediktor yang hanya dipilih backward:

setdiff(vars_backward, vars_forward)
## character(0)

Forward dan backward dapat menghasilkan kombinasi prediktor yang berbeda karena keduanya melakukan pencarian model dari arah yang berbeda.

8 7. Pemeriksaan Diagnostik Model

Untuk pemeriksaan diagnostik, digunakan model hasil backward selection sebagaimana alur pada praktikum.

8.1 7.1 Empat Plot Diagnostik

par(mfrow = c(2, 2))
plot(mod_backward)

par(mfrow = c(1, 1))

Keempat grafik digunakan untuk melihat pola residual, normalitas residual, pengamatan berpengaruh, dan hubungan antara residual dengan nilai prediksi.

8.2 7.2 Uji Normalitas Residual

shapiro.test(residuals(mod_backward))
## 
##  Shapiro-Wilk normality test
## 
## data:  residuals(mod_backward)
## W = 0.92458, p-value = 2.73e-10

Hipotesis uji Shapiro-Wilk:

  • H0: residual berdistribusi normal.
  • H1: residual tidak berdistribusi normal.

Keputusan dapat dibuat berdasarkan nilai-p dengan taraf signifikansi yang digunakan.

8.3 7.3 Residual terhadap Fitted Value

plot(
  fitted(mod_backward),
  residuals(mod_backward),
  xlab = "Fitted Value",
  ylab = "Residual"
)

abline(h = 0, lty = 2)

Plot ini digunakan untuk melihat apakah terdapat pola tertentu pada residual. Residual yang tersebar secara acak di sekitar nol lebih sesuai dengan asumsi linearitas dan ragam konstan.

8.4 7.4 Cook’s Distance

cook <- cooks.distance(mod_backward)

plot(
  cook,
  type = "h",
  ylab = "Cook's Distance"
)

abline(
  h = 4 / nrow(hitters),
  lty = 2
)

order(
  cook,
  decreasing = TRUE
)[1:5]
## [1] 173 201 189 183  55

Cook’s distance digunakan untuk mengidentifikasi pengamatan yang memiliki pengaruh besar terhadap hasil estimasi model.

9 8. Prediksi dan Evaluasi Model

pred <- predict(
  mod_backward,
  newdata = hitters
)

hasil_prediksi <- tibble(
  aktual = hitters$Salary,
  prediksi = pred
)

head(hasil_prediksi)

9.1 Actual vs Predicted

hasil_prediksi %>%
  ggplot(aes(x = aktual, y = prediksi)) +
  geom_point(alpha = 0.6) +
  geom_abline(
    intercept = 0,
    slope = 1,
    linetype = 2
  ) +
  labs(
    x = "Salary Aktual",
    y = "Salary Prediksi",
    title = "Salary Aktual vs Salary Prediksi"
  ) +
  theme_bw()

9.2 RMSE dan R-squared

rmse <- sqrt(
  mean(
    (hitters$Salary - pred)^2
  )
)

r_squared <- summary(mod_backward)$r.squared
adj_r_squared <- summary(mod_backward)$adj.r.squared

rmse
## [1] 305.217
r_squared
## [1] 0.540495
adj_r_squared
## [1] 0.5222606

10 9. Ringkasan Model

cat("MODEL FULL:\n")
## MODEL FULL:
print(formula(mod_full))
## Salary ~ AtBat + Hits + HmRun + Runs + RBI + Walks + Years + 
##     CAtBat + CHits + CHmRun + CRuns + CRBI + CWalks + League + 
##     Division + PutOuts + Assists + Errors + NewLeague
cat("\nMODEL BACKWARD:\n")
## 
## MODEL BACKWARD:
print(formula(mod_backward))
## Salary ~ AtBat + Hits + Walks + CAtBat + CRuns + CRBI + CWalks + 
##     Division + PutOuts + Assists
cat("\nMODEL FORWARD:\n")
## 
## MODEL FORWARD:
print(formula(mod_forward))
## Salary ~ CRBI + Hits + PutOuts + Division + AtBat + Walks + CWalks + 
##     CRuns + CAtBat + Assists
cat("\nBIC TERKECIL - FORWARD:\n")
## 
## BIC TERKECIL - FORWARD:
print(best_forward_bic)
## [1] 6
cat("\nBIC TERKECIL - BACKWARD:\n")
## 
## BIC TERKECIL - BACKWARD:
print(best_backward_bic)
## [1] 8
cat("\nRMSE MODEL BACKWARD:\n")
## 
## RMSE MODEL BACKWARD:
print(rmse)
## [1] 305.217
cat("\nR-SQUARED MODEL BACKWARD:\n")
## 
## R-SQUARED MODEL BACKWARD:
print(r_squared)
## [1] 0.540495
cat("\nADJUSTED R-SQUARED MODEL BACKWARD:\n")
## 
## ADJUSTED R-SQUARED MODEL BACKWARD:
print(adj_r_squared)
## [1] 0.5222606

11 10. Interpretasi Hasil Analisis

Interpretasi berikut dibuat berdasarkan output yang dihasilkan saat dokumen di-knit.

11.1 Model regresi penuh

Model regresi penuh menggunakan Salary sebagai peubah respon dan seluruh peubah lainnya sebagai prediktor. Koefisien masing-masing prediktor menunjukkan perubahan rata-rata Salary ketika prediktor tersebut berubah satu satuan, dengan prediktor lain dianggap tetap.

11.2 Backward selection

Backward selection dimulai dari model yang memuat seluruh prediktor. Prediktor kemudian dieliminasi secara bertahap berdasarkan AIC. Prediktor yang tersisa pada model akhir merupakan prediktor yang dipertahankan oleh prosedur tersebut.

11.3 Forward selection

Forward selection dimulai dari model kosong. Prediktor kemudian dimasukkan satu per satu berdasarkan perbaikan AIC. Prediktor yang terdapat pada model akhir merupakan prediktor yang dipilih oleh prosedur forward.

11.4 BIC

BIC memberikan penalti terhadap kompleksitas model. Model dengan nilai BIC paling kecil dipilih sebagai model berdasarkan kriteria BIC.

11.5 Perbandingan forward dan backward

Hasil forward dan backward tidak harus identik. Perbedaan dapat terjadi karena kedua prosedur melakukan pencarian model dari titik awal yang berbeda. Oleh karena itu, tabel comparison digunakan untuk melihat prediktor yang sama-sama dipilih dan prediktor yang hanya muncul pada salah satu metode.

11.6 Diagnostik

Pemeriksaan diagnostik digunakan untuk mengevaluasi apakah model memenuhi asumsi yang diperlukan. Residual diperiksa melalui plot diagnostik, uji Shapiro-Wilk digunakan untuk memeriksa normalitas residual, sedangkan Cook’s distance digunakan untuk melihat pengamatan yang berpengaruh.

11.7 Evaluasi prediksi

RMSE menunjukkan besarnya kesalahan prediksi model pada skala Salary. Nilai R-squared menunjukkan proporsi variasi Salary yang dapat dijelaskan oleh model, sedangkan adjusted R-squared mempertimbangkan jumlah prediktor dalam model.

12 Kesimpulan

Analisis dilakukan dengan membentuk model regresi linear penuh, melakukan backward dan forward selection, menentukan model berdasarkan BIC, membandingkan prediktor yang terpilih, serta melakukan pemeriksaan diagnostik. Model akhir dan interpretasi numeriknya mengikuti output yang dihasilkan ketika dokumen ini dijalankan pada data Hitters.