Linearboston-ref

Load the data
EDA (ggplot for all variables and plotly for select), including collinearity(use vif)
linear regression lm on all predictor variables
var importance and check plots for model performance
tuning (use stepAIC, glmnet and compare using resamples)

EnsurePackage <- function(x) {
  x <- as.character(x)
  if(!require(x,character.only = T))
    install.packages(x,repos = "https://cran.rstudio.com/")
  require(x,character.only = T)
}


EnsurePackage("mlbench")

## Loading required package: mlbench

EnsurePackage("tidyverse")

## Loading required package: tidyverse

## -- Attaching packages -------------------------------------------------------------- tidyverse 1.2.1 --

## v ggplot2 3.2.1     v purrr   0.3.3
## v tibble  2.1.3     v dplyr   0.8.3
## v tidyr   1.0.0     v stringr 1.4.0
## v readr   1.3.1     v forcats 0.4.0

## -- Conflicts ----------------------------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag()    masks stats::lag()

EnsurePackage("caret")

## Loading required package: caret

## Loading required package: lattice

## 
## Attaching package: 'caret'

## The following object is masked from 'package:purrr':
## 
##     lift

EnsurePackage("MASS")

## Loading required package: MASS

## 
## Attaching package: 'MASS'

## The following object is masked from 'package:dplyr':
## 
##     select

EnsurePackage("ggplot2")
EnsurePackage("corrplot")

## Loading required package: corrplot

## corrplot 0.84 loaded

EnsurePackage("Hmisc")

## Loading required package: Hmisc

## Loading required package: survival

## 
## Attaching package: 'survival'

## The following object is masked from 'package:caret':
## 
##     cluster

## Loading required package: Formula

## 
## Attaching package: 'Hmisc'

## The following objects are masked from 'package:dplyr':
## 
##     src, summarize

## The following objects are masked from 'package:base':
## 
##     format.pval, units

EnsurePackage("leaps")

## Loading required package: leaps

EnsurePackage("plotly")

## Loading required package: plotly

## 
## Attaching package: 'plotly'

## The following object is masked from 'package:Hmisc':
## 
##     subplot

## The following object is masked from 'package:MASS':
## 
##     select

## The following object is masked from 'package:ggplot2':
## 
##     last_plot

## The following object is masked from 'package:stats':
## 
##     filter

## The following object is masked from 'package:graphics':
## 
##     layout

EnsurePackage("car")

## Loading required package: car

## Loading required package: carData

## 
## Attaching package: 'car'

## The following object is masked from 'package:dplyr':
## 
##     recode

## The following object is masked from 'package:purrr':
## 
##     some

Load the data

data("BostonHousing")
bston <- BostonHousing

str(bston)

## 'data.frame':    506 obs. of  14 variables:
##  $ crim   : num  0.00632 0.02731 0.02729 0.03237 0.06905 ...
##  $ zn     : num  18 0 0 0 0 0 12.5 12.5 12.5 12.5 ...
##  $ indus  : num  2.31 7.07 7.07 2.18 2.18 2.18 7.87 7.87 7.87 7.87 ...
##  $ chas   : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
##  $ nox    : num  0.538 0.469 0.469 0.458 0.458 0.458 0.524 0.524 0.524 0.524 ...
##  $ rm     : num  6.58 6.42 7.18 7 7.15 ...
##  $ age    : num  65.2 78.9 61.1 45.8 54.2 58.7 66.6 96.1 100 85.9 ...
##  $ dis    : num  4.09 4.97 4.97 6.06 6.06 ...
##  $ rad    : num  1 2 2 3 3 3 5 5 5 5 ...
##  $ tax    : num  296 242 242 222 222 222 311 311 311 311 ...
##  $ ptratio: num  15.3 17.8 17.8 18.7 18.7 18.7 15.2 15.2 15.2 15.2 ...
##  $ b      : num  397 397 393 395 397 ...
##  $ lstat  : num  4.98 9.14 4.03 2.94 5.33 ...
##  $ medv   : num  24 21.6 34.7 33.4 36.2 28.7 22.9 27.1 16.5 18.9 ...

summary(bston)

##       crim                zn             indus       chas   
##  Min.   : 0.00632   Min.   :  0.00   Min.   : 0.46   0:471  
##  1st Qu.: 0.08204   1st Qu.:  0.00   1st Qu.: 5.19   1: 35  
##  Median : 0.25651   Median :  0.00   Median : 9.69          
##  Mean   : 3.61352   Mean   : 11.36   Mean   :11.14          
##  3rd Qu.: 3.67708   3rd Qu.: 12.50   3rd Qu.:18.10          
##  Max.   :88.97620   Max.   :100.00   Max.   :27.74          
##       nox               rm             age              dis        
##  Min.   :0.3850   Min.   :3.561   Min.   :  2.90   Min.   : 1.130  
##  1st Qu.:0.4490   1st Qu.:5.886   1st Qu.: 45.02   1st Qu.: 2.100  
##  Median :0.5380   Median :6.208   Median : 77.50   Median : 3.207  
##  Mean   :0.5547   Mean   :6.285   Mean   : 68.57   Mean   : 3.795  
##  3rd Qu.:0.6240   3rd Qu.:6.623   3rd Qu.: 94.08   3rd Qu.: 5.188  
##  Max.   :0.8710   Max.   :8.780   Max.   :100.00   Max.   :12.127  
##       rad              tax           ptratio            b         
##  Min.   : 1.000   Min.   :187.0   Min.   :12.60   Min.   :  0.32  
##  1st Qu.: 4.000   1st Qu.:279.0   1st Qu.:17.40   1st Qu.:375.38  
##  Median : 5.000   Median :330.0   Median :19.05   Median :391.44  
##  Mean   : 9.549   Mean   :408.2   Mean   :18.46   Mean   :356.67  
##  3rd Qu.:24.000   3rd Qu.:666.0   3rd Qu.:20.20   3rd Qu.:396.23  
##  Max.   :24.000   Max.   :711.0   Max.   :22.00   Max.   :396.90  
##      lstat            medv      
##  Min.   : 1.73   Min.   : 5.00  
##  1st Qu.: 6.95   1st Qu.:17.02  
##  Median :11.36   Median :21.20  
##  Mean   :12.65   Mean   :22.53  
##  3rd Qu.:16.95   3rd Qu.:25.00  
##  Max.   :37.97   Max.   :50.00

Visualize plots

You can also embed plots, for example:

##    crim      zn   indus    chas     nox      rm     age     dis     rad 
##       0       0       0       0       0       0       0       0       0 
##     tax ptratio       b   lstat    medv 
##       0       0       0       0       0

##  [1] crim    zn      indus   chas    nox     rm      age     dis    
##  [9] rad     tax     ptratio b       lstat   medv   
## <0 rows> (or 0-length row.names)

## [1] 1

## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

##         [,1] [,2]  [,3] [,4]  [,5]   [,6]  [,7]    [,8] [,9] [,10] [,11]
## [1,] 0.00906  0.0  0.46    1 0.385 4.8800   2.9 1.12960    1   187  13.6
## [2,] 0.08221  0.0  5.19    1 0.449 5.8850  45.0 2.10000    4   279  17.4
## [3,] 0.25651  0.0  9.69    1 0.538 6.2085  77.7 3.19095    5   330  19.1
## [4,] 3.67822 12.5 18.10    1 0.624 6.6290  94.1 5.21190   24   666  20.2
## [5,] 8.98296 30.0 27.74    1 0.871 7.6910 100.0 9.22290   24   711  22.0
##       [,12] [,13] [,14]
## [1,] 344.05  1.73   5.0
## [2,] 375.33  6.93  17.0
## [3,] 391.44 11.36  21.2
## [4,] 396.21 16.96  25.0
## [5,] 396.90 31.99  37.0

## `geom_smooth()` using method = 'loess' and formula 'y ~ x'

## Warning in simpleLoess(y, x, w, span, degree = degree, parametric =
## parametric, : pseudoinverse used at -0.5

## Warning in simpleLoess(y, x, w, span, degree = degree, parametric =
## parametric, : neighborhood radius 13

## Warning in simpleLoess(y, x, w, span, degree = degree, parametric =
## parametric, : reciprocal condition number 0

## Warning in simpleLoess(y, x, w, span, degree = degree, parametric =
## parametric, : There are other near singularities as well. 156.25

## Warning in predLoess(object$y, object$x, newx = if
## (is.null(newdata)) object$x else if (is.data.frame(newdata))
## as.matrix(model.frame(delete.response(terms(object)), : pseudoinverse used
## at -0.5

## Warning in predLoess(object$y, object$x, newx = if
## (is.null(newdata)) object$x else if (is.data.frame(newdata))
## as.matrix(model.frame(delete.response(terms(object)), : neighborhood radius
## 13

## Warning in predLoess(object$y, object$x, newx = if
## (is.null(newdata)) object$x else if (is.data.frame(newdata))
## as.matrix(model.frame(delete.response(terms(object)), : reciprocal
## condition number 0

## Warning in predLoess(object$y, object$x, newx = if
## (is.null(newdata)) object$x else if (is.data.frame(newdata))
## as.matrix(model.frame(delete.response(terms(object)), : There are other
## near singularities as well. 156.25

Create linear models

set.seed(8)

split <- createDataPartition(bston$medv,p=0.8,list=FALSE)
train <- bston[split,]
test <- bston[-split,]

lm1 <- lm(medv~lstat+rm,data=train)
plot(lm1)

pred1 <- predict(lm1,test)

error1 <- test$medv - pred1

#calculate rmse
rmse1 <- sqrt(mean(error1^2))
rmse1

## [1] 5.723487

lm2 <- stepAIC(lm(medv~.,data=train))

## Start:  AIC=1273.4
## medv ~ crim + zn + indus + chas + nox + rm + age + dis + rad + 
##     tax + ptratio + b + lstat
## 
##           Df Sum of Sq     RSS    AIC
## - age      1      1.98  8681.8 1271.5
## - indus    1      8.08  8687.9 1271.8
## <none>                  8679.8 1273.4
## - zn       1    194.42  8874.3 1280.4
## - tax      1    209.25  8889.1 1281.1
## - crim     1    211.26  8891.1 1281.2
## - chas     1    212.76  8892.6 1281.2
## - nox      1    304.88  8984.7 1285.5
## - b        1    324.73  9004.6 1286.3
## - rad      1    427.45  9107.3 1291.0
## - dis      1    847.12  9526.9 1309.3
## - ptratio  1    982.58  9662.4 1315.0
## - rm       1   1419.19 10099.0 1333.0
## - lstat    1   1796.37 10476.2 1348.0
## 
## Step:  AIC=1271.49
## medv ~ crim + zn + indus + chas + nox + rm + dis + rad + tax + 
##     ptratio + b + lstat
## 
##           Df Sum of Sq     RSS    AIC
## - indus    1      8.08  8689.9 1269.9
## <none>                  8681.8 1271.5
## - zn       1    192.47  8874.3 1278.4
## - tax      1    208.02  8889.8 1279.1
## - crim     1    212.35  8894.2 1279.3
## - chas     1    215.12  8896.9 1279.5
## - nox      1    317.67  8999.5 1284.1
## - b        1    330.38  9012.2 1284.7
## - rad      1    425.96  9107.8 1289.0
## - dis      1    943.59  9625.4 1311.5
## - ptratio  1    989.49  9671.3 1313.4
## - rm       1   1543.39 10225.2 1336.1
## - lstat    1   2018.05 10699.9 1354.5
## 
## Step:  AIC=1269.87
## medv ~ crim + zn + chas + nox + rm + dis + rad + tax + ptratio + 
##     b + lstat
## 
##           Df Sum of Sq     RSS    AIC
## <none>                  8689.9 1269.9
## - zn       1    185.66  8875.6 1276.5
## - crim     1    214.86  8904.8 1277.8
## - tax      1    216.14  8906.0 1277.9
## - chas     1    226.82  8916.7 1278.4
## - nox      1    316.83  9006.7 1282.4
## - b        1    326.22  9016.1 1282.9
## - rad      1    427.88  9117.8 1287.4
## - ptratio  1    982.24  9672.1 1311.5
## - dis      1   1031.51  9721.4 1313.5
## - rm       1   1535.58 10225.5 1334.1
## - lstat    1   2014.26 10704.2 1352.7

plot(lm2)

pred2 <- predict(lm2,test)

error2 <- test$medv - pred2

#calculate rmse
rmse2 <- sqrt(mean(error2^2))
rmse2

## [1] 4.937934

x <-varImp(lm2)
x$features <- as.factor(row.names(x))

x$features <- factor(x$features, levels = x$features[order(x$Overall,decreasing = TRUE)])

xo<-x

ggplot(data=xo,aes(x=features,y=Overall)) + geom_bar(stat="identity")

vif(lm2)

##     crim       zn     chas      nox       rm      dis      rad      tax 
## 1.749133 2.231565 1.063728 3.762859 1.888166 3.425955 6.515874 7.080225 
##  ptratio        b    lstat 
## 1.710590 1.326574 2.698204

lm3 <- stepAIC(lm(medv ~ lstat+I(lstat^2)+rm+dis+ptratio+nox+b+zn+crim,data=train))

## Start:  AIC=1223.54
## medv ~ lstat + I(lstat^2) + rm + dis + ptratio + nox + b + zn + 
##     crim
## 
##              Df Sum of Sq     RSS    AIC
## - zn          1     26.69  7858.2 1222.9
## <none>                     7831.5 1223.5
## - nox         1    143.02  7974.5 1228.9
## - b           1    222.28  8053.8 1232.9
## - crim        1    262.09  8093.6 1234.9
## - ptratio     1    599.61  8431.1 1251.6
## - dis         1    870.86  8702.3 1264.5
## - I(lstat^2)  1   1562.71  9394.2 1295.6
## - rm          1   1645.78  9477.3 1299.2
## - lstat       1   2899.68 10731.2 1349.7
## 
## Step:  AIC=1222.92
## medv ~ lstat + I(lstat^2) + rm + dis + ptratio + nox + b + crim
## 
##              Df Sum of Sq     RSS    AIC
## <none>                     7858.2 1222.9
## - nox         1    141.33  7999.5 1228.2
## - b           1    218.97  8077.2 1232.1
## - crim        1    246.25  8104.4 1233.5
## - ptratio     1    695.14  8553.3 1255.4
## - dis         1    965.23  8823.4 1268.1
## - I(lstat^2)  1   1698.36  9556.5 1300.6
## - rm          1   1754.25  9612.4 1302.9
## - lstat       1   3078.64 10936.8 1355.5

plot(lm3)

lm3

## 
## Call:
## lm(formula = medv ~ lstat + I(lstat^2) + rm + dis + ptratio + 
##     nox + b + crim, data = train)
## 
## Coefficients:
## (Intercept)        lstat   I(lstat^2)           rm          dis  
##   30.153242    -1.678169     0.033666     4.159900    -1.157996  
##     ptratio          nox            b         crim  
##   -0.710432    -8.953024     0.009357    -0.106574

pred3 <- predict(lm3,test)

error3 <- test$medv - pred3

#calculate rmse
rmse3 <- sqrt(mean(error3^2))
rmse3

## [1] 4.492579

g1 <- ggplot(data = test, aes(test$medv,pred2)) + 
  geom_point() + geom_smooth(method = "lm",aes(colour="red"))

g2 <- ggplot(data = test, aes(test$medv,pred3)) + 
  geom_point() + geom_smooth(method ="lm",aes(colour="red"))



gridExtra::grid.arrange(g1,g2,nrow=1)

# Linear regression using caret package
fitcontrol <- trainControl(method="cv",number=10,p=0.8)


#glmnet
lm4 <- train(medv~lstat+I(lstat^2)+rm+dis+ptratio+nox+b+zn+crim,data=train,
             method="glmnet",trControl=fitcontrol,tuneLength=20)

## Warning in nominalTrainWorkflow(x = x, y = y, wts = weights, info =
## trainInfo, : There were missing values in resampled performance measures.

lm4

## glmnet 
## 
## 407 samples
##   8 predictor
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold) 
## Summary of sample sizes: 367, 367, 366, 366, 367, 366, ... 
## Resampling results across tuning parameters:
## 
##   alpha      lambda       RMSE      Rsquared   MAE     
##   0.1000000  0.003221909  4.396388  0.7479428  3.169284
##   0.1000000  0.004995621  4.396388  0.7479428  3.169284
##   0.1000000  0.007745789  4.396064  0.7479771  3.168791
##   0.1000000  0.012009969  4.395707  0.7479758  3.167074
##   0.1000000  0.018621646  4.396159  0.7478726  3.165328
##   0.1000000  0.028873156  4.397930  0.7476351  3.162807
##   0.1000000  0.044768285  4.402930  0.7470936  3.160401
##   0.1000000  0.069413932  4.414446  0.7459705  3.159908
##   0.1000000  0.107627398  4.437815  0.7438053  3.164278
##   0.1000000  0.166877981  4.479325  0.7400452  3.174320
##   0.1000000  0.258746945  4.543763  0.7342352  3.194436
##   0.1000000  0.401191224  4.631339  0.7263092  3.231443
##   0.1000000  0.622053328  4.737568  0.7166514  3.285968
##   0.1000000  0.964503510  4.852433  0.7063612  3.352115
##   0.1000000  1.495477924  4.917386  0.7028260  3.390045
##   0.1000000  2.318762138  5.045696  0.6946792  3.473903
##   0.1000000  3.595277314  5.246548  0.6818366  3.617989
##   0.1000000  5.574534253  5.496956  0.6706768  3.789538
##   0.1000000  8.643403396  5.828241  0.6656732  4.023747
##   0.1473684  0.003221909  4.396441  0.7479335  3.169401
##   0.1473684  0.004995621  4.396441  0.7479335  3.169401
##   0.1473684  0.007745789  4.396099  0.7479732  3.168842
##   0.1473684  0.012009969  4.395844  0.7479584  3.167166
##   0.1473684  0.018621646  4.396307  0.7478534  3.165415
##   0.1473684  0.028873156  4.398182  0.7476034  3.162923
##   0.1473684  0.044768285  4.403341  0.7470435  3.160465
##   0.1473684  0.069413932  4.415265  0.7458794  3.159906
##   0.1473684  0.107627398  4.439638  0.7436216  3.164440
##   0.1473684  0.166877981  4.483596  0.7396374  3.174885
##   0.1473684  0.258746945  4.553140  0.7333515  3.197947
##   0.1473684  0.401191224  4.649980  0.7245251  3.240245
##   0.1473684  0.622053328  4.770880  0.7133691  3.305933
##   0.1473684  0.964503510  4.885388  0.7032035  3.371592
##   0.1473684  1.495477924  4.957050  0.6994022  3.417221
##   0.1473684  2.318762138  5.105654  0.6899822  3.520298
##   0.1473684  3.595277314  5.328147  0.6767022  3.676968
##   0.1473684  5.574534253  5.590101  0.6715466  3.857182
##   0.1473684  8.643403396  6.014552  0.6654767  4.156663
##   0.1947368  0.003221909  4.396456  0.7479314  3.169366
##   0.1947368  0.004995621  4.396456  0.7479314  3.169366
##   0.1947368  0.007745789  4.396175  0.7479637  3.168891
##   0.1947368  0.012009969  4.395944  0.7479457  3.167229
##   0.1947368  0.018621646  4.396456  0.7478340  3.165498
##   0.1947368  0.028873156  4.398404  0.7475747  3.163032
##   0.1947368  0.044768285  4.403726  0.7469970  3.160533
##   0.1947368  0.069413932  4.416084  0.7457904  3.159911
##   0.1947368  0.107627398  4.441571  0.7434270  3.164664
##   0.1947368  0.166877981  4.488150  0.7392027  3.175683
##   0.1947368  0.258746945  4.563318  0.7323868  3.201889
##   0.1947368  0.401191224  4.670472  0.7225458  3.250675
##   0.1947368  0.622053328  4.808128  0.7096359  3.328051
##   0.1947368  0.964503510  4.910810  0.7008424  3.386273
##   0.1947368  1.495477924  5.002877  0.6951884  3.450204
##   0.1947368  2.318762138  5.166120  0.6850168  3.569060
##   0.1947368  3.595277314  5.377190  0.6770940  3.708401
##   0.1947368  5.574534253  5.698266  0.6707119  3.934021
##   0.1947368  8.643403396  6.226187  0.6601065  4.309749
##   0.2421053  0.003221909  4.396269  0.7479587  3.169356
##   0.2421053  0.004995621  4.396269  0.7479587  3.169356
##   0.2421053  0.007745789  4.396000  0.7479858  3.168829
##   0.2421053  0.012009969  4.396026  0.7479349  3.167299
##   0.2421053  0.018621646  4.396610  0.7478145  3.165568
##   0.2421053  0.028873156  4.398649  0.7475432  3.163132
##   0.2421053  0.044768285  4.404156  0.7469450  3.160642
##   0.2421053  0.069413932  4.416957  0.7456941  3.159933
##   0.2421053  0.107627398  4.443594  0.7432247  3.164923
##   0.2421053  0.166877981  4.493037  0.7387330  3.176786
##   0.2421053  0.258746945  4.574342  0.7313365  3.206584
##   0.2421053  0.401191224  4.692916  0.7203552  3.262903
##   0.2421053  0.622053328  4.847374  0.7056474  3.351148
##   0.2421053  0.964503510  4.939817  0.6980602  3.406245
##   0.2421053  1.495477924  5.048186  0.6909468  3.485650
##   0.2421053  2.318762138  5.205935  0.6828505  3.593815
##   0.2421053  3.595277314  5.432048  0.6771298  3.742885
##   0.2421053  5.574534253  5.819363  0.6681776  4.015500
##   0.2421053  8.643403396  6.428396  0.6567319  4.461109
##   0.2894737  0.003221909  4.396546  0.7479167  3.169465
##   0.2894737  0.004995621  4.396546  0.7479167  3.169465
##   0.2894737  0.007745789  4.396302  0.7479474  3.168956
##   0.2894737  0.012009969  4.396124  0.7479219  3.167360
##   0.2894737  0.018621646  4.396752  0.7477958  3.165633
##   0.2894737  0.028873156  4.398881  0.7475132  3.163216
##   0.2894737  0.044768285  4.404562  0.7468961  3.160712
##   0.2894737  0.069413932  4.417857  0.7455956  3.159937
##   0.2894737  0.107627398  4.445728  0.7430092  3.165186
##   0.2894737  0.166877981  4.498179  0.7382393  3.178278
##   0.2894737  0.258746945  4.586183  0.7302006  3.211529
##   0.2894737  0.401191224  4.717568  0.7179203  3.277551
##   0.2894737  0.622053328  4.885465  0.7017361  3.372324
##   0.2894737  0.964503510  4.970959  0.6949935  3.428830
##   0.2894737  1.495477924  5.094196  0.6864501  3.522962
##   0.2894737  2.318762138  5.236249  0.6823895  3.612113
##   0.2894737  3.595277314  5.494686  0.6764716  3.784440
##   0.2894737  5.574534253  5.949265  0.6645345  4.099454
##   0.2894737  8.643403396  6.634063  0.6549193  4.613100
##   0.3368421  0.003221909  4.396565  0.7479167  3.169534
##   0.3368421  0.004995621  4.396565  0.7479167  3.169534
##   0.3368421  0.007745789  4.396219  0.7479575  3.168943
##   0.3368421  0.012009969  4.396223  0.7479090  3.167421
##   0.3368421  0.018621646  4.396920  0.7477733  3.165715
##   0.3368421  0.028873156  4.399126  0.7474814  3.163317
##   0.3368421  0.044768285  4.405026  0.7468397  3.160810
##   0.3368421  0.069413932  4.418799  0.7454924  3.159956
##   0.3368421  0.107627398  4.447977  0.7427849  3.165455
##   0.3368421  0.166877981  4.503685  0.7377102  3.179995
##   0.3368421  0.258746945  4.598971  0.7289654  3.217287
##   0.3368421  0.401191224  4.744629  0.7152172  3.293304
##   0.3368421  0.622053328  4.912967  0.6989274  3.388863
##   0.3368421  0.964503510  5.000321  0.6921067  3.450833
##   0.3368421  1.495477924  5.128127  0.6834400  3.546151
##   0.3368421  2.318762138  5.271653  0.6813922  3.635399
##   0.3368421  3.595277314  5.563422  0.6752628  3.828687
##   0.3368421  5.574534253  6.071454  0.6632419  4.186218
##   0.3368421  8.643403396  6.857983  0.6508765  4.782141
##   0.3842105  0.003221909  4.396620  0.7479061  3.169497
##   0.3842105  0.004995621  4.396620  0.7479061  3.169497
##   0.3842105  0.007745789  4.396421  0.7479308  3.169016
##   0.3842105  0.012009969  4.396322  0.7478953  3.167476
##   0.3842105  0.018621646  4.397057  0.7477552  3.165774
##   0.3842105  0.028873156  4.399380  0.7474494  3.163404
##   0.3842105  0.044768285  4.405476  0.7467866  3.160881
##   0.3842105  0.069413932  4.419793  0.7453843  3.160013
##   0.3842105  0.107627398  4.450339  0.7425472  3.165770
##   0.3842105  0.166877981  4.509565  0.7371415  3.181932
##   0.3842105  0.258746945  4.612803  0.7276236  3.224186
##   0.3842105  0.401191224  4.774077  0.7122480  3.310782
##   0.3842105  0.622053328  4.934647  0.6967129  3.404114
##   0.3842105  0.964503510  5.031769  0.6889050  3.476497
##   0.3842105  1.495477924  5.154720  0.6816004  3.563139
##   0.3842105  2.318762138  5.310922  0.6799608  3.659728
##   0.3842105  3.595277314  5.637070  0.6735561  3.876838
##   0.3842105  5.574534253  6.195236  0.6632722  4.277288
##   0.3842105  8.643403396  7.099139  0.6427046  4.970199
##   0.4315789  0.003221909  4.396094  0.7479555  3.169292
##   0.4315789  0.004995621  4.396094  0.7479555  3.169292
##   0.4315789  0.007745789  4.396317  0.7479428  3.169032
##   0.4315789  0.012009969  4.396350  0.7478900  3.167586
##   0.4315789  0.018621646  4.397150  0.7477402  3.165915
##   0.4315789  0.028873156  4.399594  0.7474193  3.163554
##   0.4315789  0.044768285  4.405911  0.7467325  3.161000
##   0.4315789  0.069413932  4.420771  0.7452777  3.160159
##   0.4315789  0.107627398  4.452803  0.7423000  3.166373
##   0.4315789  0.166877981  4.515783  0.7365380  3.183955
##   0.4315789  0.258746945  4.627700  0.7261639  3.231472
##   0.4315789  0.401191224  4.806553  0.7089317  3.329580
##   0.4315789  0.622053328  4.955346  0.6945929  3.419061
##   0.4315789  0.964503510  5.058404  0.6862967  3.498300
##   0.4315789  1.495477924  5.175835  0.6805636  3.575695
##   0.4315789  2.318762138  5.354697  0.6779485  3.686325
##   0.4315789  3.595277314  5.715004  0.6710669  3.928974
##   0.4315789  5.574534253  6.326473  0.6627925  4.375708
##   0.4315789  8.643403396  7.329032  0.6322870  5.157025
##   0.4789474  0.003221909  4.396317  0.7479481  3.169470
##   0.4789474  0.004995621  4.396317  0.7479481  3.169470
##   0.4789474  0.007745789  4.396402  0.7479317  3.169100
##   0.4789474  0.012009969  4.396461  0.7478745  3.167682
##   0.4789474  0.018621646  4.397325  0.7477173  3.166029
##   0.4789474  0.028873156  4.399876  0.7473838  3.163675
##   0.4789474  0.044768285  4.406434  0.7466701  3.161146
##   0.4789474  0.069413932  4.421869  0.7451578  3.160352
##   0.4789474  0.107627398  4.455446  0.7420349  3.167136
##   0.4789474  0.166877981  4.522404  0.7358957  3.186214
##   0.4789474  0.258746945  4.643753  0.7245839  3.239643
##   0.4789474  0.401191224  4.841403  0.7053399  3.349300
##   0.4789474  0.622053328  4.973045  0.6928528  3.433422
##   0.4789474  0.964503510  5.077414  0.6846588  3.513140
##   0.4789474  1.495477924  5.198095  0.6794719  3.588353
##   0.4789474  2.318762138  5.399003  0.6760514  3.715639
##   0.4789474  3.595277314  5.789526  0.6690913  3.982026
##   0.4789474  5.574534253  6.467860  0.6588831  4.483910
##   0.4789474  8.643403396  7.539469  0.6249562  5.325972
##   0.5263158  0.003221909  4.396758  0.7478917  3.169774
##   0.5263158  0.004995621  4.396758  0.7478917  3.169774
##   0.5263158  0.007745789  4.396455  0.7479216  3.169141
##   0.5263158  0.012009969  4.396568  0.7478582  3.167755
##   0.5263158  0.018621646  4.397498  0.7476936  3.166123
##   0.5263158  0.028873156  4.400143  0.7473490  3.163784
##   0.5263158  0.044768285  4.406933  0.7466094  3.161269
##   0.5263158  0.069413932  4.423011  0.7450327  3.160562
##   0.5263158  0.107627398  4.458222  0.7417545  3.168084
##   0.5263158  0.166877981  4.529446  0.7352089  3.188549
##   0.5263158  0.258746945  4.661008  0.7228760  3.248840
##   0.5263158  0.401191224  4.877269  0.7016168  3.369248
##   0.5263158  0.622053328  4.992081  0.6909354  3.448928
##   0.5263158  0.964503510  5.095987  0.6830838  3.527027
##   0.5263158  1.495477924  5.221954  0.6782471  3.601543
##   0.5263158  2.318762138  5.445518  0.6739545  3.746704
##   0.5263158  3.595277314  5.865651  0.6670695  4.035235
##   0.5263158  5.574534253  6.621224  0.6505551  4.604192
##   0.5263158  8.643403396  7.758493  0.6176948  5.499757
##   0.5736842  0.003221909  4.396480  0.7479294  3.169712
##   0.5736842  0.004995621  4.396480  0.7479294  3.169712
##   0.5736842  0.007745789  4.396399  0.7479281  3.169146
##   0.5736842  0.012009969  4.396665  0.7478453  3.167822
##   0.5736842  0.018621646  4.397648  0.7476730  3.166210
##   0.5736842  0.028873156  4.400392  0.7473162  3.163908
##   0.5736842  0.044768285  4.407450  0.7465488  3.161418
##   0.5736842  0.069413932  4.424140  0.7449118  3.160749
##   0.5736842  0.107627398  4.461057  0.7414714  3.169033
##   0.5736842  0.166877981  4.536846  0.7344877  3.191183
##   0.5736842  0.258746945  4.679511  0.7210346  3.259360
##   0.5736842  0.401191224  4.910168  0.6982009  3.389661
##   0.5736842  0.622053328  5.012009  0.6888748  3.466059
##   0.5736842  0.964503510  5.117143  0.6811642  3.541623
##   0.5736842  1.495477924  5.245787  0.6771229  3.616414
##   0.5736842  2.318762138  5.496099  0.6712742  3.779870
##   0.5736842  3.595277314  5.946983  0.6643854  4.093809
##   0.5736842  5.574534253  6.781471  0.6390873  4.728872
##   0.5736842  8.643403396  7.993612  0.6043327  5.687645
##   0.6210526  0.003221909  4.396510  0.7479171  3.169751
##   0.6210526  0.004995621  4.396510  0.7479171  3.169751
##   0.6210526  0.007745789  4.396466  0.7479193  3.169177
##   0.6210526  0.012009969  4.396761  0.7478324  3.167878
##   0.6210526  0.018621646  4.397799  0.7476530  3.166288
##   0.6210526  0.028873156  4.400675  0.7472803  3.164021
##   0.6210526  0.044768285  4.408011  0.7464836  3.161559
##   0.6210526  0.069413932  4.425336  0.7447841  3.160946
##   0.6210526  0.107627398  4.464044  0.7411715  3.169982
##   0.6210526  0.166877981  4.544696  0.7337195  3.194302
##   0.6210526  0.258746945  4.699463  0.7190368  3.270978
##   0.6210526  0.401191224  4.933239  0.6958509  3.405323
##   0.6210526  0.622053328  5.029718  0.6870417  3.479610
##   0.6210526  0.964503510  5.136508  0.6795305  3.554456
##   0.6210526  1.495477924  5.270290  0.6760031  3.632056
##   0.6210526  2.318762138  5.546120  0.6685804  3.813403
##   0.6210526  3.595277314  6.033116  0.6609943  4.154629
##   0.6210526  5.574534253  6.934434  0.6283115  4.848460
##   0.6210526  8.643403396  8.243724  0.5770572  5.891942
##   0.6684211  0.003221909  4.396354  0.7479326  3.169699
##   0.6684211  0.004995621  4.396354  0.7479326  3.169699
##   0.6684211  0.007745789  4.396529  0.7479109  3.169206
##   0.6684211  0.012009969  4.396873  0.7478180  3.167933
##   0.6684211  0.018621646  4.397961  0.7476312  3.166370
##   0.6684211  0.028873156  4.400975  0.7472425  3.164145
##   0.6684211  0.044768285  4.408540  0.7464215  3.161709
##   0.6684211  0.069413932  4.426567  0.7446510  3.161134
##   0.6684211  0.107627398  4.467154  0.7408595  3.170946
##   0.6684211  0.166877981  4.552988  0.7329060  3.198251
##   0.6684211  0.258746945  4.720932  0.7168752  3.283152
##   0.6684211  0.401191224  4.950305  0.6940940  3.417866
##   0.6684211  0.622053328  5.044630  0.6855863  3.491401
##   0.6684211  0.964503510  5.153492  0.6782281  3.566293
##   0.6684211  1.495477924  5.296335  0.6747348  3.649290
##   0.6684211  2.318762138  5.591198  0.6668460  3.846288
##   0.6684211  3.595277314  6.124146  0.6567191  4.221578
##   0.6684211  5.574534253  7.075352  0.6227797  4.954695
##   0.6684211  8.643403396  8.482355  0.5463821  6.092957
##   0.7157895  0.003221909  4.396606  0.7479129  3.169880
##   0.7157895  0.004995621  4.396606  0.7479129  3.169880
##   0.7157895  0.007745789  4.396566  0.7479042  3.169209
##   0.7157895  0.012009969  4.396938  0.7478090  3.167964
##   0.7157895  0.018621646  4.398101  0.7476140  3.166432
##   0.7157895  0.028873156  4.401222  0.7472122  3.164234
##   0.7157895  0.044768285  4.409097  0.7463567  3.161862
##   0.7157895  0.069413932  4.427838  0.7445171  3.161453
##   0.7157895  0.107627398  4.470396  0.7405367  3.172040
##   0.7157895  0.166877981  4.561737  0.7320473  3.202812
##   0.7157895  0.258746945  4.744003  0.7145364  3.296005
##   0.7157895  0.401191224  4.962577  0.6928445  3.427699
##   0.7157895  0.622053328  5.056409  0.6845257  3.500463
##   0.7157895  0.964503510  5.167125  0.6773966  3.575348
##   0.7157895  1.495477924  5.324902  0.6731757  3.668144
##   0.7157895  2.318762138  5.635335  0.6655023  3.879258
##   0.7157895  3.595277314  6.220223  0.6513399  4.295116
##   0.7157895  5.574534253  7.219475  0.6180672  5.063820
##   0.7157895  8.643403396  8.709467  0.5202712  6.289749
##   0.7631579  0.003221909  4.396363  0.7479335  3.169668
##   0.7631579  0.004995621  4.396363  0.7479335  3.169668
##   0.7631579  0.007745789  4.396640  0.7478933  3.169251
##   0.7631579  0.012009969  4.397043  0.7477944  3.168028
##   0.7631579  0.018621646  4.398278  0.7475906  3.166511
##   0.7631579  0.028873156  4.401545  0.7471725  3.164370
##   0.7631579  0.044768285  4.409670  0.7462909  3.162042
##   0.7631579  0.069413932  4.429167  0.7443743  3.161895
##   0.7631579  0.107627398  4.473822  0.7401943  3.173421
##   0.7631579  0.166877981  4.571028  0.7311307  3.207674
##   0.7631579  0.258746945  4.768998  0.7119872  3.309827
##   0.7631579  0.401191224  4.974727  0.6915886  3.437762
##   0.7631579  0.622053328  5.068308  0.6834546  3.509537
##   0.7631579  0.964503510  5.180519  0.6766737  3.583800
##   0.7631579  1.495477924  5.355494  0.6713489  3.688021
##   0.7631579  2.318762138  5.682480  0.6637900  3.912930
##   0.7631579  3.595277314  6.321418  0.6445805  4.376347
##   0.7631579  5.574534253  7.372407  0.6117323  5.182817
##   0.7631579  8.643403396  8.912443  0.4652069  6.468904
##   0.8105263  0.003221909  4.396734  0.7478992  3.169873
##   0.8105263  0.004995621  4.396734  0.7478992  3.169873
##   0.8105263  0.007745789  4.396704  0.7478854  3.169282
##   0.8105263  0.012009969  4.397159  0.7477793  3.168096
##   0.8105263  0.018621646  4.398461  0.7475670  3.166596
##   0.8105263  0.028873156  4.401832  0.7471368  3.164482
##   0.8105263  0.044768285  4.410290  0.7462190  3.162269
##   0.8105263  0.069413932  4.430558  0.7442259  3.162418
##   0.8105263  0.107627398  4.477379  0.7398369  3.174824
##   0.8105263  0.166877981  4.580845  0.7301581  3.212698
##   0.8105263  0.258746945  4.795889  0.7092296  3.324835
##   0.8105263  0.401191224  4.987771  0.6902198  3.448422
##   0.8105263  0.622053328  5.080562  0.6823404  3.519124
##   0.8105263  0.964503510  5.194193  0.6759918  3.591955
##   0.8105263  1.495477924  5.387167  0.6693196  3.710060
##   0.8105263  2.318762138  5.732433  0.6617057  3.947186
##   0.8105263  3.595277314  6.426244  0.6363927  4.461284
##   0.8105263  5.574534253  7.534424  0.6026538  5.308946
##   0.8105263  8.643403396  8.981890        NaN  6.527839
##   0.8578947  0.003221909  4.396751  0.7478927  3.169834
##   0.8578947  0.004995621  4.396751  0.7478927  3.169834
##   0.8578947  0.007745789  4.396750  0.7478791  3.169318
##   0.8578947  0.012009969  4.397253  0.7477667  3.168143
##   0.8578947  0.018621646  4.398623  0.7475463  3.166681
##   0.8578947  0.028873156  4.402178  0.7470952  3.164612
##   0.8578947  0.044768285  4.410921  0.7461473  3.162485
##   0.8578947  0.069413932  4.431990  0.7440749  3.163014
##   0.8578947  0.107627398  4.481082  0.7394653  3.176244
##   0.8578947  0.166877981  4.591271  0.7291239  3.218056
##   0.8578947  0.258746945  4.824770  0.7062491  3.340539
##   0.8578947  0.401191224  5.001290  0.6887674  3.459518
##   0.8578947  0.622053328  5.093533  0.6811296  3.528964
##   0.8578947  0.964503510  5.208653  0.6752270  3.600870
##   0.8578947  1.495477924  5.418560  0.6672807  3.732391
##   0.8578947  2.318762138  5.784808  0.6592828  3.982984
##   0.8578947  3.595277314  6.530254  0.6276456  4.545689
##   0.8578947  5.574534253  7.706031  0.5890867  5.443531
##   0.8578947  8.643403396  8.981890        NaN  6.527839
##   0.9052632  0.003221909  4.396584  0.7479019  3.169849
##   0.9052632  0.004995621  4.396584  0.7479019  3.169849
##   0.9052632  0.007745789  4.396814  0.7478701  3.169353
##   0.9052632  0.012009969  4.397308  0.7477584  3.168192
##   0.9052632  0.018621646  4.398733  0.7475309  3.166731
##   0.9052632  0.028873156  4.402429  0.7470631  3.164708
##   0.9052632  0.044768285  4.411502  0.7460799  3.162683
##   0.9052632  0.069413932  4.433379  0.7439276  3.163697
##   0.9052632  0.107627398  4.484870  0.7390860  3.177725
##   0.9052632  0.166877981  4.602195  0.7280355  3.223637
##   0.9052632  0.258746945  4.854125  0.7032185  3.356857
##   0.9052632  0.401191224  5.013030  0.6875286  3.469590
##   0.9052632  0.622053328  5.106940  0.6798535  3.539067
##   0.9052632  0.964503510  5.224166  0.6743568  3.610885
##   0.9052632  1.495477924  5.447461  0.6657119  3.753275
##   0.9052632  2.318762138  5.839623  0.6564724  4.021264
##   0.9052632  3.595277314  6.626443  0.6208489  4.621850
##   0.9052632  5.574534253  7.884157  0.5701239  5.585276
##   0.9052632  8.643403396  8.981890        NaN  6.527839
##   0.9526316  0.003221909  4.396752  0.7478887  3.169910
##   0.9526316  0.004995621  4.396752  0.7478887  3.169910
##   0.9526316  0.007745789  4.396854  0.7478654  3.169373
##   0.9526316  0.012009969  4.397428  0.7477432  3.168257
##   0.9526316  0.018621646  4.398944  0.7475047  3.166838
##   0.9526316  0.028873156  4.402749  0.7470246  3.164828
##   0.9526316  0.044768285  4.412164  0.7460051  3.162907
##   0.9526316  0.069413932  4.434907  0.7437651  3.164498
##   0.9526316  0.107627398  4.488904  0.7386795  3.179445
##   0.9526316  0.166877981  4.613834  0.7268745  3.229684
##   0.9526316  0.258746945  4.882768  0.7002946  3.374424
##   0.9526316  0.401191224  5.023694  0.6864030  3.478287
##   0.9526316  0.622053328  5.120647  0.6785400  3.549371
##   0.9526316  0.964503510  5.240929  0.6733655  3.621166
##   0.9526316  1.495477924  5.474103  0.6645096  3.771833
##   0.9526316  2.318762138  5.896946  0.6532126  4.061429
##   0.9526316  3.595277314  6.714244  0.6173640  4.685875
##   0.9526316  5.574534253  8.060539  0.5493237  5.726207
##   0.9526316  8.643403396  8.981890        NaN  6.527839
##   1.0000000  0.003221909  4.396680  0.7478873  3.169905
##   1.0000000  0.004995621  4.396680  0.7478873  3.169905
##   1.0000000  0.007745789  4.396957  0.7478527  3.169432
##   1.0000000  0.012009969  4.397539  0.7477299  3.168314
##   1.0000000  0.018621646  4.399131  0.7474823  3.166921
##   1.0000000  0.028873156  4.403087  0.7469849  3.164956
##   1.0000000  0.044768285  4.412830  0.7459301  3.163118
##   1.0000000  0.069413932  4.436482  0.7435991  3.165298
##   1.0000000  0.107627398  4.493079  0.7382599  3.181433
##   1.0000000  0.166877981  4.626159  0.7256406  3.236360
##   1.0000000  0.258746945  4.907690  0.6977800  3.390756
##   1.0000000  0.401191224  5.032621  0.6855163  3.485032
##   1.0000000  0.622053328  5.133679  0.6773621  3.559064
##   1.0000000  0.964503510  5.258624  0.6722645  3.631711
##   1.0000000  1.495477924  5.500174  0.6635281  3.790162
##   1.0000000  2.318762138  5.956837  0.6494340  4.103891
##   1.0000000  3.595277314  6.805997  0.6133316  4.752670
##   1.0000000  5.574534253  8.230544  0.5321871  5.868702
##   1.0000000  8.643403396  8.981890        NaN  6.527839
## 
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were alpha = 0.1 and lambda
##  = 0.01200997.

coef(lm4$finalModel,lm4$bestTune$lambda)

## 10 x 1 sparse Matrix of class "dgCMatrix"
##                        1
## (Intercept) 29.690474348
## lstat       -1.588736865
## I(lstat^2)   0.031167422
## rm           4.118561191
## dis         -1.263648344
## ptratio     -0.689451274
## nox         -9.184303117
## b            0.009552204
## zn           0.017107038
## crim        -0.108734020

pred4 <- predict(lm4,test)

error4 <- test$medv - pred4

#calculate rmse
rmse4 <- sqrt(mean(error4^2))
rmse4

## [1] 4.482597

plot(varImp(lm4))

Checking whether our assumptions are true

# Linear relationship between the dependent and independent variables
#check eda lm stat smooth


# Constant variance among residuals

#12 features
p1 <- ggplot(test, aes(x=pred1,y=test$medv-pred1))+geom_point(size = 1, alpha=0.4)

#10 features
p2 <- ggplot(test,aes(x=pred2,y=test$medv-pred2))+geom_point(size = 1,alpha=0.4)

#2 features
p3 <- ggplot(test,aes(x=pred3,y=test$medv-pred3))+geom_point(size = 1,alpha=0.4)

#glm all features
p4 <- ggplot(test,aes(x=pred4,y=test$medv-pred4))+geom_point(size = 1,alpha=0.4)

gridExtra::grid.arrange(p1,p2,p3,p4,nrow=2)

# No auto correlation
a1 <- ggplot(test,aes(x=row.names(test),y=test$medv-pred1)) + geom_point(size = 1, alpha=0.4)
a2 <- ggplot(test,aes(x=row.names(test),y=test$medv-pred3)) + geom_point(size = 1, alpha=0.4)
a3 <- ggplot(test,aes(x=row.names(test),y=test$medv-pred4)) + geom_point(size = 1, alpha=0.4)

gridExtra::grid.arrange(a1,a2,a3,nrow=2)

# Impact of outliers on the data via cooks distance
cooksd <- cooks.distance(lm3)
cooksddf <- data.frame(cooksdv=cooksd,id=c(1:length(cooksd)))

ggplot(data=cooksddf,aes(x=id,y=cooksdv,label=ifelse(cooksdv>4*mean(cooksdv),names(cooksd),"")))+
  geom_text() + geom_point() +
  geom_hline(yintercept = 4*mean(cooksd),col="red")

plot(cooksd, pch="*", cex=2, main="Influential Obs by Cooks distance")  # plot cook's distance
abline(h = 4*mean(cooksd, na.rm=T), col="red")  # add cutoff line
text(x=1:length(cooksd)+1, y=cooksd, labels=ifelse(cooksd>4*mean(cooksd, na.rm=T),names(cooksd),""), col="red")  # add labels

#get all the influential outliers
Ioutliers <- as.numeric(names(cooksd)[cooksd > 4*mean(cooksd,na.rm = T)])
bston[Ioutliers,]

##         crim   zn indus chas   nox    rm   age    dis rad tax ptratio
## 9    0.21124 12.5  7.87    0 0.524 5.631 100.0 6.0821   5 311    15.2
## 161  1.27346  0.0 19.58    1 0.605 6.250  92.6 1.7984   5 403    14.7
## 188  0.07875 45.0  3.44    0 0.437 6.782  41.1 3.7886   5 398    15.2
## 216  0.19802  0.0 10.59    0 0.489 6.182  42.4 3.9454   4 277    18.6
## 227  0.38214  0.0  6.20    0 0.504 8.040  86.5 3.2157   8 307    17.4
## 269  0.54050 20.0  3.97    0 0.575 7.470  52.6 2.8720   5 264    13.0
## 367  3.69695  0.0 18.10    0 0.718 4.963  91.4 1.7523  24 666    20.2
## 370  5.66998  0.0 18.10    1 0.631 6.683  96.8 1.3567  24 666    20.2
## 372  9.23230  0.0 18.10    0 0.631 6.216 100.0 1.1691  24 666    20.2
## 373  8.26725  0.0 18.10    1 0.668 5.875  89.6 1.1296  24 666    20.2
## 374 11.10810  0.0 18.10    0 0.668 4.906 100.0 1.1742  24 666    20.2
## 377 15.28800  0.0 18.10    0 0.671 6.649  93.3 1.3449  24 666    20.2
## 382 15.87440  0.0 18.10    0 0.671 6.545  99.1 1.5192  24 666    20.2
## 411 51.13580  0.0 18.10    0 0.597 5.757 100.0 1.4130  24 666    20.2
## 420 11.81230  0.0 18.10    0 0.718 6.824  76.5 1.7940  24 666    20.2
## 507  0.10959  0.0 11.93    0 0.573 6.794  89.3 2.3889   1 273    21.0
##          b lstat medv
## 9   386.63 29.93 16.5
## 161 338.92  5.50 27.0
## 188 393.87  6.68 32.0
## 216 393.63  9.47 25.0
## 227 387.38  3.13 37.6
## 269 390.30  3.16 43.5
## 367 316.03 14.00 21.9
## 370 375.33  3.73 50.0
## 372 366.15  9.53 50.0
## 373 347.88  8.88 50.0
## 374 396.90 34.77 13.8
## 377 363.02 23.24 13.9
## 382 396.90 21.08 10.9
## 411   2.60 10.11 15.0
## 420  48.45 22.74  8.4
## 507 393.45  6.48 22.0

#Capp outliers and run glm again

funout <- function(x){
    outlier <- quantile(x,c(0.05,0.95))
    quantiles <- quantile( x, c(.05, .95 ) )
    x[ x < outlier[1] ] <- quantiles[1]
    x[ x > outlier[2] ] <- quantiles[2]
    x
}

boston_o <-bston
for (i in colnames(bston[,-4])){boston_o[Ioutliers,i]<-funout(bston[Ioutliers,i])}



boston_o[Ioutliers,]

##         crim    zn   indus chas   nox      rm     age      dis   rad
## 9    0.21124 12.50  7.8700    0 0.524 5.63100 100.000 4.479575  5.00
## 161  1.27346  0.00 18.4700    1 0.605 6.25000  92.600 1.798400  5.00
## 188  0.10188 26.25  3.8375    0 0.476 6.78200  42.075 3.788600  5.00
## 216  0.19802  0.00 10.5900    0 0.489 6.18200  42.400 3.945400  4.00
## 227  0.38214  0.00  6.2000    0 0.504 7.61250  86.500 3.215700  8.00
## 269  0.54050 20.00  3.9700    0 0.575 7.47000  52.600 2.872000  5.00
## 367  3.69695  0.00 18.1000    0 0.718 4.96300  91.400 1.752300 24.00
## 370  5.66998  0.00 18.1000    1 0.631 6.68300  96.800 1.356700 24.00
## 372  9.23230  0.00 18.1000    0 0.631 6.21600 100.000 1.169100 24.00
## 373  8.26725  0.00 18.1000    1 0.668 5.87500  89.600 1.159225 24.00
## 374 11.10810  0.00 18.1000    0 0.668 4.94875 100.000 1.174200 24.00
## 377 15.28800  0.00 18.1000    0 0.671 6.64900  93.300 1.344900 24.00
## 382 15.87440  0.00 18.1000    0 0.671 6.54500  99.100 1.519200 24.00
## 411 24.68975  0.00 18.1000    0 0.597 5.75700 100.000 1.413000 24.00
## 420 11.81230  0.00 18.1000    0 0.718 6.82400  76.500 1.794000 24.00
## 507  0.10959  0.00 11.9300    0 0.573 6.79400  89.300 2.388900  3.25
##        tax ptratio        b   lstat   medv
## 9   311.00  15.200 386.6300 29.9300 16.500
## 161 403.00  14.700 338.9200  5.5000 27.000
## 188 398.00  15.200 393.8700  6.6800 32.000
## 216 277.00  18.600 393.6300  9.4700 25.000
## 227 307.00  17.400 387.3800  3.1525 37.600
## 269 270.75  14.275 390.3000  3.1600 43.500
## 367 666.00  20.200 316.0300 14.0000 21.900
## 370 666.00  20.200 375.3300  3.7300 50.000
## 372 666.00  20.200 366.1500  9.5300 50.000
## 373 666.00  20.200 347.8800  8.8800 50.000
## 374 666.00  20.200 396.9000 31.1400 13.800
## 377 666.00  20.200 363.0200 23.2400 13.900
## 382 666.00  20.200 396.9000 21.0800 10.900
## 411 666.00  20.200  36.9875 10.1100 15.000
## 420 666.00  20.200  48.4500 22.7400 10.275
## 507 273.00  20.400 393.4500  6.4800 22.000

bston[Ioutliers,]

##         crim   zn indus chas   nox    rm   age    dis rad tax ptratio
## 9    0.21124 12.5  7.87    0 0.524 5.631 100.0 6.0821   5 311    15.2
## 161  1.27346  0.0 19.58    1 0.605 6.250  92.6 1.7984   5 403    14.7
## 188  0.07875 45.0  3.44    0 0.437 6.782  41.1 3.7886   5 398    15.2
## 216  0.19802  0.0 10.59    0 0.489 6.182  42.4 3.9454   4 277    18.6
## 227  0.38214  0.0  6.20    0 0.504 8.040  86.5 3.2157   8 307    17.4
## 269  0.54050 20.0  3.97    0 0.575 7.470  52.6 2.8720   5 264    13.0
## 367  3.69695  0.0 18.10    0 0.718 4.963  91.4 1.7523  24 666    20.2
## 370  5.66998  0.0 18.10    1 0.631 6.683  96.8 1.3567  24 666    20.2
## 372  9.23230  0.0 18.10    0 0.631 6.216 100.0 1.1691  24 666    20.2
## 373  8.26725  0.0 18.10    1 0.668 5.875  89.6 1.1296  24 666    20.2
## 374 11.10810  0.0 18.10    0 0.668 4.906 100.0 1.1742  24 666    20.2
## 377 15.28800  0.0 18.10    0 0.671 6.649  93.3 1.3449  24 666    20.2
## 382 15.87440  0.0 18.10    0 0.671 6.545  99.1 1.5192  24 666    20.2
## 411 51.13580  0.0 18.10    0 0.597 5.757 100.0 1.4130  24 666    20.2
## 420 11.81230  0.0 18.10    0 0.718 6.824  76.5 1.7940  24 666    20.2
## 507  0.10959  0.0 11.93    0 0.573 6.794  89.3 2.3889   1 273    21.0
##          b lstat medv
## 9   386.63 29.93 16.5
## 161 338.92  5.50 27.0
## 188 393.87  6.68 32.0
## 216 393.63  9.47 25.0
## 227 387.38  3.13 37.6
## 269 390.30  3.16 43.5
## 367 316.03 14.00 21.9
## 370 375.33  3.73 50.0
## 372 366.15  9.53 50.0
## 373 347.88  8.88 50.0
## 374 396.90 34.77 13.8
## 377 363.02 23.24 13.9
## 382 396.90 21.08 10.9
## 411   2.60 10.11 15.0
## 420  48.45 22.74  8.4
## 507 393.45  6.48 22.0

Try regression again

set.seed(8)
split <- sample(1:2,size = nrow(boston_o),replace = T,prob = c(0.8,0.2))
train2 <- boston_o[split==1,]
test2 <- boston_o[split==2,]

#glmnet
lm5 <- train(medv~lstat+I(lstat^2)+rm+dis+ptratio+nox+b+zn+crim,data=train2,
             method="glmnet",trControl=fitcontrol,tuneLength=20)

## Warning in nominalTrainWorkflow(x = x, y = y, wts = weights, info =
## trainInfo, : There were missing values in resampled performance measures.

lm5

## glmnet 
## 
## 398 samples
##   8 predictor
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold) 
## Summary of sample sizes: 359, 358, 358, 358, 358, 358, ... 
## Resampling results across tuning parameters:
## 
##   alpha      lambda       RMSE      Rsquared   MAE     
##   0.1000000  0.003222242  4.486507  0.7665059  3.229965
##   0.1000000  0.004996137  4.486507  0.7665059  3.229965
##   0.1000000  0.007746589  4.485983  0.7665683  3.229601
##   0.1000000  0.012011209  4.486052  0.7666291  3.229074
##   0.1000000  0.018623570  4.486736  0.7666688  3.228337
##   0.1000000  0.028876139  4.488794  0.7666374  3.227202
##   0.1000000  0.044772909  4.493895  0.7664044  3.226990
##   0.1000000  0.069421103  4.505069  0.7657220  3.229748
##   0.1000000  0.107638517  4.526870  0.7641997  3.237325
##   0.1000000  0.166895220  4.564858  0.7613408  3.252867
##   0.1000000  0.258773675  4.621028  0.7569876  3.275618
##   0.1000000  0.401232669  4.696134  0.7510588  3.306627
##   0.1000000  0.622117590  4.789737  0.7435687  3.357951
##   0.1000000  0.964603149  4.895586  0.7352057  3.430169
##   0.1000000  1.495632416  4.968217  0.7302089  3.489355
##   0.1000000  2.319001680  5.100009  0.7211397  3.589121
##   0.1000000  3.595648728  5.300988  0.7067464  3.746572
##   0.1000000  5.575110137  5.532095  0.6962140  3.921731
##   0.1000000  8.644296312  5.845681  0.6905864  4.161053
##   0.1473684  0.003222242  4.486985  0.7664599  3.230278
##   0.1473684  0.004996137  4.486985  0.7664599  3.230278
##   0.1473684  0.007746589  4.486024  0.7665593  3.229693
##   0.1473684  0.012011209  4.486145  0.7666126  3.229230
##   0.1473684  0.018623570  4.486902  0.7666431  3.228578
##   0.1473684  0.028876139  4.489049  0.7665986  3.227573
##   0.1473684  0.044772909  4.494374  0.7663411  3.227431
##   0.1473684  0.069421103  4.505928  0.7656204  3.230497
##   0.1473684  0.107638517  4.528836  0.7639987  3.238649
##   0.1473684  0.166895220  4.569154  0.7609372  3.255593
##   0.1473684  0.258773675  4.628830  0.7562989  3.280297
##   0.1473684  0.401232669  4.710458  0.7498129  3.316173
##   0.1473684  0.622117590  4.815195  0.7413858  3.375650
##   0.1473684  0.964603149  4.920130  0.7330240  3.453017
##   0.1473684  1.495632416  5.001555  0.7271573  3.517520
##   0.1473684  2.319001680  5.148363  0.7165588  3.631471
##   0.1473684  3.595648728  5.359072  0.7033732  3.792429
##   0.1473684  5.575110137  5.607998  0.6980163  3.986952
##   0.1473684  8.644296312  6.014161  0.6917788  4.290046
##   0.1947368  0.003222242  4.487106  0.7664427  3.230410
##   0.1947368  0.004996137  4.487106  0.7664427  3.230410
##   0.1947368  0.007746589  4.486092  0.7665476  3.229797
##   0.1947368  0.012011209  4.486266  0.7665937  3.229397
##   0.1947368  0.018623570  4.487092  0.7666142  3.228835
##   0.1947368  0.028876139  4.489333  0.7665572  3.227949
##   0.1947368  0.044772909  4.494817  0.7662799  3.227863
##   0.1947368  0.069421103  4.506896  0.7655092  3.231279
##   0.1947368  0.107638517  4.530870  0.7637912  3.240058
##   0.1947368  0.166895220  4.573646  0.7605171  3.258410
##   0.1947368  0.258773675  4.636933  0.7555861  3.284887
##   0.1947368  0.401232669  4.726800  0.7483676  3.327848
##   0.1947368  0.622117590  4.845605  0.7388213  3.398671
##   0.1947368  0.964603149  4.941879  0.7310534  3.473235
##   0.1947368  1.495632416  5.038425  0.7236690  3.549891
##   0.1947368  2.319001680  5.200970  0.7113155  3.674937
##   0.1947368  3.595648728  5.396527  0.7043008  3.822028
##   0.1947368  5.575110137  5.699578  0.6982996  4.056714
##   0.1947368  8.644296312  6.210330  0.6888518  4.438767
##   0.2421053  0.003222242  4.486610  0.7664811  3.230152
##   0.2421053  0.004996137  4.486610  0.7664811  3.230152
##   0.2421053  0.007746589  4.486167  0.7665356  3.229891
##   0.2421053  0.012011209  4.486403  0.7665737  3.229564
##   0.2421053  0.018623570  4.487249  0.7665888  3.229060
##   0.2421053  0.028876139  4.489614  0.7665156  3.228320
##   0.2421053  0.044772909  4.495319  0.7662141  3.228290
##   0.2421053  0.069421103  4.507868  0.7653970  3.232104
##   0.2421053  0.107638517  4.533063  0.7635689  3.241544
##   0.2421053  0.166895220  4.577929  0.7601212  3.261116
##   0.2421053  0.258773675  4.645657  0.7548170  3.289641
##   0.2421053  0.401232669  4.744622  0.7468098  3.340790
##   0.2421053  0.622117590  4.879261  0.7359231  3.424150
##   0.2421053  0.964603149  4.967050  0.7286722  3.496424
##   0.2421053  1.495632416  5.077918  0.7197233  3.585268
##   0.2421053  2.319001680  5.238608  0.7090422  3.706476
##   0.2421053  3.595648728  5.444150  0.7046720  3.857630
##   0.2421053  5.575110137  5.810802  0.6968289  4.141279
##   0.2421053  8.644296312  6.412643  0.6858435  4.592161
##   0.2894737  0.003222242  4.486945  0.7664519  3.230352
##   0.2894737  0.004996137  4.486945  0.7664519  3.230352
##   0.2894737  0.007746589  4.486221  0.7665253  3.229985
##   0.2894737  0.012011209  4.486511  0.7665563  3.229713
##   0.2894737  0.018623570  4.487441  0.7665603  3.229303
##   0.2894737  0.028876139  4.489904  0.7664739  3.228684
##   0.2894737  0.044772909  4.495846  0.7661458  3.228733
##   0.2894737  0.069421103  4.508920  0.7652774  3.232936
##   0.2894737  0.107638517  4.535340  0.7633389  3.243128
##   0.2894737  0.166895220  4.582402  0.7597097  3.263903
##   0.2894737  0.258773675  4.655166  0.7539723  3.295196
##   0.2894737  0.401232669  4.764360  0.7451484  3.354452
##   0.2894737  0.622117590  4.907176  0.7333911  3.446674
##   0.2894737  0.964603149  4.992017  0.7262222  3.518459
##   0.2894737  1.495632416  5.125254  0.7147444  3.625527
##   0.2894737  2.319001680  5.267858  0.7083496  3.732741
##   0.2894737  3.595648728  5.499603  0.7043491  3.902252
##   0.2894737  5.575110137  5.936807  0.6932207  4.233550
##   0.2894737  8.644296312  6.610293  0.6852780  4.735670
##   0.3368421  0.003222242  4.486823  0.7664518  3.230402
##   0.3368421  0.004996137  4.486823  0.7664518  3.230402
##   0.3368421  0.007746589  4.486270  0.7665161  3.230067
##   0.3368421  0.012011209  4.486613  0.7665391  3.229864
##   0.3368421  0.018623570  4.487624  0.7665319  3.229555
##   0.3368421  0.028876139  4.490197  0.7664316  3.229045
##   0.3368421  0.044772909  4.496370  0.7660776  3.229154
##   0.3368421  0.069421103  4.510007  0.7651541  3.233874
##   0.3368421  0.107638517  4.537755  0.7630955  3.244797
##   0.3368421  0.166895220  4.587172  0.7592708  3.267094
##   0.3368421  0.258773675  4.665546  0.7530440  3.301770
##   0.3368421  0.401232669  4.786345  0.7432943  3.369015
##   0.3368421  0.622117590  4.929596  0.7312749  3.465779
##   0.3368421  0.964603149  5.017704  0.7236214  3.541423
##   0.3368421  1.495632416  5.173519  0.7096327  3.664488
##   0.3368421  2.319001680  5.300881  0.7073060  3.760786
##   0.3368421  3.595648728  5.561594  0.7032372  3.952031
##   0.3368421  5.575110137  6.051427  0.6923458  4.321150
##   0.3368421  8.644296312  6.828892  0.6824409  4.895198
##   0.3842105  0.003222242  4.486832  0.7664546  3.230414
##   0.3842105  0.004996137  4.486832  0.7664546  3.230414
##   0.3842105  0.007746589  4.486311  0.7665074  3.230146
##   0.3842105  0.012011209  4.486693  0.7665247  3.230008
##   0.3842105  0.018623570  4.487735  0.7665112  3.229762
##   0.3842105  0.028876139  4.490462  0.7663924  3.229402
##   0.3842105  0.044772909  4.496859  0.7660132  3.229612
##   0.3842105  0.069421103  4.511070  0.7650333  3.234805
##   0.3842105  0.107638517  4.540279  0.7628424  3.246498
##   0.3842105  0.166895220  4.592158  0.7588132  3.270395
##   0.3842105  0.258773675  4.676777  0.7520373  3.309224
##   0.3842105  0.401232669  4.810491  0.7412230  3.384796
##   0.3842105  0.622117590  4.948010  0.7294906  3.483317
##   0.3842105  0.964603149  5.046766  0.7206033  3.566717
##   0.3842105  1.495632416  5.199200  0.7077512  3.685343
##   0.3842105  2.319001680  5.336732  0.7059796  3.789605
##   0.3842105  3.595648728  5.632233  0.7009203  4.009701
##   0.3842105  5.575110137  6.165170  0.6929774  4.409691
##   0.3842105  8.644296312  7.065454  0.6759154  5.074781
##   0.4315789  0.003222242  4.486263  0.7664993  3.230172
##   0.4315789  0.004996137  4.486263  0.7664993  3.230172
##   0.4315789  0.007746589  4.486490  0.7664862  3.230326
##   0.4315789  0.012011209  4.486788  0.7665079  3.230180
##   0.4315789  0.018623570  4.487922  0.7664826  3.230029
##   0.4315789  0.028876139  4.490740  0.7663509  3.229787
##   0.4315789  0.044772909  4.497420  0.7659407  3.230132
##   0.4315789  0.069421103  4.512220  0.7649040  3.235823
##   0.4315789  0.107638517  4.543039  0.7625667  3.248314
##   0.4315789  0.166895220  4.597432  0.7583299  3.273746
##   0.4315789  0.258773675  4.688905  0.7509509  3.317078
##   0.4315789  0.401232669  4.837901  0.7388274  3.403138
##   0.4315789  0.622117590  4.963638  0.7279419  3.497339
##   0.4315789  0.964603149  5.079433  0.7170978  3.593408
##   0.4315789  1.495632416  5.216795  0.7070705  3.702049
##   0.4315789  2.319001680  5.374036  0.7046823  3.818989
##   0.4315789  3.595648728  5.708062  0.6973571  4.069312
##   0.4315789  5.575110137  6.290503  0.6916297  4.505035
##   0.4315789  8.644296312  7.300813  0.6634869  5.254911
##   0.4789474  0.003222242  4.487129  0.7664271  3.230603
##   0.4789474  0.004996137  4.487129  0.7664271  3.230603
##   0.4789474  0.007746589  4.486566  0.7664760  3.230453
##   0.4789474  0.012011209  4.486921  0.7664892  3.230362
##   0.4789474  0.018623570  4.488134  0.7664533  3.230300
##   0.4789474  0.028876139  4.491107  0.7663031  3.230195
##   0.4789474  0.044772909  4.498020  0.7658660  3.230677
##   0.4789474  0.069421103  4.513458  0.7647679  3.236830
##   0.4789474  0.107638517  4.545924  0.7622795  3.250146
##   0.4789474  0.166895220  4.602870  0.7578353  3.276998
##   0.4789474  0.258773675  4.701752  0.7498317  3.325460
##   0.4789474  0.401232669  4.867824  0.7361948  3.423142
##   0.4789474  0.622117590  4.979676  0.7263025  3.511403
##   0.4789474  0.964603149  5.115036  0.7131460  3.623678
##   0.4789474  1.495632416  5.234320  0.7065259  3.718460
##   0.4789474  2.319001680  5.413687  0.7030570  3.849987
##   0.4789474  3.595648728  5.773127  0.6959009  4.120926
##   0.4789474  5.575110137  6.429179  0.6867913  4.609469
##   0.4789474  8.644296312  7.509875  0.6600502  5.406470
##   0.5263158  0.003222242  4.486963  0.7664295  3.230668
##   0.5263158  0.004996137  4.486963  0.7664295  3.230668
##   0.5263158  0.007746589  4.486603  0.7664672  3.230550
##   0.5263158  0.012011209  4.487012  0.7664732  3.230518
##   0.5263158  0.018623570  4.488310  0.7664261  3.230539
##   0.5263158  0.028876139  4.491396  0.7662605  3.230571
##   0.5263158  0.044772909  4.498630  0.7657895  3.231230
##   0.5263158  0.069421103  4.514739  0.7646255  3.237841
##   0.5263158  0.107638517  4.548824  0.7619923  3.252107
##   0.5263158  0.166895220  4.608597  0.7573133  3.280412
##   0.5263158  0.258773675  4.715672  0.7486512  3.335061
##   0.5263158  0.401232669  4.894743  0.7337765  3.442530
##   0.5263158  0.622117590  4.997034  0.7245201  3.526725
##   0.5263158  0.964603149  5.149741  0.7092965  3.651976
##   0.5263158  1.495632416  5.253453  0.7058202  3.735383
##   0.5263158  2.319001680  5.455333  0.7009699  3.884661
##   0.5263158  3.595648728  5.843057  0.6943164  4.175790
##   0.5263158  5.575110137  6.577975  0.6796695  4.722607
##   0.5263158  8.644296312  7.729412  0.6579140  5.569711
##   0.5736842  0.003222242  4.486623  0.7664518  3.230553
##   0.5736842  0.004996137  4.486623  0.7664518  3.230553
##   0.5736842  0.007746589  4.486712  0.7664521  3.230683
##   0.5736842  0.012011209  4.487151  0.7664523  3.230678
##   0.5736842  0.018623570  4.488505  0.7663965  3.230784
##   0.5736842  0.028876139  4.491738  0.7662126  3.230940
##   0.5736842  0.044772909  4.499275  0.7657093  3.231781
##   0.5736842  0.069421103  4.516039  0.7644817  3.238828
##   0.5736842  0.107638517  4.551716  0.7617052  3.253932
##   0.5736842  0.166895220  4.614724  0.7567536  3.283887
##   0.5736842  0.258773675  4.730787  0.7473715  3.345336
##   0.5736842  0.401232669  4.917410  0.7317427  3.459834
##   0.5736842  0.622117590  5.016331  0.7225009  3.544019
##   0.5736842  0.964603149  5.170940  0.7072953  3.668425
##   0.5736842  1.495632416  5.273321  0.7050226  3.752483
##   0.5736842  2.319001680  5.500062  0.6983878  3.921090
##   0.5736842  3.595648728  5.918677  0.6921300  4.232685
##   0.5736842  5.575110137  6.737015  0.6694086  4.845863
##   0.5736842  8.644296312  7.965451  0.6539365  5.746677
##   0.6210526  0.003222242  4.486773  0.7664384  3.230649
##   0.6210526  0.004996137  4.486773  0.7664384  3.230649
##   0.6210526  0.007746589  4.486758  0.7664421  3.230782
##   0.6210526  0.012011209  4.487265  0.7664343  3.230832
##   0.6210526  0.018623570  4.488704  0.7663664  3.231038
##   0.6210526  0.028876139  4.492079  0.7661651  3.231328
##   0.6210526  0.044772909  4.499873  0.7656325  3.232334
##   0.6210526  0.069421103  4.517417  0.7643291  3.239939
##   0.6210526  0.107638517  4.554644  0.7614144  3.256012
##   0.6210526  0.166895220  4.621241  0.7561561  3.287534
##   0.6210526  0.258773675  4.747112  0.7459733  3.356236
##   0.6210526  0.401232669  4.934693  0.7301405  3.474030
##   0.6210526  0.622117590  5.037444  0.7202385  3.562322
##   0.6210526  0.964603149  5.181897  0.7067783  3.678324
##   0.6210526  1.495632416  5.294147  0.7040525  3.770188
##   0.6210526  2.319001680  5.542328  0.6962334  3.956737
##   0.6210526  3.595648728  5.999237  0.6893363  4.294519
##   0.6210526  5.575110137  6.887382  0.6617040  4.962000
##   0.6210526  8.644296312  8.219066  0.6451542  5.940658
##   0.6684211  0.003222242  4.486194  0.7664655  3.230428
##   0.6684211  0.004996137  4.486194  0.7664655  3.230428
##   0.6684211  0.007746589  4.486834  0.7664297  3.230826
##   0.6684211  0.012011209  4.487386  0.7664154  3.230989
##   0.6684211  0.018623570  4.488897  0.7663373  3.231277
##   0.6684211  0.028876139  4.492418  0.7661180  3.231697
##   0.6684211  0.044772909  4.500564  0.7655476  3.232921
##   0.6684211  0.069421103  4.518808  0.7641761  3.241067
##   0.6684211  0.107638517  4.557479  0.7611364  3.257965
##   0.6684211  0.166895220  4.628187  0.7555189  3.291531
##   0.6684211  0.258773675  4.764872  0.7444429  3.367451
##   0.6684211  0.401232669  4.949336  0.7287492  3.486944
##   0.6684211  0.622117590  5.060275  0.7177260  3.581064
##   0.6684211  0.964603149  5.191754  0.7063868  3.688533
##   0.6684211  1.495632416  5.316494  0.7028641  3.788497
##   0.6684211  2.319001680  5.580171  0.6951125  3.988823
##   0.6684211  3.595648728  6.084804  0.6857902  4.361617
##   0.6684211  5.575110137  7.024281  0.6593422  5.059383
##   0.6684211  8.644296312  8.485320  0.6255156  6.148585
##   0.7157895  0.003222242  4.486618  0.7664263  3.230721
##   0.7157895  0.004996137  4.486618  0.7664263  3.230721
##   0.7157895  0.007746589  4.486978  0.7664125  3.230949
##   0.7157895  0.012011209  4.487531  0.7663943  3.231159
##   0.7157895  0.018623570  4.489101  0.7663069  3.231523
##   0.7157895  0.028876139  4.492777  0.7660688  3.232101
##   0.7157895  0.044772909  4.501216  0.7654662  3.233505
##   0.7157895  0.069421103  4.520308  0.7640125  3.242233
##   0.7157895  0.107638517  4.560453  0.7608455  3.259953
##   0.7157895  0.166895220  4.635568  0.7548400  3.295990
##   0.7157895  0.258773675  4.784253  0.7427665  3.379779
##   0.7157895  0.401232669  4.960606  0.7276048  3.496421
##   0.7157895  0.622117590  5.084794  0.7149597  3.601363
##   0.7157895  0.964603149  5.201805  0.7059712  3.698618
##   0.7157895  1.495632416  5.340345  0.7014828  3.807411
##   0.7157895  2.319001680  5.620597  0.6939186  4.021197
##   0.7157895  3.595648728  6.175454  0.6813044  4.432389
##   0.7157895  5.575110137  7.166080  0.6579556  5.161048
##   0.7157895  8.644296312  8.750105  0.5870388  6.360626
##   0.7631579  0.003222242  4.486619  0.7664233  3.230768
##   0.7631579  0.004996137  4.486619  0.7664233  3.230768
##   0.7631579  0.007746589  4.487035  0.7664022  3.231025
##   0.7631579  0.012011209  4.487627  0.7663779  3.231293
##   0.7631579  0.018623570  4.489323  0.7662754  3.231767
##   0.7631579  0.028876139  4.493174  0.7660158  3.232529
##   0.7631579  0.044772909  4.501973  0.7653750  3.234178
##   0.7631579  0.069421103  4.521858  0.7638437  3.243416
##   0.7631579  0.107638517  4.563540  0.7605437  3.261993
##   0.7631579  0.166895220  4.643445  0.7541120  3.301007
##   0.7631579  0.258773675  4.805153  0.7409503  3.392726
##   0.7631579  0.401232669  4.971313  0.7265039  3.506160
##   0.7631579  0.622117590  5.110750  0.7119628  3.623452
##   0.7631579  0.964603149  5.212373  0.7054819  3.708966
##   0.7631579  1.495632416  5.365740  0.6998899  3.827068
##   0.7631579  2.319001680  5.663247  0.6925216  4.054436
##   0.7631579  3.595648728  6.271444  0.6756361  4.506718
##   0.7631579  5.575110137  7.317116  0.6560076  5.267156
##   0.7631579  8.644296312  8.986873  0.5563805  6.558086
##   0.8105263  0.003222242  4.486643  0.7664202  3.230812
##   0.8105263  0.004996137  4.486643  0.7664202  3.230812
##   0.8105263  0.007746589  4.487062  0.7663943  3.231153
##   0.8105263  0.012011209  4.487757  0.7663580  3.231459
##   0.8105263  0.018623570  4.489524  0.7662454  3.232010
##   0.8105263  0.028876139  4.493503  0.7659692  3.232924
##   0.8105263  0.044772909  4.502653  0.7652905  3.234872
##   0.8105263  0.069421103  4.523428  0.7636733  3.244573
##   0.8105263  0.107638517  4.566850  0.7602199  3.264235
##   0.8105263  0.166895220  4.651912  0.7533387  3.306188
##   0.8105263  0.258773675  4.827728  0.7389796  3.406515
##   0.8105263  0.401232669  4.982881  0.7253064  3.516614
##   0.8105263  0.622117590  5.135252  0.7091693  3.643490
##   0.8105263  0.964603149  5.223621  0.7049002  3.719696
##   0.8105263  1.495632416  5.392464  0.6981016  3.846922
##   0.8105263  2.319001680  5.708229  0.6908785  4.089993
##   0.8105263  3.595648728  6.372874  0.6685343  4.586080
##   0.8105263  5.575110137  7.477245  0.6531587  5.379379
##   0.8105263  8.644296312  9.061105        NaN  6.620150
##   0.8578947  0.003222242  4.486688  0.7664085  3.230937
##   0.8578947  0.004996137  4.486688  0.7664085  3.230937
##   0.8578947  0.007746589  4.487123  0.7663841  3.231166
##   0.8578947  0.012011209  4.487875  0.7663391  3.231617
##   0.8578947  0.018623570  4.489714  0.7662161  3.232249
##   0.8578947  0.028876139  4.493865  0.7659198  3.233328
##   0.8578947  0.044772909  4.503394  0.7652005  3.235642
##   0.8578947  0.069421103  4.525054  0.7634965  3.245783
##   0.8578947  0.107638517  4.570271  0.7598876  3.266536
##   0.8578947  0.166895220  4.660281  0.7525965  3.311633
##   0.8578947  0.258773675  4.852130  0.7368325  3.420933
##   0.8578947  0.401232669  4.995259  0.7240058  3.527584
##   0.8578947  0.622117590  5.150688  0.7075959  3.655650
##   0.8578947  0.964603149  5.235531  0.7042394  3.730778
##   0.8578947  1.495632416  5.419601  0.6962770  3.867779
##   0.8578947  2.319001680  5.755599  0.6889551  4.126969
##   0.8578947  3.595648728  6.468745  0.6630114  4.661438
##   0.8578947  5.575110137  7.647252  0.6488033  5.504475
##   0.8578947  8.644296312  9.061105        NaN  6.620150
##   0.9052632  0.003222242  4.486590  0.7664163  3.230898
##   0.9052632  0.004996137  4.486590  0.7664163  3.230898
##   0.9052632  0.007746589  4.487205  0.7663703  3.231315
##   0.9052632  0.012011209  4.488027  0.7663172  3.231790
##   0.9052632  0.018623570  4.489964  0.7661817  3.232509
##   0.9052632  0.028876139  4.494283  0.7658641  3.233769
##   0.9052632  0.044772909  4.504149  0.7651084  3.236454
##   0.9052632  0.069421103  4.526734  0.7633148  3.247028
##   0.9052632  0.107638517  4.573856  0.7595402  3.269151
##   0.9052632  0.166895220  4.669088  0.7518235  3.317177
##   0.9052632  0.258773675  4.876744  0.7346910  3.435379
##   0.9052632  0.401232669  5.008566  0.7225902  3.539479
##   0.9052632  0.622117590  5.160852  0.7067255  3.664974
##   0.9052632  0.964603149  5.248106  0.7035037  3.742198
##   0.9052632  1.495632416  5.443289  0.6950770  3.886550
##   0.9052632  2.319001680  5.805368  0.6867082  4.165220
##   0.9052632  3.595648728  6.559015  0.6590726  4.729818
##   0.9052632  5.575110137  7.827663  0.6417069  5.638018
##   0.9052632  8.644296312  9.061105        NaN  6.620150
##   0.9526316  0.003222242  4.486712  0.7663932  3.231083
##   0.9526316  0.004996137  4.486712  0.7663932  3.231083
##   0.9526316  0.007746589  4.487228  0.7663621  3.231378
##   0.9526316  0.012011209  4.488172  0.7662962  3.231948
##   0.9526316  0.018623570  4.490148  0.7661528  3.232735
##   0.9526316  0.028876139  4.494676  0.7658116  3.234185
##   0.9526316  0.044772909  4.504910  0.7650158  3.237254
##   0.9526316  0.069421103  4.528486  0.7631256  3.248319
##   0.9526316  0.107638517  4.577317  0.7592047  3.271692
##   0.9526316  0.166895220  4.678336  0.7510294  3.323310
##   0.9526316  0.258773675  4.895331  0.7330685  3.447181
##   0.9526316  0.401232669  5.022735  0.7210556  3.551970
##   0.9526316  0.622117590  5.167815  0.7063294  3.671082
##   0.9526316  0.964603149  5.261343  0.7026898  3.753883
##   0.9526316  1.495632416  5.465881  0.6942316  3.905305
##   0.9526316  2.319001680  5.857652  0.6840858  4.204977
##   0.9526316  3.595648728  6.643412  0.6577154  4.790451
##   0.9526316  5.575110137  8.018595  0.6292279  5.780188
##   0.9526316  8.644296312  9.061105        NaN  6.620150
##   1.0000000  0.003222242  4.487009  0.7663680  3.231205
##   1.0000000  0.004996137  4.487009  0.7663680  3.231205
##   1.0000000  0.007746589  4.487359  0.7663454  3.231526
##   1.0000000  0.012011209  4.488268  0.7662791  3.232099
##   1.0000000  0.018623570  4.490390  0.7661195  3.232990
##   1.0000000  0.028876139  4.495083  0.7657574  3.234616
##   1.0000000  0.044772909  4.505744  0.7649162  3.238089
##   1.0000000  0.069421103  4.530300  0.7629306  3.249675
##   1.0000000  0.107638517  4.580924  0.7588596  3.274318
##   1.0000000  0.166895220  4.688039  0.7502008  3.329427
##   1.0000000  0.258773675  4.911963  0.7315922  3.458503
##   1.0000000  0.401232669  5.037714  0.7194037  3.564920
##   1.0000000  0.622117590  5.173498  0.7060614  3.677207
##   1.0000000  0.964603149  5.275274  0.7017898  3.765707
##   1.0000000  1.495632416  5.489042  0.6933963  3.924306
##   1.0000000  2.319001680  5.912470  0.6810328  4.246484
##   1.0000000  3.595648728  6.730852  0.6564972  4.851725
##   1.0000000  5.575110137  8.217080  0.6082194  5.931233
##   1.0000000  8.644296312  9.061105        NaN  6.620150
## 
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were alpha = 0.1 and lambda
##  = 0.007746589.

coef(lm5$finalModel,lm5$bestTune$lambda)

## 10 x 1 sparse Matrix of class "dgCMatrix"
##                         1
## (Intercept)  35.970873199
## lstat        -1.590139720
## I(lstat^2)    0.032050619
## rm            3.783947122
## dis          -1.236610615
## ptratio      -0.794952993
## nox         -12.919261187
## b             0.007793906
## zn            0.009427535
## crim         -0.096192822

pred5 <- predict(lm5,test)

error5 <- test$medv - pred5

#calculate rmse
rmse5 <- sqrt(mean(error5^2))
rmse5

## [1] 4.366865

summary(
  resamples(
    list(
      model2 = lm4, 
      model3 = lm5
    )
  )
)$statistics

## $MAE
##            Min.  1st Qu.   Median     Mean  3rd Qu.     Max. NA's
## model2 2.154784 2.750033 3.031127 3.167074 3.417708 4.873727    0
## model3 2.459318 3.083728 3.197260 3.229601 3.435272 3.944517    0
## 
## $RMSE
##            Min.  1st Qu.   Median     Mean  3rd Qu.     Max. NA's
## model2 2.813613 3.572523 3.913188 4.395707 4.661682 7.955098    0
## model3 3.507457 4.062833 4.384034 4.485983 4.565483 5.891778    0
## 
## $Rsquared
##             Min.   1st Qu.    Median      Mean   3rd Qu.      Max. NA's
## model2 0.4242442 0.6714829 0.8291640 0.7479758 0.8548048 0.9163700    0
## model3 0.6243192 0.7558034 0.7765594 0.7665683 0.8072759 0.8369336    0

Linearboston-ref

Vishwa

11/30/2019

Load the data

Visualize plots

Create linear models

Checking whether our assumptions are true

Try regression again