Kaggle - House Prices
Saurabh
31 July 2017
Problem Statement
Ask a home buyer to describe their dream house, and they probably won’t begin with the height of the basement ceiling or the proximity to an east-west railroad. But this playground competition’s dataset proves that much more influences price negotiations than the number of bedrooms or a white-picket fence.
With 79 explanatory variables describing (almost) every aspect of residential homes in Ames, Iowa, this competition challenges you to predict the final price of each home.
Data Loading
train <- read.csv("./input/train.csv")
test <- read.csv("./input/test.csv")
sample_submission <- read.csv("./input/sample_submission.csv")Data Preperation
# Add the sales price variable to the test set
test$SalePrice <- 0
# Combine the train and test data sets
combi <- rbind(train,test)NA Treatment
str(train)## 'data.frame': 1460 obs. of 81 variables:
## $ Id : int 1 2 3 4 5 6 7 8 9 10 ...
## $ MSSubClass : int 60 20 60 70 60 50 20 60 50 190 ...
## $ MSZoning : Factor w/ 5 levels "C (all)","FV",..: 4 4 4 4 4 4 4 4 5 4 ...
## $ LotFrontage : int 65 80 68 60 84 85 75 NA 51 50 ...
## $ LotArea : int 8450 9600 11250 9550 14260 14115 10084 10382 6120 7420 ...
## $ Street : Factor w/ 2 levels "Grvl","Pave": 2 2 2 2 2 2 2 2 2 2 ...
## $ Alley : Factor w/ 2 levels "Grvl","Pave": NA NA NA NA NA NA NA NA NA NA ...
## $ LotShape : Factor w/ 4 levels "IR1","IR2","IR3",..: 4 4 1 1 1 1 4 1 4 4 ...
## $ LandContour : Factor w/ 4 levels "Bnk","HLS","Low",..: 4 4 4 4 4 4 4 4 4 4 ...
## $ Utilities : Factor w/ 2 levels "AllPub","NoSeWa": 1 1 1 1 1 1 1 1 1 1 ...
## $ LotConfig : Factor w/ 5 levels "Corner","CulDSac",..: 5 3 5 1 3 5 5 1 5 1 ...
## $ LandSlope : Factor w/ 3 levels "Gtl","Mod","Sev": 1 1 1 1 1 1 1 1 1 1 ...
## $ Neighborhood : Factor w/ 25 levels "Blmngtn","Blueste",..: 6 25 6 7 14 12 21 17 18 4 ...
## $ Condition1 : Factor w/ 9 levels "Artery","Feedr",..: 3 2 3 3 3 3 3 5 1 1 ...
## $ Condition2 : Factor w/ 8 levels "Artery","Feedr",..: 3 3 3 3 3 3 3 3 3 1 ...
## $ BldgType : Factor w/ 5 levels "1Fam","2fmCon",..: 1 1 1 1 1 1 1 1 1 2 ...
## $ HouseStyle : Factor w/ 8 levels "1.5Fin","1.5Unf",..: 6 3 6 6 6 1 3 6 1 2 ...
## $ OverallQual : int 7 6 7 7 8 5 8 7 7 5 ...
## $ OverallCond : int 5 8 5 5 5 5 5 6 5 6 ...
## $ YearBuilt : int 2003 1976 2001 1915 2000 1993 2004 1973 1931 1939 ...
## $ YearRemodAdd : int 2003 1976 2002 1970 2000 1995 2005 1973 1950 1950 ...
## $ RoofStyle : Factor w/ 6 levels "Flat","Gable",..: 2 2 2 2 2 2 2 2 2 2 ...
## $ RoofMatl : Factor w/ 8 levels "ClyTile","CompShg",..: 2 2 2 2 2 2 2 2 2 2 ...
## $ Exterior1st : Factor w/ 15 levels "AsbShng","AsphShn",..: 13 9 13 14 13 13 13 7 4 9 ...
## $ Exterior2nd : Factor w/ 16 levels "AsbShng","AsphShn",..: 14 9 14 16 14 14 14 7 16 9 ...
## $ MasVnrType : Factor w/ 4 levels "BrkCmn","BrkFace",..: 2 3 2 3 2 3 4 4 3 3 ...
## $ MasVnrArea : int 196 0 162 0 350 0 186 240 0 0 ...
## $ ExterQual : Factor w/ 4 levels "Ex","Fa","Gd",..: 3 4 3 4 3 4 3 4 4 4 ...
## $ ExterCond : Factor w/ 5 levels "Ex","Fa","Gd",..: 5 5 5 5 5 5 5 5 5 5 ...
## $ Foundation : Factor w/ 6 levels "BrkTil","CBlock",..: 3 2 3 1 3 6 3 2 1 1 ...
## $ BsmtQual : Factor w/ 4 levels "Ex","Fa","Gd",..: 3 3 3 4 3 3 1 3 4 4 ...
## $ BsmtCond : Factor w/ 4 levels "Fa","Gd","Po",..: 4 4 4 2 4 4 4 4 4 4 ...
## $ BsmtExposure : Factor w/ 4 levels "Av","Gd","Mn",..: 4 2 3 4 1 4 1 3 4 4 ...
## $ BsmtFinType1 : Factor w/ 6 levels "ALQ","BLQ","GLQ",..: 3 1 3 1 3 3 3 1 6 3 ...
## $ BsmtFinSF1 : int 706 978 486 216 655 732 1369 859 0 851 ...
## $ BsmtFinType2 : Factor w/ 6 levels "ALQ","BLQ","GLQ",..: 6 6 6 6 6 6 6 2 6 6 ...
## $ BsmtFinSF2 : int 0 0 0 0 0 0 0 32 0 0 ...
## $ BsmtUnfSF : int 150 284 434 540 490 64 317 216 952 140 ...
## $ TotalBsmtSF : int 856 1262 920 756 1145 796 1686 1107 952 991 ...
## $ Heating : Factor w/ 6 levels "Floor","GasA",..: 2 2 2 2 2 2 2 2 2 2 ...
## $ HeatingQC : Factor w/ 5 levels "Ex","Fa","Gd",..: 1 1 1 3 1 1 1 1 3 1 ...
## $ CentralAir : Factor w/ 2 levels "N","Y": 2 2 2 2 2 2 2 2 2 2 ...
## $ Electrical : Factor w/ 5 levels "FuseA","FuseF",..: 5 5 5 5 5 5 5 5 2 5 ...
## $ X1stFlrSF : int 856 1262 920 961 1145 796 1694 1107 1022 1077 ...
## $ X2ndFlrSF : int 854 0 866 756 1053 566 0 983 752 0 ...
## $ LowQualFinSF : int 0 0 0 0 0 0 0 0 0 0 ...
## $ GrLivArea : int 1710 1262 1786 1717 2198 1362 1694 2090 1774 1077 ...
## $ BsmtFullBath : int 1 0 1 1 1 1 1 1 0 1 ...
## $ BsmtHalfBath : int 0 1 0 0 0 0 0 0 0 0 ...
## $ FullBath : int 2 2 2 1 2 1 2 2 2 1 ...
## $ HalfBath : int 1 0 1 0 1 1 0 1 0 0 ...
## $ BedroomAbvGr : int 3 3 3 3 4 1 3 3 2 2 ...
## $ KitchenAbvGr : int 1 1 1 1 1 1 1 1 2 2 ...
## $ KitchenQual : Factor w/ 4 levels "Ex","Fa","Gd",..: 3 4 3 3 3 4 3 4 4 4 ...
## $ TotRmsAbvGrd : int 8 6 6 7 9 5 7 7 8 5 ...
## $ Functional : Factor w/ 7 levels "Maj1","Maj2",..: 7 7 7 7 7 7 7 7 3 7 ...
## $ Fireplaces : int 0 1 1 1 1 0 1 2 2 2 ...
## $ FireplaceQu : Factor w/ 5 levels "Ex","Fa","Gd",..: NA 5 5 3 5 NA 3 5 5 5 ...
## $ GarageType : Factor w/ 6 levels "2Types","Attchd",..: 2 2 2 6 2 2 2 2 6 2 ...
## $ GarageYrBlt : int 2003 1976 2001 1998 2000 1993 2004 1973 1931 1939 ...
## $ GarageFinish : Factor w/ 3 levels "Fin","RFn","Unf": 2 2 2 3 2 3 2 2 3 2 ...
## $ GarageCars : int 2 2 2 3 3 2 2 2 2 1 ...
## $ GarageArea : int 548 460 608 642 836 480 636 484 468 205 ...
## $ GarageQual : Factor w/ 5 levels "Ex","Fa","Gd",..: 5 5 5 5 5 5 5 5 2 3 ...
## $ GarageCond : Factor w/ 5 levels "Ex","Fa","Gd",..: 5 5 5 5 5 5 5 5 5 5 ...
## $ PavedDrive : Factor w/ 3 levels "N","P","Y": 3 3 3 3 3 3 3 3 3 3 ...
## $ WoodDeckSF : int 0 298 0 0 192 40 255 235 90 0 ...
## $ OpenPorchSF : int 61 0 42 35 84 30 57 204 0 4 ...
## $ EnclosedPorch: int 0 0 0 272 0 0 0 228 205 0 ...
## $ X3SsnPorch : int 0 0 0 0 0 320 0 0 0 0 ...
## $ ScreenPorch : int 0 0 0 0 0 0 0 0 0 0 ...
## $ PoolArea : int 0 0 0 0 0 0 0 0 0 0 ...
## $ PoolQC : Factor w/ 3 levels "Ex","Fa","Gd": NA NA NA NA NA NA NA NA NA NA ...
## $ Fence : Factor w/ 4 levels "GdPrv","GdWo",..: NA NA NA NA NA 3 NA NA NA NA ...
## $ MiscFeature : Factor w/ 4 levels "Gar2","Othr",..: NA NA NA NA NA 3 NA 3 NA NA ...
## $ MiscVal : int 0 0 0 0 0 700 0 350 0 0 ...
## $ MoSold : int 2 5 9 2 12 10 8 11 4 1 ...
## $ YrSold : int 2008 2007 2008 2006 2008 2009 2007 2009 2008 2008 ...
## $ SaleType : Factor w/ 9 levels "COD","Con","ConLD",..: 9 9 9 9 9 9 9 9 9 9 ...
## $ SaleCondition: Factor w/ 6 levels "Abnorml","AdjLand",..: 5 5 5 1 5 5 5 5 1 5 ...
## $ SalePrice : int 208500 181500 223500 140000 250000 143000 307000 200000 129900 118000 ...rev(sort(sapply(train, function(x) sum(is.na(x)))))## PoolQC MiscFeature Alley Fence FireplaceQu
## 1453 1406 1369 1179 690
## LotFrontage GarageCond GarageQual GarageFinish GarageYrBlt
## 259 81 81 81 81
## GarageType BsmtFinType2 BsmtExposure BsmtFinType1 BsmtCond
## 81 38 38 37 37
## BsmtQual MasVnrArea MasVnrType Electrical SalePrice
## 37 8 8 1 0
## SaleCondition SaleType YrSold MoSold MiscVal
## 0 0 0 0 0
## PoolArea ScreenPorch X3SsnPorch EnclosedPorch OpenPorchSF
## 0 0 0 0 0
## WoodDeckSF PavedDrive GarageArea GarageCars Fireplaces
## 0 0 0 0 0
## Functional TotRmsAbvGrd KitchenQual KitchenAbvGr BedroomAbvGr
## 0 0 0 0 0
## HalfBath FullBath BsmtHalfBath BsmtFullBath GrLivArea
## 0 0 0 0 0
## LowQualFinSF X2ndFlrSF X1stFlrSF CentralAir HeatingQC
## 0 0 0 0 0
## Heating TotalBsmtSF BsmtUnfSF BsmtFinSF2 BsmtFinSF1
## 0 0 0 0 0
## Foundation ExterCond ExterQual Exterior2nd Exterior1st
## 0 0 0 0 0
## RoofMatl RoofStyle YearRemodAdd YearBuilt OverallCond
## 0 0 0 0 0
## OverallQual HouseStyle BldgType Condition2 Condition1
## 0 0 0 0 0
## Neighborhood LandSlope LotConfig Utilities LandContour
## 0 0 0 0 0
## LotShape Street LotArea MSZoning MSSubClass
## 0 0 0 0 0
## Id
## 0Looking at the number of missing values and taking a closer look at the data description, we can see that NA does not always mean that the values are missing. e.g.: In case of the feature Alley, NA just means that there is “no alley access” to the house. Lets code such NA values to say “None”. There are 2 types of variables in the data set, int and Factors. For int, mark NAs as -1 and for factors, mark NAs as “None”.
# Mark NA's in factors as "None"
for (col in colnames(combi)){
if (class(combi[,col]) == "factor"){
new_col = as.character(combi[,col])
new_col[is.na(new_col)] <- "None"
combi[col] = as.factor(new_col)
}
}
# Fill remaining NA values (for numeric types) with -1
combi[is.na(combi)] = -1
# No NA's in the data now!!Feature Engineering
A couple key variables I was expecting to see are missing. Namely, total square footage and total number of bathrooms are common features used to classify homes, but these features are split up into different parts in the data set, such as above grade square footage, basement square footage and so on. Let’s add two new features for total square footage and total bathrooms:
# Drop Id
combi$Id <- NULL
# Add variable that combines above grade living area with basement sq footage
combi$total_sq_footage = combi$GrLivArea + combi$TotalBsmtSF
# Add variable that combines above ground and basement full and half baths
combi$total_baths = combi$BsmtFullBath + combi$FullBath + (0.5 * (combi$BsmtHalfBath + combi$HalfBath))Convert Factors to Nemericals.
As we will use XGBoost as the predictive modeling technique, we will convert factor variables to numeric.
for (col in colnames(combi)) {
if (class(combi[[col]])=="factor") {
#cat("VARIABLE : ",col,"\n")
levels <- levels(combi[[col]])
combi[[col]] <- as.numeric(factor(combi[[col]], levels=levels))
}
}Separate out the train and test sets
train <- combi[1:nrow(train),]
test <- combi[(nrow(train)+1):nrow(combi),]Evaluating Corelations
As there are 80 variables, let’s find variables that are highly corelated to the SalePrice
library(reshape2)## Warning: package 'reshape2' was built under R version 3.3.3cors = cor(train[ , sapply(train, is.numeric)])
cors_melted <- melt(cors)
cors_df1 <- cors_melted[cors_melted$Var1 == "SalePrice" & cors_melted$Var2 != "SalePrice" & abs(cors_melted$value)>0.5 ,]
cors_df1[order(-cors_df1$value),]## Var1 Var2 value
## 1392 SalePrice OverallQual 0.7909816
## 6640 SalePrice total_sq_footage 0.7789588
## 3770 SalePrice GrLivArea 0.7086245
## 5000 SalePrice GarageCars 0.6404092
## 6722 SalePrice total_baths 0.6317311
## 5082 SalePrice GarageArea 0.6234314
## 3114 SalePrice TotalBsmtSF 0.6135806
## 3524 SalePrice X1stFlrSF 0.6058522
## 4016 SalePrice FullBath 0.5606638
## 4426 SalePrice TotRmsAbvGrd 0.5337232
## 1556 SalePrice YearBuilt 0.5228973
## 1638 SalePrice YearRemodAdd 0.5071010
## 4344 SalePrice KitchenQual -0.5911753
## 2458 SalePrice BsmtQual -0.5937339
## 2212 SalePrice ExterQual -0.6368837Only 15 above 0.5 corelation level with OverallQuality being the highest.
Let’s look at the ones that are poorly coreleated to the SalePrice
cors_df2 <- cors_melted[cors_melted$Var1 == "SalePrice" & cors_melted$Var2 != "SalePrice" & abs(cors_melted$value)<0.2 ,]
cors_df2[order(-cors_df2$value),]## Var1 Var2 value
## 1310 SalePrice HouseStyle 0.180162623
## 982 SalePrice Neighborhood 0.170941316
## 4180 SalePrice BedroomAbvGr 0.168213154
## 5984 SalePrice Fence 0.140640344
## 1802 SalePrice RoofMatl 0.132383112
## 2294 SalePrice ExterCond 0.117302657
## 4508 SalePrice Functional 0.116326284
## 1884 SalePrice Exterior1st 0.112194540
## 5738 SalePrice ScreenPorch 0.111446571
## 1966 SalePrice Exterior2nd 0.108132602
## 5820 SalePrice PoolArea 0.092403549
## 2540 SalePrice BsmtCond 0.091503032
## 1064 SalePrice Condition1 0.091154912
## 490 SalePrice Alley 0.083121121
## 2868 SalePrice BsmtFinType2 0.072717495
## 900 SalePrice LandSlope 0.051152248
## 6230 SalePrice MoSold 0.046432245
## 5656 SalePrice X3SsnPorch 0.044583665
## 408 SalePrice Street 0.041035536
## 2048 SalePrice MasVnrType 0.018215771
## 654 SalePrice LandContour 0.015453242
## 1146 SalePrice Condition2 0.007512734
## 2950 SalePrice BsmtFinSF2 -0.011378121
## 736 SalePrice Utilities -0.014314296
## 3934 SalePrice BsmtHalfBath -0.016844154
## 6148 SalePrice MiscVal -0.021189580
## 3688 SalePrice LowQualFinSF -0.025606130
## 6312 SalePrice YrSold -0.028922585
## 6066 SalePrice MiscFeature -0.066316149
## 818 SalePrice LotConfig -0.067396023
## 1474 SalePrice OverallCond -0.077855894
## 80 SalePrice MSSubClass -0.084284135
## 1228 SalePrice BldgType -0.085590608
## 6394 SalePrice SaleType -0.088080929
## 4672 SalePrice FireplaceQu -0.097176387
## 2704 SalePrice BsmtFinType1 -0.098734368
## 3196 SalePrice Heating -0.098812076
## 5902 SalePrice PoolQC -0.126069739
## 5574 SalePrice EnclosedPorch -0.128577958
## 4262 SalePrice KitchenAbvGr -0.135907371
## 162 SalePrice MSZoning -0.140253906The year and month sold don’t appear to have much of a connection to sales prices. Interestingly, “overall condition” doesn’t have a strong correlation to sales price, while “overall quality” had the strongest correlation.
Next, let’s determine whether any of the numeric variables are highly correlated with one another. Though multicollinearity is not an issue in tree based models, we would still check. As there are so many variables, graphs will be difficult to decipher, thus, let’s find out programatically
cors_df3 <- cors_melted[cors_melted$value>0.5 & cors_melted$value<1 & cors_melted$Var1 != "SalePrice" & cors_melted$Var2 != "SalePrice" ,]
cors_df3[order(-cors_df3$value),]## Var1 Var2 value
## 4982 GarageArea GarageCars 0.8824754
## 5063 GarageCars GarageArea 0.8824754
## 3771 total_sq_footage GrLivArea 0.8803240
## 6606 GrLivArea total_sq_footage 0.8803240
## 1828 Exterior2nd Exterior1st 0.8589584
## 1909 Exterior1st Exterior2nd 0.8589584
## 3744 TotRmsAbvGrd GrLivArea 0.8254894
## 4392 GrLivArea TotRmsAbvGrd 0.8254894
## 3115 total_sq_footage TotalBsmtSF 0.8228884
## 6598 TotalBsmtSF total_sq_footage 0.8228884
## 3077 X1stFlrSF TotalBsmtSF 0.8195300
## 3482 TotalBsmtSF X1stFlrSF 0.8195300
## 3525 total_sq_footage X1stFlrSF 0.7976783
## 6603 X1stFlrSF total_sq_footage 0.7976783
## 15 BldgType MSSubClass 0.7460629
## 1149 MSSubClass BldgType 0.7460629
## 4018 total_baths FullBath 0.6941971
## 6691 FullBath total_baths 0.6941971
## 3572 GrLivArea X2ndFlrSF 0.6875011
## 3734 X2ndFlrSF GrLivArea 0.6875011
## 4427 total_sq_footage TotRmsAbvGrd 0.6788025
## 6614 TotRmsAbvGrd total_sq_footage 0.6788025
## 4154 TotRmsAbvGrd BedroomAbvGr 0.6766199
## 4397 BedroomAbvGr TotRmsAbvGrd 0.6766199
## 1393 total_sq_footage OverallQual 0.6648303
## 6577 OverallQual total_sq_footage 0.6648303
## 2753 BsmtFullBath BsmtFinSF1 0.6492118
## 3806 BsmtFinSF1 BsmtFullBath 0.6492118
## 2185 KitchenQual ExterQual 0.6444689
## 4291 ExterQual KitchenQual 0.6444689
## 1505 Foundation YearBuilt 0.6348419
## 2315 YearBuilt Foundation 0.6348419
## 3739 FullBath GrLivArea 0.6300116
## 3982 GrLivArea FullBath 0.6300116
## 5148 GarageCond GarageQual 0.6183826
## 5229 GarageQual GarageCond 0.6183826
## 3580 TotRmsAbvGrd X2ndFlrSF 0.6164226
## 4390 X2ndFlrSF TotRmsAbvGrd 0.6164226
## 3576 HalfBath X2ndFlrSF 0.6097073
## 4062 X2ndFlrSF HalfBath 0.6097073
## 1373 GarageCars OverallQual 0.6006707
## 4937 OverallQual GarageCars 0.6006707
## 6642 total_baths total_sq_footage 0.6005093
## 6723 total_sq_footage total_baths 0.6005093
## 4817 GarageCars GarageYrBlt 0.5979926
## 4979 GarageYrBlt GarageCars 0.5979926
## 3772 total_baths GrLivArea 0.5951685
## 6688 GrLivArea total_baths 0.5951685
## 1358 GrLivArea OverallQual 0.5930074
## 3707 OverallQual GrLivArea 0.5930074
## 1496 YearRemodAdd YearBuilt 0.5928550
## 1577 YearBuilt YearRemodAdd 0.5928550
## 3854 total_baths BsmtFullBath 0.5830757
## 6689 BsmtFullBath total_baths 0.5830757
## 4017 total_sq_footage FullBath 0.5744031
## 6609 FullBath total_sq_footage 0.5744031
## 2162 BsmtQual ExterQual 0.5723270
## 2405 ExterQual BsmtQual 0.5723270
## 1331 YearBuilt OverallQual 0.5723228
## 1493 OverallQual YearBuilt 0.5723228
## 3490 GrLivArea X1stFlrSF 0.5660240
## 3733 X1stFlrSF GrLivArea 0.5660240
## 1374 GarageArea OverallQual 0.5620218
## 5019 OverallQual GarageArea 0.5620218
## 4818 GarageArea GarageYrBlt 0.5607709
## 5061 GarageYrBlt GarageArea 0.5607709
## 5083 total_sq_footage GarageArea 0.5584659
## 6622 GarageArea total_sq_footage 0.5584659
## 3990 TotRmsAbvGrd FullBath 0.5547843
## 4395 FullBath TotRmsAbvGrd 0.5547843
## 1332 YearRemodAdd OverallQual 0.5506839
## 1575 OverallQual YearRemodAdd 0.5506839
## 1361 FullBath OverallQual 0.5505997
## 3953 OverallQual FullBath 0.5505997
## 4820 GarageCond GarageYrBlt 0.5430096
## 5225 GarageYrBlt GarageCond 0.5430096
## 1394 total_baths OverallQual 0.5410628
## 6659 OverallQual total_baths 0.5410628
## 1537 GarageCars YearBuilt 0.5378501
## 4939 YearBuilt GarageCars 0.5378501
## 1350 TotalBsmtSF OverallQual 0.5378085
## 3051 OverallQual TotalBsmtSF 0.5378085
## 5001 total_sq_footage GarageCars 0.5296076
## 6621 GarageCars total_sq_footage 0.5296076
## 1558 total_baths YearBuilt 0.5242983
## 6661 YearBuilt total_baths 0.5242983
## 2744 TotalBsmtSF BsmtFinSF1 0.5223961
## 3068 BsmtFinSF1 TotalBsmtSF 0.5223961
## 3741 BedroomAbvGr GrLivArea 0.5212695
## 4146 GrLivArea BedroomAbvGr 0.5212695
## 3577 BedroomAbvGr X2ndFlrSF 0.5029006
## 4144 X2ndFlrSF BedroomAbvGr 0.5029006
## 2431 KitchenQual BsmtQual 0.5001692
## 4294 BsmtQual KitchenQual 0.5001692The highest correlation is between GarageCars and GarageArea, which makes sense because we’d expect a garage that can park more cars to have more area. Highly correlated variables can cause problems with certain types of predictive models but since no variable pairs have a correlations above 0.9 and we will be using a tree-based model, let’s keep them all.
Skewness
Now let’s explore the distributions of the numeric variables with density plots. This can help us get identify outlines and whether different variable and our target variable are roughly normal, skewed or exhibit other oddities.
for (col in colnames(train)){
if(is.numeric(train[,col])){
plot(density(train[,col]), main=col)
}
}
A quick glance reveals that many of the numeric variables show right skew. Also, many variables have significant density near zero, indicating certain features are only present in subset of homes. Lastly, the target variable SalePrice appears roughly normal, but it has tail that goes off to the right, so a handful of homes sell for significantly more than the average. Making accurate predictions for these pricey homes may be the most difficult part of making a good predictive model.
Predictive Modeling
We will use XGBoost with the default tree booster, objective being regression. We will use 10 fold cross validation 20 times over with different seeds. We will then average the results for the final prediction. We can also try to tune hyperparameters using Caret or MLR libraries to see if it further improves the performance.
library(xgboost)## Warning: package 'xgboost' was built under R version 3.3.3library(Metrics)## Warning: package 'Metrics' was built under R version 3.3.3y_train <- train$SalePrice
train$SalePrice <- NULL
test$SalePrice <- NULL
x_train <- train
x_test <- test
dtrain = xgb.DMatrix(as.matrix(x_train), label=y_train)
dtest = xgb.DMatrix(as.matrix(x_test))
prediction <- as.data.frame(matrix(0,1459,20))
for(i in 1:20)
{
xgb_params = list(
seed = i,
colsample_bytree = 0.7,
subsample = 0.7,
eta = 0.03,
objective = 'reg:linear',
max_depth = 6,
num_parallel_tree = 1,
min_child_weight = 1,
base_score = 7
)
res = xgb.cv(xgb_params,
dtrain,
nrounds=500,
nfold=10,
early_stopping_rounds=20,
print_every_n = 10,
verbose= 1,
maximize=F)
best_nrounds = res$best_iteration
gbdt = xgb.train(xgb_params, dtrain, best_nrounds)
prediction[,i] <- predict(gbdt,dtest)
}## [1] train-rmse:191974.273438+747.807960 test-rmse:191857.081250+6498.886301
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144233.275000+565.950538 test-rmse:144756.260937+5328.522293
## [21] train-rmse:108886.430469+419.052516 test-rmse:109978.570313+4830.469696
## [31] train-rmse:82742.857812+365.433582 test-rmse:84748.409375+4786.009338
## [41] train-rmse:63420.429688+344.251875 test-rmse:66551.042969+4862.372881
## [51] train-rmse:49116.505469+233.130798 test-rmse:53417.498047+4982.012663
## [61] train-rmse:38533.952344+220.629556 test-rmse:44214.550000+4886.222737
## [71] train-rmse:30709.712500+182.570035 test-rmse:38075.900586+4976.887959
## [81] train-rmse:24971.192969+147.883125 test-rmse:33969.823438+4953.032476
## [91] train-rmse:20734.295508+149.135678 test-rmse:31236.688867+4998.604589
## [101] train-rmse:17646.850586+140.862337 test-rmse:29478.029492+5005.060686
## [111] train-rmse:15390.275781+142.566089 test-rmse:28305.132031+5012.927399
## [121] train-rmse:13732.147754+139.462112 test-rmse:27556.939649+5010.570892
## [131] train-rmse:12500.880371+135.595264 test-rmse:27037.867969+4996.377496
## [141] train-rmse:11547.209082+130.420033 test-rmse:26694.181836+5054.815540
## [151] train-rmse:10809.347363+134.164858 test-rmse:26441.799219+5112.196545
## [161] train-rmse:10235.172070+133.250514 test-rmse:26267.431445+5141.141969
## [171] train-rmse:9773.706055+131.128444 test-rmse:26125.214648+5120.212969
## [181] train-rmse:9392.825781+137.225725 test-rmse:25968.968945+5123.539409
## [191] train-rmse:9056.785254+141.533115 test-rmse:25842.778711+5132.873097
## [201] train-rmse:8765.980176+134.951740 test-rmse:25755.571484+5128.493997
## [211] train-rmse:8494.271484+144.891672 test-rmse:25677.250977+5136.022323
## [221] train-rmse:8256.772168+134.373739 test-rmse:25631.049414+5132.678442
## [231] train-rmse:8023.834277+122.592157 test-rmse:25568.374219+5143.369843
## [241] train-rmse:7803.133838+115.253713 test-rmse:25509.897852+5138.346881
## [251] train-rmse:7598.403467+120.423757 test-rmse:25479.225000+5164.181008
## [261] train-rmse:7403.310645+119.020911 test-rmse:25440.826367+5171.051399
## [271] train-rmse:7214.688184+120.373194 test-rmse:25427.782617+5184.491565
## [281] train-rmse:7025.374268+126.466444 test-rmse:25411.528906+5201.949162
## [291] train-rmse:6829.603418+116.711851 test-rmse:25388.654297+5209.893215
## [301] train-rmse:6651.196436+110.940772 test-rmse:25370.351953+5206.227484
## [311] train-rmse:6471.265576+118.611789 test-rmse:25356.043750+5188.176152
## [321] train-rmse:6302.311816+118.968076 test-rmse:25348.463281+5189.955095
## [331] train-rmse:6130.176172+124.128042 test-rmse:25348.061523+5191.360202
## [341] train-rmse:5962.586963+116.954183 test-rmse:25330.316406+5188.646502
## [351] train-rmse:5794.733057+107.410665 test-rmse:25314.970508+5190.594670
## [361] train-rmse:5622.258545+99.717665 test-rmse:25302.235352+5182.693566
## [371] train-rmse:5461.833447+100.596003 test-rmse:25300.891211+5184.243691
## [381] train-rmse:5309.527002+98.531535 test-rmse:25283.978711+5186.961862
## [391] train-rmse:5162.670166+102.548999 test-rmse:25279.853711+5175.098139
## [401] train-rmse:5017.694629+101.162643 test-rmse:25276.054297+5168.801889
## [411] train-rmse:4874.988281+94.854524 test-rmse:25248.914258+5164.641723
## [421] train-rmse:4740.280566+95.488683 test-rmse:25254.006641+5177.830427
## [431] train-rmse:4611.359619+94.310176 test-rmse:25257.732422+5165.726862
## Stopping. Best iteration:
## [411] train-rmse:4874.988281+94.854524 test-rmse:25248.914258+5164.641723
##
## [1] train-rmse:191965.429688+694.495877 test-rmse:191870.159375+6320.923888
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144224.290625+547.667062 test-rmse:144424.585938+5707.328795
## [21] train-rmse:108881.286719+479.212003 test-rmse:109732.914844+5409.224983
## [31] train-rmse:82734.690625+381.034249 test-rmse:84415.959375+5328.635143
## [41] train-rmse:63400.872656+299.503570 test-rmse:66113.017578+5199.362957
## [51] train-rmse:49131.392969+228.654496 test-rmse:53005.696094+5208.523869
## [61] train-rmse:38551.725391+202.776933 test-rmse:43853.888281+5288.026883
## [71] train-rmse:30773.524610+163.611185 test-rmse:37769.537305+5377.921705
## [81] train-rmse:25030.927148+159.817735 test-rmse:33646.614453+5436.951138
## [91] train-rmse:20846.109766+154.964830 test-rmse:30928.747070+5567.480993
## [101] train-rmse:17742.333203+140.515962 test-rmse:29309.726562+5766.536584
## [111] train-rmse:15454.575879+130.407621 test-rmse:28189.436328+5801.759511
## [121] train-rmse:13774.330566+132.063984 test-rmse:27493.760937+5794.538503
## [131] train-rmse:12543.762988+147.534356 test-rmse:26991.637500+5804.962498
## [141] train-rmse:11601.313477+127.523704 test-rmse:26637.536914+5841.758685
## [151] train-rmse:10875.117773+123.143554 test-rmse:26427.804883+5919.643810
## [161] train-rmse:10291.183594+134.097876 test-rmse:26250.906055+5946.405895
## [171] train-rmse:9826.258789+139.913037 test-rmse:26140.218555+5960.554955
## [181] train-rmse:9440.243262+148.216832 test-rmse:26010.259375+5994.304567
## [191] train-rmse:9102.381934+153.752277 test-rmse:25951.299414+6060.570139
## [201] train-rmse:8793.959570+167.345081 test-rmse:25847.135547+6054.008940
## [211] train-rmse:8526.994727+171.046434 test-rmse:25805.545117+6091.723593
## [221] train-rmse:8278.389111+172.003036 test-rmse:25769.105664+6099.657732
## [231] train-rmse:8042.618555+174.712470 test-rmse:25721.414062+6151.432237
## [241] train-rmse:7832.768555+185.764591 test-rmse:25671.274023+6167.579415
## [251] train-rmse:7625.900928+181.488628 test-rmse:25638.948633+6188.198953
## [261] train-rmse:7428.696484+171.602242 test-rmse:25582.815039+6208.504840
## [271] train-rmse:7241.915039+161.539002 test-rmse:25545.719922+6231.726869
## [281] train-rmse:7067.578955+168.160207 test-rmse:25521.892383+6220.782089
## [291] train-rmse:6889.626221+173.526222 test-rmse:25492.652930+6206.622226
## [301] train-rmse:6711.688623+161.034767 test-rmse:25481.543750+6203.069383
## [311] train-rmse:6533.649121+163.613317 test-rmse:25462.944727+6193.103225
## [321] train-rmse:6356.930078+162.151129 test-rmse:25427.027734+6186.807348
## [331] train-rmse:6167.933008+159.238470 test-rmse:25405.765625+6177.633900
## [341] train-rmse:5992.710449+148.306260 test-rmse:25407.725000+6180.198447
## [351] train-rmse:5814.786182+133.280187 test-rmse:25404.213867+6177.238628
## [361] train-rmse:5649.995264+128.810156 test-rmse:25392.586914+6192.559911
## [371] train-rmse:5493.462549+112.902070 test-rmse:25377.891016+6198.590464
## [381] train-rmse:5345.233203+110.370401 test-rmse:25370.197656+6196.797541
## [391] train-rmse:5189.639258+112.391058 test-rmse:25358.413281+6188.869661
## [401] train-rmse:5042.039453+110.527503 test-rmse:25351.953516+6172.879979
## [411] train-rmse:4899.016016+103.995909 test-rmse:25346.170703+6179.452235
## [421] train-rmse:4763.573486+99.255423 test-rmse:25331.267969+6174.618532
## [431] train-rmse:4629.968115+100.680724 test-rmse:25327.258398+6172.533055
## [441] train-rmse:4508.398779+100.419751 test-rmse:25327.635352+6167.000977
## [451] train-rmse:4393.194092+98.586870 test-rmse:25322.679102+6168.572746
## [461] train-rmse:4273.852368+97.713553 test-rmse:25315.226758+6161.807896
## [471] train-rmse:4156.898413+99.424556 test-rmse:25303.799219+6160.183813
## [481] train-rmse:4046.767627+99.930483 test-rmse:25309.004297+6160.155814
## [491] train-rmse:3939.019336+92.925593 test-rmse:25303.855078+6156.232994
## Stopping. Best iteration:
## [471] train-rmse:4156.898413+99.424556 test-rmse:25303.799219+6160.183813
##
## [1] train-rmse:191965.167187+846.645873 test-rmse:191848.843750+7540.365303
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144171.565625+669.206528 test-rmse:144523.306250+6742.277186
## [21] train-rmse:108806.200000+534.153444 test-rmse:109828.784375+6452.303126
## [31] train-rmse:82676.195313+364.787154 test-rmse:84540.789844+6262.122788
## [41] train-rmse:63367.610156+333.461684 test-rmse:66406.949219+6389.774178
## [51] train-rmse:49072.856641+302.688329 test-rmse:53520.363281+6447.497864
## [61] train-rmse:38521.435156+243.050778 test-rmse:44446.927734+6494.805581
## [71] train-rmse:30714.338477+179.222855 test-rmse:38116.084766+6328.897592
## [81] train-rmse:24980.132227+158.296246 test-rmse:33942.948437+6216.036002
## [91] train-rmse:20771.742578+164.716492 test-rmse:31376.621289+6182.608538
## [101] train-rmse:17651.132422+134.696556 test-rmse:29693.708984+6113.992377
## [111] train-rmse:15371.915430+141.366689 test-rmse:28676.481055+6053.706063
## [121] train-rmse:13696.208789+134.734562 test-rmse:27927.232617+5919.858811
## [131] train-rmse:12455.735156+148.064871 test-rmse:27435.789453+5821.350889
## [141] train-rmse:11519.941211+148.933953 test-rmse:27121.373828+5749.883077
## [151] train-rmse:10813.823926+167.297274 test-rmse:26920.758984+5741.213925
## [161] train-rmse:10231.425000+163.225680 test-rmse:26685.206445+5736.355327
## [171] train-rmse:9757.580078+182.414185 test-rmse:26572.199219+5713.407776
## [181] train-rmse:9371.668359+184.453250 test-rmse:26431.601562+5665.679357
## [191] train-rmse:9031.826172+192.460713 test-rmse:26374.471484+5671.093712
## [201] train-rmse:8728.716504+189.669857 test-rmse:26311.643164+5679.862424
## [211] train-rmse:8475.942383+186.325323 test-rmse:26232.097266+5669.899885
## [221] train-rmse:8231.646338+187.421379 test-rmse:26196.300586+5689.888221
## [231] train-rmse:8001.918945+184.094309 test-rmse:26151.523828+5729.135431
## [241] train-rmse:7802.530566+184.131164 test-rmse:26099.634375+5709.926175
## [251] train-rmse:7605.572119+182.945508 test-rmse:26066.804883+5691.915031
## [261] train-rmse:7413.943457+176.568050 test-rmse:26043.520313+5683.750190
## [271] train-rmse:7205.241846+160.760792 test-rmse:26015.130859+5696.146083
## [281] train-rmse:7005.123584+145.979481 test-rmse:25987.308203+5698.523098
## [291] train-rmse:6810.246533+141.032509 test-rmse:25960.786914+5712.711359
## [301] train-rmse:6627.347559+131.949662 test-rmse:25937.283984+5695.832967
## [311] train-rmse:6447.233936+134.143171 test-rmse:25924.486328+5685.277934
## [321] train-rmse:6271.116602+132.439804 test-rmse:25917.413086+5663.636103
## [331] train-rmse:6102.816553+134.708186 test-rmse:25898.579492+5664.775321
## [341] train-rmse:5937.658301+136.582171 test-rmse:25894.714062+5665.757602
## [351] train-rmse:5777.222998+136.273645 test-rmse:25871.765039+5658.387362
## [361] train-rmse:5630.241650+128.137124 test-rmse:25863.697852+5653.624327
## [371] train-rmse:5476.127930+124.013576 test-rmse:25851.007031+5650.992473
## [381] train-rmse:5321.837402+119.134615 test-rmse:25840.700195+5649.338560
## [391] train-rmse:5177.834424+125.935894 test-rmse:25837.873437+5640.701019
## [401] train-rmse:5040.937353+124.794619 test-rmse:25830.878320+5635.558356
## [411] train-rmse:4905.872022+123.366508 test-rmse:25818.153515+5631.193601
## [421] train-rmse:4771.806738+117.121975 test-rmse:25816.057617+5634.348411
## [431] train-rmse:4639.251855+109.353354 test-rmse:25816.739453+5636.876310
## [441] train-rmse:4507.619922+106.881589 test-rmse:25807.612305+5633.588467
## [451] train-rmse:4378.075342+97.859701 test-rmse:25801.129883+5626.025224
## [461] train-rmse:4255.991650+90.828022 test-rmse:25797.681836+5623.162500
## [471] train-rmse:4143.668018+88.072773 test-rmse:25794.074805+5621.236966
## [481] train-rmse:4033.433423+85.772355 test-rmse:25792.220898+5608.455771
## [491] train-rmse:3925.759741+85.059660 test-rmse:25784.863867+5604.558536
## [500] train-rmse:3828.143384+87.040573 test-rmse:25780.019336+5599.292209
## [1] train-rmse:191966.790625+833.173238 test-rmse:191804.284375+7755.347082
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144192.660938+702.064793 test-rmse:144490.455469+7025.636440
## [21] train-rmse:108896.664844+580.956535 test-rmse:109886.667969+6662.521829
## [31] train-rmse:82757.957813+472.941742 test-rmse:84620.970312+6799.095515
## [41] train-rmse:63462.105078+373.256742 test-rmse:66528.352344+7094.968735
## [51] train-rmse:49164.917969+312.828121 test-rmse:53615.806641+7255.185190
## [61] train-rmse:38602.766797+227.455371 test-rmse:44648.461719+7272.401128
## [71] train-rmse:30801.291016+217.225812 test-rmse:38583.057227+7219.473134
## [81] train-rmse:25053.360156+181.944898 test-rmse:34485.676758+7286.133005
## [91] train-rmse:20832.994336+153.557888 test-rmse:31824.175195+7278.344193
## [101] train-rmse:17737.549023+144.330887 test-rmse:30084.483789+7323.026512
## [111] train-rmse:15491.656348+160.392088 test-rmse:28946.771875+7336.479676
## [121] train-rmse:13830.800195+153.462679 test-rmse:28200.830664+7232.433852
## [131] train-rmse:12591.891113+161.601367 test-rmse:27743.690429+7277.508337
## [141] train-rmse:11639.451074+149.206745 test-rmse:27387.562695+7196.193806
## [151] train-rmse:10920.666797+130.858649 test-rmse:27136.790820+7164.636921
## [161] train-rmse:10329.360156+143.976424 test-rmse:26983.384961+7136.385990
## [171] train-rmse:9843.989551+132.254460 test-rmse:26834.892383+7132.419281
## [181] train-rmse:9458.920606+132.686713 test-rmse:26706.503125+7161.000323
## [191] train-rmse:9133.123242+119.329052 test-rmse:26615.218359+7163.014022
## [201] train-rmse:8835.462695+124.263068 test-rmse:26516.466992+7128.353269
## [211] train-rmse:8569.537890+129.823028 test-rmse:26457.441797+7155.980303
## [221] train-rmse:8335.597705+133.487464 test-rmse:26397.082422+7152.942284
## [231] train-rmse:8111.784180+124.774821 test-rmse:26350.141211+7152.517516
## [241] train-rmse:7894.436670+119.085658 test-rmse:26319.781836+7182.993804
## [251] train-rmse:7687.105127+130.160753 test-rmse:26283.716211+7157.464761
## [261] train-rmse:7483.965772+133.451174 test-rmse:26242.129687+7158.240683
## [271] train-rmse:7288.141455+140.766773 test-rmse:26210.996875+7178.481149
## [281] train-rmse:7103.246289+137.860520 test-rmse:26169.004492+7157.045911
## [291] train-rmse:6920.843604+137.055777 test-rmse:26142.358984+7140.671562
## [301] train-rmse:6746.922705+124.757879 test-rmse:26120.640820+7139.363538
## [311] train-rmse:6566.755029+127.016476 test-rmse:26101.670703+7121.749393
## [321] train-rmse:6400.376611+121.201829 test-rmse:26076.881055+7104.046734
## [331] train-rmse:6221.585010+115.143945 test-rmse:26068.146484+7097.952616
## [341] train-rmse:6062.702393+108.863687 test-rmse:26055.116602+7104.426702
## [351] train-rmse:5903.942822+114.496232 test-rmse:26041.466797+7107.787807
## [361] train-rmse:5748.238428+118.581009 test-rmse:26031.132812+7098.429854
## [371] train-rmse:5596.796387+121.698120 test-rmse:26017.775000+7093.999244
## [381] train-rmse:5445.722559+119.488223 test-rmse:26006.761719+7093.652782
## [391] train-rmse:5293.282666+114.641700 test-rmse:25996.182227+7084.219977
## [401] train-rmse:5146.454053+107.589056 test-rmse:25985.055469+7083.731362
## [411] train-rmse:5009.101123+98.717159 test-rmse:25975.081445+7079.164080
## [421] train-rmse:4866.472754+98.519495 test-rmse:25958.244336+7058.577127
## [431] train-rmse:4732.192432+98.375534 test-rmse:25950.834570+7046.832876
## [441] train-rmse:4602.434180+100.823840 test-rmse:25949.711523+7041.118589
## [451] train-rmse:4482.350537+101.280567 test-rmse:25949.216211+7033.030239
## Stopping. Best iteration:
## [437] train-rmse:4658.903662+103.155207 test-rmse:25945.866602+7041.825178
##
## [1] train-rmse:191975.940625+712.124651 test-rmse:191911.671875+6580.575818
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144239.396875+547.656910 test-rmse:144792.285938+5389.276567
## [21] train-rmse:108894.327344+420.921121 test-rmse:110028.809375+4563.798335
## [31] train-rmse:82780.320312+318.283604 test-rmse:84676.867188+4035.131022
## [41] train-rmse:63460.441797+216.405552 test-rmse:66348.061719+3770.982806
## [51] train-rmse:49124.102734+163.927033 test-rmse:53202.139453+3866.016520
## [61] train-rmse:38522.838281+151.616349 test-rmse:43962.650000+3957.502044
## [71] train-rmse:30714.729492+144.744528 test-rmse:37714.393750+4114.009760
## [81] train-rmse:24986.013086+155.017640 test-rmse:33479.500000+4266.231715
## [91] train-rmse:20766.067383+152.469321 test-rmse:30577.678711+4339.786352
## [101] train-rmse:17679.633789+155.008259 test-rmse:28798.278906+4509.488593
## [111] train-rmse:15425.153223+180.218150 test-rmse:27572.958008+4552.106544
## [121] train-rmse:13760.339356+183.652162 test-rmse:26746.086328+4597.832991
## [131] train-rmse:12524.122461+195.664988 test-rmse:26192.311328+4630.954014
## [141] train-rmse:11598.825684+203.926213 test-rmse:25823.178125+4638.862199
## [151] train-rmse:10882.844727+218.098025 test-rmse:25628.519141+4686.774031
## [161] train-rmse:10311.903027+230.330724 test-rmse:25423.338867+4794.915232
## [171] train-rmse:9850.784375+235.430067 test-rmse:25246.774609+4826.438955
## [181] train-rmse:9469.400879+238.046035 test-rmse:25126.389453+4861.383301
## [191] train-rmse:9133.892676+237.435970 test-rmse:25053.761914+4890.473511
## [201] train-rmse:8844.579395+245.262396 test-rmse:24977.225195+4954.860762
## [211] train-rmse:8589.437500+251.617841 test-rmse:24892.711523+4949.732355
## [221] train-rmse:8346.431885+248.662645 test-rmse:24816.597070+4969.948078
## [231] train-rmse:8118.636084+269.994206 test-rmse:24755.336328+4958.831307
## [241] train-rmse:7900.024463+267.562761 test-rmse:24726.044727+4938.903531
## [251] train-rmse:7700.658056+273.557286 test-rmse:24680.636328+4936.035400
## [261] train-rmse:7511.164258+269.862046 test-rmse:24641.204297+4923.822721
## [271] train-rmse:7317.642237+275.801689 test-rmse:24616.579102+4907.224295
## [281] train-rmse:7120.333203+267.186848 test-rmse:24591.733789+4910.812446
## [291] train-rmse:6929.627881+272.976915 test-rmse:24570.140039+4918.950550
## [301] train-rmse:6744.408399+284.459682 test-rmse:24535.946680+4900.474573
## [311] train-rmse:6576.258203+292.320392 test-rmse:24498.360547+4891.383386
## [321] train-rmse:6412.317432+291.138176 test-rmse:24479.932422+4879.209451
## [331] train-rmse:6226.378027+290.724304 test-rmse:24460.264062+4874.103974
## [341] train-rmse:6053.856494+275.566327 test-rmse:24441.056445+4886.154944
## [351] train-rmse:5881.377344+263.557385 test-rmse:24430.025781+4901.301297
## [361] train-rmse:5716.777295+253.939256 test-rmse:24417.214648+4902.112452
## [371] train-rmse:5553.344824+232.094220 test-rmse:24413.792773+4905.061028
## [381] train-rmse:5404.338965+226.147038 test-rmse:24407.711523+4905.988663
## [391] train-rmse:5257.484375+218.715604 test-rmse:24407.140430+4907.268985
## [401] train-rmse:5115.139209+215.021760 test-rmse:24397.654883+4902.002785
## [411] train-rmse:4965.438086+209.392993 test-rmse:24387.352149+4902.877512
## [421] train-rmse:4829.271826+202.912531 test-rmse:24378.442774+4896.346875
## [431] train-rmse:4693.798193+198.246897 test-rmse:24373.963281+4893.617033
## [441] train-rmse:4570.184912+192.863569 test-rmse:24372.643359+4898.244465
## [451] train-rmse:4443.843457+194.678539 test-rmse:24370.152930+4898.592375
## [461] train-rmse:4320.410815+187.024821 test-rmse:24364.741602+4899.793035
## [471] train-rmse:4202.463843+182.502637 test-rmse:24363.462110+4891.197429
## [481] train-rmse:4091.309839+177.338848 test-rmse:24365.317188+4895.981512
## [491] train-rmse:3981.186206+172.823675 test-rmse:24362.920313+4891.723830
## [500] train-rmse:3883.571460+169.599356 test-rmse:24353.532031+4895.862086
## [1] train-rmse:191977.226562+1290.214443 test-rmse:191757.018750+11978.466876
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144242.806250+939.087493 test-rmse:144530.421094+10368.618086
## [21] train-rmse:108900.967188+701.553616 test-rmse:109800.846875+9015.194295
## [31] train-rmse:82820.107813+555.206282 test-rmse:84504.868750+7980.380714
## [41] train-rmse:63470.935547+437.834100 test-rmse:66136.824219+7398.461626
## [51] train-rmse:49178.038672+353.334263 test-rmse:53020.787109+6977.174924
## [61] train-rmse:38596.148438+279.885214 test-rmse:43773.502734+6572.194374
## [71] train-rmse:30779.648828+226.530696 test-rmse:37472.182227+6341.307226
## [81] train-rmse:25031.942578+186.652009 test-rmse:33341.256641+6074.589529
## [91] train-rmse:20803.265625+190.688218 test-rmse:30620.023633+5933.287213
## [101] train-rmse:17703.973633+177.068901 test-rmse:28852.075586+5735.748302
## [111] train-rmse:15435.081348+159.343265 test-rmse:27739.593359+5651.126273
## [121] train-rmse:13760.884375+163.461274 test-rmse:26951.125976+5637.616094
## [131] train-rmse:12544.362988+171.943950 test-rmse:26401.006641+5589.138629
## [141] train-rmse:11604.310449+163.345290 test-rmse:26040.165039+5524.574938
## [151] train-rmse:10865.898730+161.033629 test-rmse:25763.772461+5482.115831
## [161] train-rmse:10284.125781+179.438511 test-rmse:25571.184180+5463.135564
## [171] train-rmse:9809.162988+189.664321 test-rmse:25418.661914+5447.978236
## [181] train-rmse:9417.592383+205.885586 test-rmse:25255.703906+5447.349004
## [191] train-rmse:9081.157129+199.279178 test-rmse:25182.127148+5441.867711
## [201] train-rmse:8777.854590+201.136412 test-rmse:25099.005273+5438.504916
## [211] train-rmse:8513.776172+199.749405 test-rmse:25037.229687+5443.807326
## [221] train-rmse:8273.968652+205.782187 test-rmse:24958.841016+5437.675939
## [231] train-rmse:8053.253613+215.645716 test-rmse:24875.787695+5429.206545
## [241] train-rmse:7839.912597+213.587169 test-rmse:24822.807617+5427.256666
## [251] train-rmse:7639.761328+218.969417 test-rmse:24769.058789+5404.024315
## [261] train-rmse:7453.305225+218.624309 test-rmse:24755.431055+5393.744072
## [271] train-rmse:7253.020166+215.408854 test-rmse:24741.811524+5383.450470
## [281] train-rmse:7066.703369+213.396860 test-rmse:24718.540234+5378.061841
## [291] train-rmse:6875.227197+195.851669 test-rmse:24685.174805+5373.518192
## [301] train-rmse:6690.856982+193.528390 test-rmse:24636.474609+5372.992914
## [311] train-rmse:6507.604688+184.154929 test-rmse:24611.138281+5365.852683
## [321] train-rmse:6326.912549+176.624527 test-rmse:24588.333984+5376.436432
## [331] train-rmse:6157.762939+169.204977 test-rmse:24578.864258+5371.517302
## [341] train-rmse:5983.562402+170.067764 test-rmse:24574.113281+5362.970536
## [351] train-rmse:5820.099463+166.002978 test-rmse:24567.761133+5365.129403
## [361] train-rmse:5651.266748+161.491889 test-rmse:24541.822070+5362.800625
## [371] train-rmse:5503.997363+157.809017 test-rmse:24529.355078+5364.286317
## [381] train-rmse:5350.341992+156.237170 test-rmse:24519.259570+5361.821991
## [391] train-rmse:5210.213916+156.556683 test-rmse:24515.240039+5359.131670
## [401] train-rmse:5052.790918+150.626956 test-rmse:24493.834961+5347.544809
## [411] train-rmse:4911.739600+149.356698 test-rmse:24483.302734+5341.180527
## [421] train-rmse:4774.933105+147.210190 test-rmse:24478.433203+5323.012328
## [431] train-rmse:4646.990283+138.986054 test-rmse:24472.900000+5316.441999
## [441] train-rmse:4519.649756+133.919384 test-rmse:24473.604102+5311.888978
## [451] train-rmse:4398.310693+131.367457 test-rmse:24481.154297+5314.138944
## Stopping. Best iteration:
## [432] train-rmse:4635.027637+137.880449 test-rmse:24471.584961+5314.721536
##
## [1] train-rmse:191940.543750+1192.124498 test-rmse:191645.500000+10783.194075
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144229.365625+959.774311 test-rmse:144445.314062+9725.114578
## [21] train-rmse:108911.921094+742.897885 test-rmse:109866.260156+8893.144700
## [31] train-rmse:82741.989062+589.213759 test-rmse:84390.557031+8419.264283
## [41] train-rmse:63371.192187+508.436923 test-rmse:65991.506641+8076.047468
## [51] train-rmse:49095.421484+411.694680 test-rmse:52912.903516+7748.553768
## [61] train-rmse:38544.328125+327.854492 test-rmse:43745.219141+7582.530756
## [71] train-rmse:30749.609961+270.926526 test-rmse:37531.902539+7531.339741
## [81] train-rmse:25020.413086+255.223246 test-rmse:33294.138086+7458.746692
## [91] train-rmse:20821.752344+220.002904 test-rmse:30486.990625+7432.612618
## [101] train-rmse:17747.467578+191.062819 test-rmse:28698.110938+7375.731236
## [111] train-rmse:15465.110254+158.127905 test-rmse:27567.461133+7392.520772
## [121] train-rmse:13818.189746+148.613513 test-rmse:26823.919141+7407.492910
## [131] train-rmse:12579.600488+152.601276 test-rmse:26308.089453+7380.753803
## [141] train-rmse:11647.640625+155.537008 test-rmse:25972.251953+7429.583175
## [151] train-rmse:10910.897852+180.392830 test-rmse:25706.470996+7375.645866
## [161] train-rmse:10331.555371+187.947158 test-rmse:25525.953516+7344.534569
## [171] train-rmse:9869.731250+194.178895 test-rmse:25370.250683+7392.328750
## [181] train-rmse:9481.761230+187.960776 test-rmse:25254.955762+7441.235398
## [191] train-rmse:9140.946289+190.308462 test-rmse:25175.467676+7483.440864
## [201] train-rmse:8851.586719+180.708222 test-rmse:25095.073828+7540.982640
## [211] train-rmse:8584.424219+186.692130 test-rmse:25042.874024+7573.458261
## [221] train-rmse:8336.662646+175.770704 test-rmse:24992.265625+7581.897967
## [231] train-rmse:8107.943164+185.178531 test-rmse:24939.608105+7594.338304
## [241] train-rmse:7890.471435+185.632083 test-rmse:24876.861328+7607.474030
## [251] train-rmse:7697.591357+178.766035 test-rmse:24839.936426+7630.593919
## [261] train-rmse:7494.180274+192.699155 test-rmse:24807.938086+7628.100110
## [271] train-rmse:7299.709326+188.483077 test-rmse:24775.324805+7628.657154
## [281] train-rmse:7104.602197+180.923649 test-rmse:24774.371191+7661.599394
## [291] train-rmse:6922.902490+193.457255 test-rmse:24742.747559+7670.191652
## [301] train-rmse:6733.740088+188.897010 test-rmse:24724.791699+7671.976650
## [311] train-rmse:6553.791455+193.505447 test-rmse:24703.646582+7667.773169
## [321] train-rmse:6383.160987+209.163848 test-rmse:24685.991601+7669.239923
## [331] train-rmse:6211.505371+210.770929 test-rmse:24667.068555+7663.737671
## [341] train-rmse:6039.073438+201.200127 test-rmse:24649.388086+7652.719791
## [351] train-rmse:5879.631055+194.112585 test-rmse:24630.201953+7655.408750
## [361] train-rmse:5710.790527+193.820186 test-rmse:24620.798730+7674.667863
## [371] train-rmse:5561.350683+194.819372 test-rmse:24606.004883+7687.729540
## [381] train-rmse:5409.183105+192.772751 test-rmse:24599.588672+7681.533901
## [391] train-rmse:5261.333203+183.938840 test-rmse:24591.827930+7670.993346
## [401] train-rmse:5115.902343+179.736539 test-rmse:24578.804101+7666.187263
## [411] train-rmse:4977.749072+173.343887 test-rmse:24582.247949+7677.678300
## [421] train-rmse:4849.334619+164.097081 test-rmse:24575.831836+7681.400972
## [431] train-rmse:4712.457715+156.206586 test-rmse:24575.022559+7680.527684
## [441] train-rmse:4575.504687+153.624015 test-rmse:24581.581055+7679.175930
## [451] train-rmse:4453.364014+153.225808 test-rmse:24579.036621+7674.951290
## Stopping. Best iteration:
## [433] train-rmse:4686.400488+158.228648 test-rmse:24572.875488+7683.686311
##
## [1] train-rmse:191985.862500+1048.601747 test-rmse:191823.637500+9433.320515
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144245.139063+795.146094 test-rmse:144583.890625+8678.767898
## [21] train-rmse:108884.589063+611.295304 test-rmse:109850.267187+8069.472119
## [31] train-rmse:82698.495313+527.402954 test-rmse:84590.456250+7633.403984
## [41] train-rmse:63433.369531+407.693791 test-rmse:66241.141797+7199.966821
## [51] train-rmse:49133.324610+351.931795 test-rmse:53255.536719+6846.114527
## [61] train-rmse:38585.187109+290.986353 test-rmse:44210.363281+6523.036512
## [71] train-rmse:30776.165820+250.082404 test-rmse:38102.610938+6082.440123
## [81] train-rmse:25032.772265+197.689189 test-rmse:34015.083203+5953.706625
## [91] train-rmse:20833.344531+156.900192 test-rmse:31385.790039+5763.976286
## [101] train-rmse:17754.188281+117.332457 test-rmse:29639.941992+5646.157993
## [111] train-rmse:15481.293945+123.736251 test-rmse:28586.178906+5694.445684
## [121] train-rmse:13814.895898+128.753603 test-rmse:27767.619727+5680.431292
## [131] train-rmse:12554.327930+120.369959 test-rmse:27222.007422+5639.812754
## [141] train-rmse:11599.612012+117.184196 test-rmse:26861.617578+5563.105020
## [151] train-rmse:10871.701758+126.608295 test-rmse:26601.731836+5614.835567
## [161] train-rmse:10300.147168+133.994205 test-rmse:26392.072656+5558.545175
## [171] train-rmse:9814.191602+138.288580 test-rmse:26233.146875+5528.714227
## [181] train-rmse:9421.279688+138.485382 test-rmse:26090.612109+5540.319135
## [191] train-rmse:9080.183692+142.853889 test-rmse:25997.107031+5527.698944
## [201] train-rmse:8784.309082+134.196372 test-rmse:25947.316211+5537.922581
## [211] train-rmse:8516.551660+132.573892 test-rmse:25873.423828+5552.790662
## [221] train-rmse:8284.167676+127.579402 test-rmse:25806.084766+5520.054373
## [231] train-rmse:8051.773438+129.531318 test-rmse:25758.458203+5516.882305
## [241] train-rmse:7838.788135+128.983017 test-rmse:25710.601758+5524.939614
## [251] train-rmse:7631.631787+126.007163 test-rmse:25666.364258+5503.065452
## [261] train-rmse:7422.489941+128.774850 test-rmse:25633.203516+5511.691858
## [271] train-rmse:7229.758057+134.213400 test-rmse:25609.170312+5488.574312
## [281] train-rmse:7036.665430+140.025531 test-rmse:25580.540234+5483.237374
## [291] train-rmse:6854.144922+140.205974 test-rmse:25548.261719+5468.545850
## [301] train-rmse:6672.455810+143.366295 test-rmse:25519.095899+5464.837925
## [311] train-rmse:6493.828906+150.264535 test-rmse:25489.347461+5458.736583
## [321] train-rmse:6322.184033+150.908084 test-rmse:25468.682812+5454.840748
## [331] train-rmse:6157.989111+158.698589 test-rmse:25444.541211+5441.654423
## [341] train-rmse:5987.942236+145.132869 test-rmse:25436.604492+5439.933145
## [351] train-rmse:5821.562549+149.251481 test-rmse:25427.328516+5434.489923
## [361] train-rmse:5664.174121+148.216094 test-rmse:25414.118555+5426.484368
## [371] train-rmse:5506.293701+150.181187 test-rmse:25408.007227+5416.094757
## [381] train-rmse:5350.166601+144.556890 test-rmse:25412.845508+5426.084931
## Stopping. Best iteration:
## [369] train-rmse:5539.814209+147.236013 test-rmse:25404.336914+5416.782131
##
## [1] train-rmse:191955.879688+635.816264 test-rmse:191944.260937+5600.772415
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144195.931250+509.476685 test-rmse:144639.240625+5239.309509
## [21] train-rmse:108942.105469+465.356921 test-rmse:110073.741406+4891.389604
## [31] train-rmse:82788.792969+405.801723 test-rmse:85029.234375+4940.653472
## [41] train-rmse:63458.905859+326.033997 test-rmse:66719.564844+4864.752830
## [51] train-rmse:49151.344922+282.850582 test-rmse:53605.815234+4791.003059
## [61] train-rmse:38578.539453+213.525955 test-rmse:44490.696484+4873.701364
## [71] train-rmse:30760.357031+210.426839 test-rmse:38231.812109+4732.517852
## [81] train-rmse:25016.233008+184.551487 test-rmse:33976.679492+4737.963795
## [91] train-rmse:20792.583789+168.397444 test-rmse:31207.435352+4854.616304
## [101] train-rmse:17669.943555+143.112410 test-rmse:29330.693164+4937.281669
## [111] train-rmse:15417.961328+149.748042 test-rmse:28125.690429+5050.129479
## [121] train-rmse:13737.469141+152.268552 test-rmse:27337.783008+5149.054110
## [131] train-rmse:12469.672949+134.910122 test-rmse:26805.229883+5190.386186
## [141] train-rmse:11527.662109+149.784298 test-rmse:26446.190039+5290.747581
## [151] train-rmse:10792.567676+147.250221 test-rmse:26160.040820+5321.704993
## [161] train-rmse:10216.590234+147.233603 test-rmse:25953.972851+5318.903139
## [171] train-rmse:9742.841504+154.899793 test-rmse:25829.227539+5360.310797
## [181] train-rmse:9348.840332+148.715117 test-rmse:25705.790234+5354.881112
## [191] train-rmse:9011.365234+129.840947 test-rmse:25597.870312+5378.919328
## [201] train-rmse:8697.912012+138.133660 test-rmse:25516.286914+5418.714298
## [211] train-rmse:8423.034375+130.230382 test-rmse:25445.039062+5434.255591
## [221] train-rmse:8185.882422+130.935930 test-rmse:25402.715820+5448.508051
## [231] train-rmse:7956.340967+126.640313 test-rmse:25358.342578+5464.952012
## [241] train-rmse:7746.277344+119.177817 test-rmse:25310.727539+5464.641659
## [251] train-rmse:7558.857617+122.490229 test-rmse:25250.717578+5465.288042
## [261] train-rmse:7354.008301+125.851021 test-rmse:25228.267578+5462.936914
## [271] train-rmse:7168.502881+139.221334 test-rmse:25186.729492+5449.878191
## [281] train-rmse:6986.185303+157.511976 test-rmse:25157.593555+5445.051402
## [291] train-rmse:6802.470312+165.767223 test-rmse:25139.356836+5451.190073
## [301] train-rmse:6627.030518+160.261052 test-rmse:25117.135352+5453.463790
## [311] train-rmse:6460.357666+157.875224 test-rmse:25098.458398+5455.773409
## [321] train-rmse:6283.105469+156.797321 test-rmse:25081.760547+5459.033996
## [331] train-rmse:6107.577637+161.435754 test-rmse:25073.169922+5455.996032
## [341] train-rmse:5941.881152+154.683006 test-rmse:25060.778906+5446.743391
## [351] train-rmse:5781.823926+156.751127 test-rmse:25069.240234+5446.177833
## [361] train-rmse:5622.123096+160.998593 test-rmse:25051.311719+5452.309640
## [371] train-rmse:5469.888526+149.279171 test-rmse:25046.428711+5447.591971
## [381] train-rmse:5317.793555+151.988705 test-rmse:25038.951758+5454.321581
## [391] train-rmse:5172.388574+158.813739 test-rmse:25033.187109+5461.645425
## [401] train-rmse:5033.175879+150.873897 test-rmse:25023.627148+5458.535927
## [411] train-rmse:4892.514648+142.130203 test-rmse:25017.687500+5443.388528
## [421] train-rmse:4752.738574+138.401527 test-rmse:25016.816016+5440.179845
## [431] train-rmse:4620.102246+131.644784 test-rmse:25007.538281+5436.045979
## [441] train-rmse:4492.987109+123.560968 test-rmse:25000.823437+5434.844392
## [451] train-rmse:4364.656299+112.075770 test-rmse:25002.044141+5441.273005
## [461] train-rmse:4241.379272+114.181778 test-rmse:24997.179688+5438.562253
## [471] train-rmse:4130.784741+112.003723 test-rmse:24997.903125+5434.951200
## Stopping. Best iteration:
## [458] train-rmse:4279.100366+112.874128 test-rmse:24994.771485+5439.118153
##
## [1] train-rmse:191955.140625+791.101742 test-rmse:191828.562500+7257.333606
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144222.734375+647.125255 test-rmse:144769.871875+6846.876012
## [21] train-rmse:108898.092188+559.245952 test-rmse:110084.590625+6231.822789
## [31] train-rmse:82719.246094+477.009968 test-rmse:84809.796875+5953.403501
## [41] train-rmse:63400.790625+420.133936 test-rmse:66439.865234+5634.853054
## [51] train-rmse:49091.086719+356.383395 test-rmse:53253.416016+5511.743103
## [61] train-rmse:38479.282422+318.643462 test-rmse:44093.710156+5279.792412
## [71] train-rmse:30688.630469+274.280930 test-rmse:37968.236328+5319.507668
## [81] train-rmse:24970.501758+261.826012 test-rmse:33916.266211+5409.838781
## [91] train-rmse:20751.399609+233.354835 test-rmse:31215.786523+5500.490164
## [101] train-rmse:17648.247461+210.150649 test-rmse:29482.262305+5561.368147
## [111] train-rmse:15384.964844+206.353877 test-rmse:28291.468945+5592.424676
## [121] train-rmse:13724.114258+183.851924 test-rmse:27590.483399+5711.641832
## [131] train-rmse:12511.227832+189.674652 test-rmse:27115.594336+5807.248752
## [141] train-rmse:11567.645215+178.633426 test-rmse:26804.332617+5897.547762
## [151] train-rmse:10846.218555+161.123661 test-rmse:26532.767383+5966.757054
## [161] train-rmse:10275.086719+149.113570 test-rmse:26339.796094+6027.886654
## [171] train-rmse:9797.721484+145.191686 test-rmse:26166.197852+6118.134124
## [181] train-rmse:9410.457324+137.441777 test-rmse:26044.925195+6189.116716
## [191] train-rmse:9076.354395+137.412116 test-rmse:25936.327734+6246.521891
## [201] train-rmse:8778.268750+142.138711 test-rmse:25853.808594+6268.232412
## [211] train-rmse:8505.901025+141.982738 test-rmse:25762.220508+6276.488692
## [221] train-rmse:8250.626856+134.932370 test-rmse:25726.255860+6319.635973
## [231] train-rmse:8034.745215+136.327277 test-rmse:25664.140820+6347.518971
## [241] train-rmse:7812.691699+134.035817 test-rmse:25609.879492+6367.210608
## [251] train-rmse:7609.789013+136.816172 test-rmse:25550.317969+6355.999386
## [261] train-rmse:7420.630469+138.203703 test-rmse:25512.535156+6368.254603
## [271] train-rmse:7232.048193+143.946669 test-rmse:25477.694922+6382.602575
## [281] train-rmse:7041.619385+133.491160 test-rmse:25452.554883+6381.029806
## [291] train-rmse:6861.845947+128.012497 test-rmse:25432.852344+6382.187161
## [301] train-rmse:6682.780615+119.253073 test-rmse:25405.719141+6389.568556
## [311] train-rmse:6496.338574+110.705063 test-rmse:25389.347461+6393.312863
## [321] train-rmse:6315.962549+97.177772 test-rmse:25368.358398+6391.636526
## [331] train-rmse:6139.110937+86.784894 test-rmse:25361.255664+6409.556018
## [341] train-rmse:5975.942920+92.951386 test-rmse:25357.587305+6417.661961
## [351] train-rmse:5818.753125+95.872915 test-rmse:25336.918945+6424.295516
## [361] train-rmse:5658.047803+93.194810 test-rmse:25328.298633+6433.584102
## [371] train-rmse:5505.331494+93.316151 test-rmse:25314.689453+6428.453151
## [381] train-rmse:5351.763525+90.432930 test-rmse:25306.834766+6426.942899
## [391] train-rmse:5206.088135+97.091637 test-rmse:25302.907227+6406.468491
## [401] train-rmse:5066.341260+92.756622 test-rmse:25306.327930+6407.408136
## [411] train-rmse:4928.490381+88.770508 test-rmse:25299.226953+6412.555723
## [421] train-rmse:4787.400781+84.310437 test-rmse:25290.191602+6404.384998
## [431] train-rmse:4664.007568+82.784073 test-rmse:25288.833985+6408.034703
## [441] train-rmse:4533.304297+78.781735 test-rmse:25284.387109+6413.419663
## [451] train-rmse:4409.894092+79.438861 test-rmse:25283.523828+6413.341072
## Stopping. Best iteration:
## [437] train-rmse:4586.364600+82.412309 test-rmse:25278.675586+6408.346020
##
## [1] train-rmse:191956.539062+941.807066 test-rmse:191817.214062+8678.285842
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144214.121875+768.706853 test-rmse:144491.850000+7492.640689
## [21] train-rmse:108884.143750+613.293004 test-rmse:109891.848438+6926.396483
## [31] train-rmse:82739.761719+501.216061 test-rmse:84571.357813+6448.347203
## [41] train-rmse:63405.762109+391.819013 test-rmse:66162.277734+6281.422361
## [51] train-rmse:49087.326563+333.383704 test-rmse:53159.400781+6164.335327
## [61] train-rmse:38496.851563+281.589974 test-rmse:43921.580469+6234.103559
## [71] train-rmse:30709.734375+221.416444 test-rmse:37525.500781+6205.928767
## [81] train-rmse:24960.194336+178.738321 test-rmse:33239.879297+6169.134047
## [91] train-rmse:20737.758594+183.324475 test-rmse:30374.406055+6062.709147
## [101] train-rmse:17630.162500+168.442497 test-rmse:28578.885742+6158.149562
## [111] train-rmse:15371.734863+177.513513 test-rmse:27443.724023+6126.548121
## [121] train-rmse:13679.594336+167.151829 test-rmse:26650.853906+6095.068696
## [131] train-rmse:12443.912793+143.225298 test-rmse:26117.749219+6051.338399
## [141] train-rmse:11514.506738+137.808850 test-rmse:25739.589648+6026.772251
## [151] train-rmse:10794.235937+140.518605 test-rmse:25513.195703+6031.906738
## [161] train-rmse:10232.383301+171.885399 test-rmse:25299.441797+6031.502234
## [171] train-rmse:9774.309375+179.393065 test-rmse:25160.961914+6028.747290
## [181] train-rmse:9387.486328+195.085818 test-rmse:25042.687891+6072.137016
## [191] train-rmse:9050.806152+203.915534 test-rmse:24918.449805+6003.391179
## [201] train-rmse:8739.166015+202.338250 test-rmse:24868.242383+6002.208802
## [211] train-rmse:8477.084961+210.562189 test-rmse:24788.280078+5972.779922
## [221] train-rmse:8243.525635+225.301060 test-rmse:24743.418945+5976.445942
## [231] train-rmse:8023.073047+218.542392 test-rmse:24671.951953+5958.739820
## [241] train-rmse:7808.285791+221.444101 test-rmse:24614.121094+5958.892017
## [251] train-rmse:7604.932666+211.470163 test-rmse:24583.078125+5969.489844
## [261] train-rmse:7421.134717+219.360318 test-rmse:24542.160351+5993.411243
## [271] train-rmse:7223.768164+225.514712 test-rmse:24529.114063+6006.783854
## [281] train-rmse:7029.259229+232.741530 test-rmse:24509.265039+5997.754182
## [291] train-rmse:6849.236377+240.591633 test-rmse:24482.948242+5994.769093
## [301] train-rmse:6667.631006+238.821707 test-rmse:24476.655273+6006.712189
## [311] train-rmse:6498.036182+227.144727 test-rmse:24452.931640+6014.325406
## [321] train-rmse:6330.260010+224.331638 test-rmse:24427.137305+6023.000621
## [331] train-rmse:6151.070215+221.314196 test-rmse:24408.498047+6019.050324
## [341] train-rmse:5977.159082+219.565495 test-rmse:24392.425781+6013.121299
## [351] train-rmse:5804.800586+217.688210 test-rmse:24368.303320+6019.403528
## [361] train-rmse:5639.927930+208.186608 test-rmse:24356.216406+6024.890192
## [371] train-rmse:5479.803906+208.235791 test-rmse:24352.359766+6025.293096
## [381] train-rmse:5324.578760+202.364819 test-rmse:24341.018360+6030.191455
## [391] train-rmse:5180.908594+198.817768 test-rmse:24325.768750+6025.242187
## [401] train-rmse:5037.182227+196.953360 test-rmse:24319.249805+6018.762929
## [411] train-rmse:4900.674561+183.010955 test-rmse:24314.001563+6027.383513
## [421] train-rmse:4767.738623+174.186490 test-rmse:24303.175000+6020.121034
## [431] train-rmse:4635.008789+170.167955 test-rmse:24299.201758+6024.718730
## [441] train-rmse:4503.631836+166.813296 test-rmse:24293.800976+6025.016677
## [451] train-rmse:4380.161035+161.618319 test-rmse:24298.784180+6022.561645
## [461] train-rmse:4259.982715+159.773174 test-rmse:24289.719141+6029.849849
## [471] train-rmse:4145.374170+156.046196 test-rmse:24288.382031+6026.831340
## [481] train-rmse:4031.801904+146.225005 test-rmse:24284.759570+6031.131470
## [491] train-rmse:3922.095264+134.953080 test-rmse:24283.125000+6031.166223
## [500] train-rmse:3823.483399+128.035217 test-rmse:24286.508594+6029.404539
## [1] train-rmse:191965.545313+643.699352 test-rmse:191912.167187+5860.683085
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144204.139063+529.330215 test-rmse:144471.010937+4519.331281
## [21] train-rmse:108910.164062+469.022811 test-rmse:109862.277344+3844.711134
## [31] train-rmse:82769.057812+394.751216 test-rmse:84360.029687+3640.545612
## [41] train-rmse:63452.321094+356.373558 test-rmse:66141.935937+3879.344428
## [51] train-rmse:49160.422656+305.981119 test-rmse:53202.902344+4169.794556
## [61] train-rmse:38590.401953+312.240620 test-rmse:44128.867969+4483.115524
## [71] train-rmse:30816.338867+271.369619 test-rmse:37990.407813+5024.014717
## [81] train-rmse:25041.851953+224.783710 test-rmse:33791.680078+5322.341208
## [91] train-rmse:20840.728320+208.754974 test-rmse:30978.108789+5493.312500
## [101] train-rmse:17748.877344+210.061279 test-rmse:29138.161133+5647.345712
## [111] train-rmse:15487.238672+206.616063 test-rmse:27903.259766+5814.476040
## [121] train-rmse:13829.500781+221.313686 test-rmse:27139.814844+5973.899383
## [131] train-rmse:12576.031055+216.197120 test-rmse:26615.831445+6109.981704
## [141] train-rmse:11621.808887+191.833097 test-rmse:26269.285351+6249.542351
## [151] train-rmse:10870.473730+190.529203 test-rmse:26034.609961+6377.753276
## [161] train-rmse:10287.201758+174.554657 test-rmse:25847.438672+6432.508655
## [171] train-rmse:9827.270605+169.673797 test-rmse:25709.388086+6465.607214
## [181] train-rmse:9434.591211+165.943102 test-rmse:25584.072070+6472.387485
## [191] train-rmse:9087.818164+158.326791 test-rmse:25480.460742+6495.454998
## [201] train-rmse:8774.523535+146.604703 test-rmse:25386.284766+6526.482211
## [211] train-rmse:8507.948340+131.063522 test-rmse:25329.879102+6539.406356
## [221] train-rmse:8258.771045+132.500848 test-rmse:25305.393359+6560.890541
## [231] train-rmse:8038.092871+135.249813 test-rmse:25243.383008+6584.946881
## [241] train-rmse:7831.082275+142.055574 test-rmse:25203.956445+6616.077923
## [251] train-rmse:7628.498486+142.196707 test-rmse:25158.565429+6636.327888
## [261] train-rmse:7431.260840+143.617764 test-rmse:25136.087109+6673.655856
## [271] train-rmse:7243.951465+131.331452 test-rmse:25091.127344+6692.930822
## [281] train-rmse:7056.028369+128.421799 test-rmse:25059.826953+6705.007179
## [291] train-rmse:6864.596728+118.639129 test-rmse:25041.452148+6709.577941
## [301] train-rmse:6690.819873+121.038103 test-rmse:25011.903906+6721.280658
## [311] train-rmse:6521.375098+120.672661 test-rmse:24994.110156+6721.262716
## [321] train-rmse:6343.209668+113.830172 test-rmse:24975.242383+6722.079503
## [331] train-rmse:6169.269775+102.444454 test-rmse:24957.858984+6718.201259
## [341] train-rmse:6005.378222+94.568001 test-rmse:24929.165625+6711.522810
## [351] train-rmse:5838.495020+88.689663 test-rmse:24908.403711+6710.942745
## [361] train-rmse:5678.736426+85.449941 test-rmse:24899.264649+6718.464666
## [371] train-rmse:5514.322510+93.539287 test-rmse:24890.205664+6727.725621
## [381] train-rmse:5362.565137+96.454265 test-rmse:24880.333203+6719.294879
## [391] train-rmse:5213.855664+100.144709 test-rmse:24873.420703+6721.225946
## [401] train-rmse:5072.147705+104.028776 test-rmse:24874.351367+6721.227662
## [411] train-rmse:4933.832324+97.550002 test-rmse:24863.666797+6722.252730
## [421] train-rmse:4796.965527+97.681370 test-rmse:24857.964258+6718.858439
## [431] train-rmse:4664.555322+100.172842 test-rmse:24848.499023+6706.763977
## [441] train-rmse:4544.056982+97.592354 test-rmse:24839.959961+6705.162880
## [451] train-rmse:4419.211035+94.713271 test-rmse:24841.257031+6705.987976
## [461] train-rmse:4297.799512+98.350711 test-rmse:24835.785547+6701.815347
## [471] train-rmse:4182.544678+101.597969 test-rmse:24842.112891+6711.885464
## [481] train-rmse:4069.093164+94.730882 test-rmse:24832.218164+6705.684911
## [491] train-rmse:3959.309228+100.011758 test-rmse:24830.758398+6705.248055
## [500] train-rmse:3864.581103+101.703505 test-rmse:24831.963867+6702.503920
## [1] train-rmse:191958.507812+689.419029 test-rmse:191854.743750+6144.050066
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144207.756250+545.417256 test-rmse:144633.825000+5844.843546
## [21] train-rmse:108813.303125+458.256737 test-rmse:109990.901563+5737.393640
## [31] train-rmse:82679.007812+384.821864 test-rmse:84517.627344+5944.775967
## [41] train-rmse:63379.517188+364.703356 test-rmse:66202.195703+6150.388765
## [51] train-rmse:49087.965234+359.083711 test-rmse:53208.523438+6327.347203
## [61] train-rmse:38541.740234+291.826393 test-rmse:44050.996875+6369.459458
## [71] train-rmse:30739.214844+278.027103 test-rmse:37763.429492+6329.190787
## [81] train-rmse:25012.019336+267.557982 test-rmse:33590.456250+6271.074157
## [91] train-rmse:20798.429102+242.676935 test-rmse:30830.444140+6149.651833
## [101] train-rmse:17693.362109+210.734107 test-rmse:29122.856836+6150.358754
## [111] train-rmse:15414.887402+177.522264 test-rmse:27992.110351+6040.908581
## [121] train-rmse:13735.937988+166.335529 test-rmse:27272.216602+5950.550770
## [131] train-rmse:12483.428418+150.326997 test-rmse:26807.853711+5711.881377
## [141] train-rmse:11538.039160+144.612523 test-rmse:26506.767969+5598.683458
## [151] train-rmse:10820.583008+134.007156 test-rmse:26324.753516+5561.021690
## [161] train-rmse:10260.925879+135.964439 test-rmse:26144.601172+5492.332518
## [171] train-rmse:9793.429101+136.545333 test-rmse:25997.441601+5425.515336
## [181] train-rmse:9400.060547+136.183068 test-rmse:25872.333984+5323.817368
## [191] train-rmse:9068.888086+129.880663 test-rmse:25794.923047+5287.934780
## [201] train-rmse:8749.293262+138.586190 test-rmse:25688.977344+5255.696793
## [211] train-rmse:8475.643164+135.391728 test-rmse:25642.139063+5213.444185
## [221] train-rmse:8241.221973+146.667192 test-rmse:25577.494531+5146.026713
## [231] train-rmse:8025.406055+149.670669 test-rmse:25513.549609+5083.465281
## [241] train-rmse:7818.034961+152.528069 test-rmse:25448.783594+5037.694300
## [251] train-rmse:7618.598877+157.745018 test-rmse:25414.036719+5006.938572
## [261] train-rmse:7437.794238+164.177824 test-rmse:25393.940234+5000.334681
## [271] train-rmse:7247.315332+156.353721 test-rmse:25357.840039+4958.235787
## [281] train-rmse:7057.729834+147.707589 test-rmse:25328.676367+4921.220499
## [291] train-rmse:6878.384912+136.918720 test-rmse:25292.539648+4891.763011
## [301] train-rmse:6689.474365+128.821208 test-rmse:25278.286914+4884.355770
## [311] train-rmse:6518.968799+130.489228 test-rmse:25257.440430+4870.057395
## [321] train-rmse:6351.515283+142.450757 test-rmse:25223.375781+4858.382700
## [331] train-rmse:6178.427686+145.139297 test-rmse:25204.555469+4849.183553
## [341] train-rmse:6016.167627+147.320545 test-rmse:25186.784961+4835.369173
## [351] train-rmse:5855.002295+148.311317 test-rmse:25169.791797+4828.581005
## [361] train-rmse:5695.879443+139.267512 test-rmse:25160.524805+4825.535933
## [371] train-rmse:5539.767187+133.147984 test-rmse:25145.840430+4811.884427
## [381] train-rmse:5371.531250+127.861778 test-rmse:25140.361719+4816.441911
## [391] train-rmse:5225.519824+123.497609 test-rmse:25131.014063+4819.896668
## [401] train-rmse:5082.836182+124.658918 test-rmse:25137.828906+4826.117573
## [411] train-rmse:4941.244238+119.849361 test-rmse:25134.979297+4822.405744
## Stopping. Best iteration:
## [392] train-rmse:5209.480664+122.707734 test-rmse:25129.536328+4820.139933
##
## [1] train-rmse:191957.181250+1002.152086 test-rmse:191756.589062+9123.777405
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144245.140625+819.904582 test-rmse:144610.142187+7497.660857
## [21] train-rmse:108870.815625+674.680725 test-rmse:109858.196094+6512.470679
## [31] train-rmse:82721.715625+541.589127 test-rmse:84528.496094+5801.171597
## [41] train-rmse:63425.216016+466.329231 test-rmse:66182.350000+5516.905957
## [51] train-rmse:49130.333594+387.842276 test-rmse:53205.852734+5204.156718
## [61] train-rmse:38561.494531+298.068550 test-rmse:44195.532031+5047.021710
## [71] train-rmse:30784.791992+262.913668 test-rmse:37948.695703+5036.107657
## [81] train-rmse:25032.268164+226.323789 test-rmse:33743.700391+5033.263691
## [91] train-rmse:20780.875977+188.152142 test-rmse:30975.478320+4961.249828
## [101] train-rmse:17690.366602+168.372256 test-rmse:29201.028516+4906.009613
## [111] train-rmse:15425.387012+149.103880 test-rmse:28088.277149+4924.570444
## [121] train-rmse:13751.037793+147.581595 test-rmse:27323.223047+4913.040222
## [131] train-rmse:12517.072949+133.160143 test-rmse:26898.643945+4939.579252
## [141] train-rmse:11573.169629+124.407926 test-rmse:26561.398633+4946.845433
## [151] train-rmse:10850.378418+112.254322 test-rmse:26292.030664+4921.344420
## [161] train-rmse:10268.342383+113.030335 test-rmse:26085.929883+4890.621990
## [171] train-rmse:9802.430273+111.772323 test-rmse:25973.931641+4930.535783
## [181] train-rmse:9404.697559+111.234063 test-rmse:25887.101758+4962.209014
## [191] train-rmse:9058.449512+105.614398 test-rmse:25739.722070+4926.418309
## [201] train-rmse:8754.473047+108.781574 test-rmse:25668.787305+4928.458128
## [211] train-rmse:8478.989941+97.129591 test-rmse:25632.555078+4928.341649
## [221] train-rmse:8248.426514+96.580985 test-rmse:25559.792383+4921.373918
## [231] train-rmse:8025.480420+105.399251 test-rmse:25537.743750+4927.933930
## [241] train-rmse:7807.372900+107.071574 test-rmse:25473.560937+4905.469960
## [251] train-rmse:7601.709570+112.515277 test-rmse:25457.743945+4934.841346
## [261] train-rmse:7423.775098+119.239538 test-rmse:25410.162305+4923.046667
## [271] train-rmse:7237.851660+116.286281 test-rmse:25398.224805+4920.834636
## [281] train-rmse:7060.820752+111.990851 test-rmse:25384.967773+4911.097086
## [291] train-rmse:6867.808447+105.084192 test-rmse:25359.514648+4914.320336
## [301] train-rmse:6678.430957+109.183502 test-rmse:25337.624609+4909.281796
## [311] train-rmse:6493.409766+104.496487 test-rmse:25324.251563+4924.595786
## [321] train-rmse:6310.913428+103.264398 test-rmse:25300.727539+4914.566636
## [331] train-rmse:6132.615869+113.809313 test-rmse:25285.096484+4916.306962
## [341] train-rmse:5960.304980+119.009244 test-rmse:25268.288281+4919.525482
## [351] train-rmse:5805.419531+125.705832 test-rmse:25262.333008+4918.216301
## [361] train-rmse:5637.957568+115.823839 test-rmse:25253.565820+4917.670777
## [371] train-rmse:5483.821728+120.534414 test-rmse:25248.742187+4922.354862
## [381] train-rmse:5333.543067+115.907951 test-rmse:25247.183399+4920.233033
## [391] train-rmse:5188.060108+112.168196 test-rmse:25250.467383+4913.182318
## [401] train-rmse:5044.123682+108.544696 test-rmse:25245.028516+4914.036826
## [411] train-rmse:4903.005615+106.114905 test-rmse:25244.311719+4915.720802
## Stopping. Best iteration:
## [397] train-rmse:5097.977100+107.425164 test-rmse:25242.756250+4908.404927
##
## [1] train-rmse:191949.346875+526.519135 test-rmse:191852.170313+4883.372151
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144208.979687+388.278542 test-rmse:144562.360937+4364.946314
## [21] train-rmse:108885.364844+341.817242 test-rmse:109923.951563+3938.659046
## [31] train-rmse:82743.058594+327.740270 test-rmse:84480.650000+3696.895892
## [41] train-rmse:63393.672656+272.110654 test-rmse:66178.246484+3650.001690
## [51] train-rmse:49064.800000+254.602688 test-rmse:53127.457031+3775.971948
## [61] train-rmse:38485.453906+225.881168 test-rmse:43985.178516+3879.263474
## [71] train-rmse:30679.780859+248.181135 test-rmse:37927.074414+4112.726727
## [81] train-rmse:24936.383594+242.217639 test-rmse:33765.282617+4237.801908
## [91] train-rmse:20735.572656+214.328136 test-rmse:30959.141992+4384.689894
## [101] train-rmse:17670.187695+216.128346 test-rmse:29101.912890+4431.056210
## [111] train-rmse:15415.540527+210.718815 test-rmse:27976.941797+4478.932051
## [121] train-rmse:13757.128906+221.850311 test-rmse:27192.286914+4465.349915
## [131] train-rmse:12523.580273+210.918410 test-rmse:26732.785156+4476.811878
## [141] train-rmse:11572.344727+188.541350 test-rmse:26372.786719+4510.830747
## [151] train-rmse:10835.616699+178.082862 test-rmse:26104.042773+4510.414350
## [161] train-rmse:10272.008887+184.122011 test-rmse:25937.060156+4447.290457
## [171] train-rmse:9810.126953+196.807609 test-rmse:25799.992383+4464.427068
## [181] train-rmse:9440.363867+199.326967 test-rmse:25710.698438+4483.872432
## [191] train-rmse:9117.086035+202.795264 test-rmse:25633.308984+4494.638193
## [201] train-rmse:8814.797656+202.489764 test-rmse:25585.790820+4530.983039
## [211] train-rmse:8548.179297+201.049569 test-rmse:25528.273242+4572.017267
## [221] train-rmse:8299.141113+205.554984 test-rmse:25479.388477+4595.468637
## [231] train-rmse:8074.205420+195.196754 test-rmse:25449.394727+4628.026502
## [241] train-rmse:7857.543067+190.702244 test-rmse:25412.785938+4628.049584
## [251] train-rmse:7661.306885+190.176722 test-rmse:25380.421484+4626.630519
## [261] train-rmse:7463.040723+186.709297 test-rmse:25349.669922+4615.824814
## [271] train-rmse:7273.197949+194.180073 test-rmse:25312.179688+4621.452497
## [281] train-rmse:7094.260400+190.331259 test-rmse:25284.065039+4619.539588
## [291] train-rmse:6906.742871+182.372044 test-rmse:25257.432422+4611.668955
## [301] train-rmse:6716.899707+180.785190 test-rmse:25233.891406+4598.382215
## [311] train-rmse:6520.545850+177.020467 test-rmse:25216.473047+4588.443862
## [321] train-rmse:6341.477344+165.556849 test-rmse:25189.971680+4569.235258
## [331] train-rmse:6176.177832+163.641652 test-rmse:25178.506055+4575.030905
## [341] train-rmse:6002.252881+162.053538 test-rmse:25156.374609+4573.253242
## [351] train-rmse:5841.548535+170.101543 test-rmse:25133.022656+4571.228383
## [361] train-rmse:5672.721826+160.301960 test-rmse:25121.887109+4556.920906
## [371] train-rmse:5512.190674+153.646909 test-rmse:25103.924414+4546.123097
## [381] train-rmse:5365.018262+151.153513 test-rmse:25094.528906+4554.317396
## [391] train-rmse:5214.423486+149.113147 test-rmse:25099.580664+4558.300297
## [401] train-rmse:5075.528418+145.935222 test-rmse:25102.154297+4547.423988
## Stopping. Best iteration:
## [384] train-rmse:5318.690576+147.305365 test-rmse:25093.543359+4555.397594
##
## [1] train-rmse:191976.250000+1100.014419 test-rmse:191722.760937+9915.740097
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144257.356250+918.070736 test-rmse:144674.081250+8812.510669
## [21] train-rmse:108866.770312+662.149856 test-rmse:109964.190625+8356.168765
## [31] train-rmse:82727.813281+500.365630 test-rmse:84584.389063+7879.903761
## [41] train-rmse:63372.979688+391.603793 test-rmse:66404.898828+7872.615414
## [51] train-rmse:49050.968359+311.136048 test-rmse:53315.879297+7813.722026
## [61] train-rmse:38470.578516+247.644888 test-rmse:44261.983594+7822.852087
## [71] train-rmse:30682.178906+178.259380 test-rmse:37977.960938+7604.119617
## [81] train-rmse:24939.909961+129.838420 test-rmse:33787.085547+7545.672110
## [91] train-rmse:20721.768750+127.394299 test-rmse:30965.627148+7421.852552
## [101] train-rmse:17631.599609+116.988480 test-rmse:29080.054688+7170.928697
## [111] train-rmse:15338.506445+116.874433 test-rmse:27863.894336+6973.950481
## [121] train-rmse:13659.480859+115.086553 test-rmse:27034.885352+6810.858361
## [131] train-rmse:12415.480859+131.209974 test-rmse:26489.434570+6701.649291
## [141] train-rmse:11473.912402+140.249104 test-rmse:26108.337305+6613.205644
## [151] train-rmse:10747.150586+150.647346 test-rmse:25868.888086+6521.526887
## [161] train-rmse:10173.215039+151.353105 test-rmse:25592.778906+6389.819208
## [171] train-rmse:9702.378320+148.894351 test-rmse:25412.283984+6295.468199
## [181] train-rmse:9307.107422+137.234795 test-rmse:25293.828125+6264.753221
## [191] train-rmse:8980.338086+141.873573 test-rmse:25178.591602+6197.228814
## [201] train-rmse:8695.226660+136.444990 test-rmse:25093.066406+6142.220572
## [211] train-rmse:8422.430176+127.826517 test-rmse:24982.517383+6090.739371
## [221] train-rmse:8173.436865+116.577862 test-rmse:24933.220313+6114.027022
## [231] train-rmse:7953.192480+125.741718 test-rmse:24894.605664+6083.494973
## [241] train-rmse:7737.590527+127.000118 test-rmse:24859.854102+6064.606752
## [251] train-rmse:7534.776758+131.967886 test-rmse:24825.432226+6064.892390
## [261] train-rmse:7328.435742+133.116867 test-rmse:24807.668555+6063.144814
## [271] train-rmse:7131.120801+137.096282 test-rmse:24771.384961+6047.259463
## [281] train-rmse:6947.037354+143.719471 test-rmse:24753.345703+6046.777967
## [291] train-rmse:6762.589160+157.637086 test-rmse:24711.845703+6026.378532
## [301] train-rmse:6588.060303+163.843831 test-rmse:24670.568945+6007.240440
## [311] train-rmse:6409.121045+171.040953 test-rmse:24655.091797+5990.251768
## [321] train-rmse:6225.993897+176.846083 test-rmse:24640.366602+5982.693597
## [331] train-rmse:6051.209326+170.359649 test-rmse:24616.943359+5971.863289
## [341] train-rmse:5892.840576+183.900480 test-rmse:24602.101758+5950.163630
## [351] train-rmse:5733.935693+191.596586 test-rmse:24579.325977+5955.658535
## [361] train-rmse:5581.904053+192.318107 test-rmse:24553.262109+5948.684329
## [371] train-rmse:5422.604053+179.620957 test-rmse:24545.737695+5943.127674
## [381] train-rmse:5270.516309+178.096525 test-rmse:24525.265039+5944.865173
## [391] train-rmse:5126.636475+175.213689 test-rmse:24519.317773+5947.232608
## [401] train-rmse:4989.408594+166.864678 test-rmse:24516.399023+5941.969023
## [411] train-rmse:4844.919336+159.227362 test-rmse:24514.302735+5927.571436
## [421] train-rmse:4714.238330+153.463602 test-rmse:24516.516797+5922.021537
## [431] train-rmse:4588.503271+153.189504 test-rmse:24509.932227+5917.678133
## [441] train-rmse:4468.744580+156.412606 test-rmse:24506.581250+5914.522154
## [451] train-rmse:4343.332812+148.759060 test-rmse:24496.904687+5906.218552
## [461] train-rmse:4217.273901+146.193478 test-rmse:24490.798242+5903.874604
## [471] train-rmse:4105.487500+139.751297 test-rmse:24478.222851+5895.939129
## [481] train-rmse:3998.430933+136.345800 test-rmse:24476.586328+5902.261139
## [491] train-rmse:3888.324756+131.868212 test-rmse:24481.275781+5902.807409
## Stopping. Best iteration:
## [476] train-rmse:4051.009888+138.292908 test-rmse:24474.344336+5899.528632
##
## [1] train-rmse:191945.804688+403.695226 test-rmse:191924.398438+3477.181742
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144203.832813+341.203350 test-rmse:144641.095312+2828.426353
## [21] train-rmse:108906.203125+280.728975 test-rmse:109884.727344+2475.644895
## [31] train-rmse:82840.993750+285.528298 test-rmse:84681.725000+2472.361199
## [41] train-rmse:63487.905078+208.142981 test-rmse:66463.731250+2451.970478
## [51] train-rmse:49188.276172+210.023838 test-rmse:53253.039453+2610.621970
## [61] train-rmse:38616.473047+208.621955 test-rmse:44079.329297+2969.876309
## [71] train-rmse:30825.861328+181.469573 test-rmse:37714.729297+3184.943234
## [81] train-rmse:25083.733203+166.289896 test-rmse:33449.728320+3422.570289
## [91] train-rmse:20849.361523+164.128728 test-rmse:30627.412500+3779.283907
## [101] train-rmse:17750.708594+164.167931 test-rmse:28768.802930+3961.548539
## [111] train-rmse:15490.413086+147.812884 test-rmse:27616.456836+4167.848655
## [121] train-rmse:13835.712891+138.589012 test-rmse:26842.044922+4312.915250
## [131] train-rmse:12566.863574+138.276423 test-rmse:26292.707422+4472.316926
## [141] train-rmse:11611.913672+139.729819 test-rmse:25900.600977+4583.592054
## [151] train-rmse:10887.389942+130.870365 test-rmse:25627.694141+4606.528932
## [161] train-rmse:10311.706055+119.589927 test-rmse:25435.890625+4640.502892
## [171] train-rmse:9837.383105+110.305295 test-rmse:25324.201953+4686.404096
## [181] train-rmse:9451.350293+109.170670 test-rmse:25215.049023+4678.264150
## [191] train-rmse:9105.674902+102.036386 test-rmse:25110.091992+4678.209978
## [201] train-rmse:8807.946094+111.919863 test-rmse:25059.422070+4656.950903
## [211] train-rmse:8538.349512+113.695640 test-rmse:25003.581836+4646.401901
## [221] train-rmse:8283.643848+108.857695 test-rmse:24946.561328+4634.313255
## [231] train-rmse:8045.380811+110.792346 test-rmse:24893.325781+4617.207734
## [241] train-rmse:7836.840234+97.226746 test-rmse:24846.468945+4580.994339
## [251] train-rmse:7642.195508+93.661514 test-rmse:24819.150391+4601.727527
## [261] train-rmse:7447.016992+94.870027 test-rmse:24786.721484+4591.079339
## [271] train-rmse:7260.574316+92.615218 test-rmse:24749.392773+4585.409424
## [281] train-rmse:7077.980859+101.713764 test-rmse:24705.402734+4567.596639
## [291] train-rmse:6898.648975+103.016651 test-rmse:24691.389258+4567.971250
## [301] train-rmse:6711.880176+109.865778 test-rmse:24668.678125+4566.840052
## [311] train-rmse:6538.590723+108.691631 test-rmse:24640.308203+4569.599332
## [321] train-rmse:6371.447461+104.112776 test-rmse:24635.156836+4571.389689
## [331] train-rmse:6198.406250+105.888415 test-rmse:24632.734961+4589.533839
## [341] train-rmse:6028.977783+113.625591 test-rmse:24640.727344+4577.879928
## Stopping. Best iteration:
## [326] train-rmse:6285.026953+107.345419 test-rmse:24628.358789+4573.772598
##
## [1] train-rmse:191961.273438+1052.155210 test-rmse:191729.335938+9455.765069
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144210.053125+813.341841 test-rmse:144572.096875+7966.237621
## [21] train-rmse:108911.680469+653.727932 test-rmse:109845.835156+6979.441593
## [31] train-rmse:82765.902344+561.170786 test-rmse:84452.055469+6232.502779
## [41] train-rmse:63447.458594+433.246151 test-rmse:66244.022266+5852.491204
## [51] train-rmse:49118.593750+386.163898 test-rmse:53222.483984+5588.513861
## [61] train-rmse:38586.830078+336.914222 test-rmse:44109.247656+5351.019629
## [71] train-rmse:30804.444141+265.989868 test-rmse:37808.186914+5099.164113
## [81] train-rmse:25047.455469+211.757799 test-rmse:33624.456836+5007.756452
## [91] train-rmse:20823.959570+184.516036 test-rmse:30919.170313+4909.860672
## [101] train-rmse:17723.912890+151.255287 test-rmse:29094.186914+4810.663543
## [111] train-rmse:15453.359863+144.730083 test-rmse:27938.053320+4748.078288
## [121] train-rmse:13776.700684+149.207914 test-rmse:27092.208984+4695.989932
## [131] train-rmse:12520.159570+155.109230 test-rmse:26516.510937+4652.903147
## [141] train-rmse:11573.650391+163.334293 test-rmse:26080.274219+4635.001013
## [151] train-rmse:10830.969043+151.193237 test-rmse:25822.306250+4661.189916
## [161] train-rmse:10247.274121+163.387295 test-rmse:25636.890625+4700.727953
## [171] train-rmse:9776.170996+152.964306 test-rmse:25478.790430+4678.058975
## [181] train-rmse:9390.267090+150.056432 test-rmse:25389.152734+4681.942432
## [191] train-rmse:9053.453809+157.219212 test-rmse:25299.073242+4715.361220
## [201] train-rmse:8764.059473+169.179838 test-rmse:25195.941797+4744.444947
## [211] train-rmse:8506.669824+161.335104 test-rmse:25136.928125+4770.995267
## [221] train-rmse:8256.681250+166.302398 test-rmse:25073.845898+4767.894878
## [231] train-rmse:8038.758887+164.999830 test-rmse:25007.214258+4794.423774
## [241] train-rmse:7809.956787+160.861661 test-rmse:24993.801758+4817.293958
## [251] train-rmse:7607.206055+161.943671 test-rmse:24955.176953+4805.362584
## [261] train-rmse:7413.035644+165.868996 test-rmse:24896.020508+4780.680417
## [271] train-rmse:7236.030762+163.261083 test-rmse:24866.730078+4788.669136
## [281] train-rmse:7042.718164+167.983971 test-rmse:24842.190234+4767.953932
## [291] train-rmse:6855.099121+172.121090 test-rmse:24804.318164+4769.217051
## [301] train-rmse:6662.552930+178.352602 test-rmse:24768.913086+4769.325977
## [311] train-rmse:6471.458398+170.704215 test-rmse:24758.432422+4771.723143
## [321] train-rmse:6300.601758+172.687968 test-rmse:24731.359766+4763.922233
## [331] train-rmse:6132.326123+175.512854 test-rmse:24707.476562+4753.325420
## [341] train-rmse:5961.081787+176.845229 test-rmse:24694.511133+4765.265784
## [351] train-rmse:5798.077979+175.939876 test-rmse:24683.459766+4765.317373
## [361] train-rmse:5636.025244+174.928084 test-rmse:24674.577930+4757.865977
## [371] train-rmse:5477.253223+173.615630 test-rmse:24662.170703+4745.483227
## [381] train-rmse:5321.502539+177.938403 test-rmse:24655.510938+4741.956361
## [391] train-rmse:5170.450781+177.187578 test-rmse:24639.903906+4734.519413
## [401] train-rmse:5028.217041+174.586414 test-rmse:24634.745899+4726.851561
## [411] train-rmse:4889.837207+164.953743 test-rmse:24626.163672+4722.137212
## [421] train-rmse:4757.599219+154.142996 test-rmse:24616.131640+4701.529722
## [431] train-rmse:4627.812305+152.023395 test-rmse:24608.441016+4703.966374
## [441] train-rmse:4498.184570+140.240951 test-rmse:24595.574414+4705.854858
## [451] train-rmse:4384.063184+146.118488 test-rmse:24594.994922+4696.373860
## [461] train-rmse:4262.529004+141.281418 test-rmse:24591.051172+4692.030200
## [471] train-rmse:4140.481104+137.177944 test-rmse:24573.455664+4685.334830
## [481] train-rmse:4024.309815+129.262788 test-rmse:24575.289258+4685.193287
## [491] train-rmse:3913.242529+123.890304 test-rmse:24571.542969+4677.844728
## [500] train-rmse:3820.727490+123.735184 test-rmse:24560.958008+4669.921041
## [1] train-rmse:191965.787500+960.376280 test-rmse:191835.468750+8706.412044
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144213.387500+760.998785 test-rmse:144485.145313+7531.863728
## [21] train-rmse:108891.650781+676.863352 test-rmse:109792.554688+6698.484154
## [31] train-rmse:82718.202344+540.185892 test-rmse:84622.589844+6282.466352
## [41] train-rmse:63384.485547+467.545323 test-rmse:66437.556641+5979.202415
## [51] train-rmse:49059.201172+362.893222 test-rmse:53310.280469+5655.913482
## [61] train-rmse:38528.836719+307.300124 test-rmse:44264.000781+5431.380629
## [71] train-rmse:30735.125781+259.766935 test-rmse:38082.359180+5237.094281
## [81] train-rmse:25004.719531+229.234365 test-rmse:33982.935156+5067.158269
## [91] train-rmse:20806.595898+192.760900 test-rmse:31149.195703+4912.578583
## [101] train-rmse:17710.565625+182.281596 test-rmse:29374.629102+4936.900421
## [111] train-rmse:15432.724512+149.347893 test-rmse:28145.794922+4879.599038
## [121] train-rmse:13776.285352+137.864400 test-rmse:27372.830078+4885.257970
## [131] train-rmse:12539.429492+154.078135 test-rmse:26856.352930+4841.379581
## [141] train-rmse:11598.602734+152.608708 test-rmse:26515.949805+4821.222539
## [151] train-rmse:10880.876269+159.743878 test-rmse:26237.750000+4875.475316
## [161] train-rmse:10314.573926+153.551515 test-rmse:26063.953906+4936.843923
## [171] train-rmse:9838.316797+159.238223 test-rmse:25924.323828+4987.445740
## [181] train-rmse:9442.711816+161.417992 test-rmse:25836.608984+5062.952617
## [191] train-rmse:9125.018359+155.469915 test-rmse:25733.185156+5083.884591
## [201] train-rmse:8835.002734+167.667132 test-rmse:25649.787305+5097.623007
## [211] train-rmse:8566.633496+171.919555 test-rmse:25603.933984+5111.118590
## [221] train-rmse:8321.734424+173.176255 test-rmse:25544.703516+5133.447058
## [231] train-rmse:8079.533203+170.658351 test-rmse:25476.619141+5172.296454
## [241] train-rmse:7865.749219+167.690039 test-rmse:25429.273633+5183.583305
## [251] train-rmse:7654.171338+170.277012 test-rmse:25410.605859+5200.683818
## [261] train-rmse:7451.248633+172.097600 test-rmse:25371.989063+5191.937473
## [271] train-rmse:7258.307910+175.848584 test-rmse:25353.413477+5207.771371
## [281] train-rmse:7055.640869+166.832468 test-rmse:25331.348047+5220.029197
## [291] train-rmse:6859.268897+167.223590 test-rmse:25305.108594+5219.951685
## [301] train-rmse:6679.278809+166.654192 test-rmse:25280.323047+5212.911439
## [311] train-rmse:6504.961230+159.139581 test-rmse:25263.099805+5212.903907
## [321] train-rmse:6323.995703+164.322753 test-rmse:25232.875586+5194.924699
## [331] train-rmse:6151.267627+158.042811 test-rmse:25211.699609+5188.940475
## [341] train-rmse:5982.399561+161.525939 test-rmse:25195.424414+5198.732483
## [351] train-rmse:5816.884668+161.744636 test-rmse:25188.091602+5198.747744
## [361] train-rmse:5663.251660+163.270505 test-rmse:25192.922656+5199.503740
## [371] train-rmse:5497.608984+145.449479 test-rmse:25176.236133+5196.516485
## [381] train-rmse:5347.677832+144.978031 test-rmse:25159.148828+5194.564605
## [391] train-rmse:5203.160254+142.993762 test-rmse:25152.921875+5194.496694
## [401] train-rmse:5059.015283+135.947762 test-rmse:25142.752539+5178.936670
## [411] train-rmse:4920.915186+130.357939 test-rmse:25147.593359+5174.145617
## [421] train-rmse:4782.077783+128.847937 test-rmse:25149.096875+5178.644124
## Stopping. Best iteration:
## [404] train-rmse:5020.128808+135.258132 test-rmse:25142.315820+5178.005536
##
## [1] train-rmse:191949.159375+846.343570 test-rmse:191846.635937+7594.403148
## Multiple eval metrics are present. Will use test_rmse for early stopping.
## Will train until test_rmse hasn't improved in 20 rounds.
##
## [11] train-rmse:144158.485937+675.299586 test-rmse:144535.120312+6817.118616
## [21] train-rmse:108839.126562+579.126983 test-rmse:109848.898438+6292.165947
## [31] train-rmse:82684.996875+511.886583 test-rmse:84456.279688+6167.180687
## [41] train-rmse:63367.209375+443.532425 test-rmse:66284.385547+6220.123382
## [51] train-rmse:49054.783985+410.024483 test-rmse:53368.414453+6520.412964
## [61] train-rmse:38509.901563+347.365656 test-rmse:44161.153906+6804.639617
## [71] train-rmse:30702.768750+330.181904 test-rmse:37815.383594+7020.959125
## [81] train-rmse:24976.743555+336.561669 test-rmse:33645.674219+7260.408838
## [91] train-rmse:20763.434570+328.413148 test-rmse:30824.851758+7454.682286
## [101] train-rmse:17648.553906+301.151524 test-rmse:28985.375977+7469.227322
## [111] train-rmse:15383.742871+309.105412 test-rmse:27760.614648+7470.306970
## [121] train-rmse:13710.271387+302.379380 test-rmse:26987.833008+7536.669830
## [131] train-rmse:12468.910742+308.615792 test-rmse:26533.594336+7485.017972
## [141] train-rmse:11533.768164+306.569994 test-rmse:26193.063476+7515.917402
## [151] train-rmse:10812.793653+318.766855 test-rmse:25941.499414+7552.993549
## [161] train-rmse:10251.687207+323.770339 test-rmse:25768.388477+7504.854390
## [171] train-rmse:9782.574414+316.525384 test-rmse:25675.889844+7490.873746
## [181] train-rmse:9383.298242+300.138708 test-rmse:25560.053320+7492.240617
## [191] train-rmse:9054.432520+302.047640 test-rmse:25485.986133+7499.049811
## [201] train-rmse:8758.736133+295.616167 test-rmse:25441.797461+7488.149677
## [211] train-rmse:8488.852442+295.892616 test-rmse:25369.776367+7486.103626
## [221] train-rmse:8239.341894+282.960203 test-rmse:25318.454883+7496.857614
## [231] train-rmse:8012.549414+275.746775 test-rmse:25251.862109+7475.751959
## [241] train-rmse:7801.670557+269.897810 test-rmse:25214.892773+7454.951317
## [251] train-rmse:7598.056592+270.205243 test-rmse:25174.520312+7448.013081
## [261] train-rmse:7405.295801+272.046843 test-rmse:25153.628906+7434.139645
## [271] train-rmse:7211.288721+277.990432 test-rmse:25136.846289+7437.231775
## [281] train-rmse:7021.318506+268.985316 test-rmse:25117.899218+7419.475264
## [291] train-rmse:6827.205127+266.109446 test-rmse:25091.586133+7407.675868
## [301] train-rmse:6650.316601+258.645285 test-rmse:25068.610742+7408.425455
## [311] train-rmse:6476.535840+243.284421 test-rmse:25045.535742+7416.101226
## [321] train-rmse:6289.839258+239.604661 test-rmse:25026.923047+7401.386304
## [331] train-rmse:6116.403711+242.019805 test-rmse:25018.314063+7407.849350
## [341] train-rmse:5946.716699+232.531142 test-rmse:25004.995508+7409.989255
## [351] train-rmse:5788.650439+223.066854 test-rmse:25002.019141+7404.142663
## [361] train-rmse:5628.797070+222.686710 test-rmse:24998.327539+7413.279110
## [371] train-rmse:5463.850146+204.165624 test-rmse:24990.315820+7419.713616
## [381] train-rmse:5308.735059+196.243682 test-rmse:24997.625976+7418.441832
## Stopping. Best iteration:
## [370] train-rmse:5479.986230+207.042532 test-rmse:24986.125000+7416.613744sample_submission$SalePrice <- rowMeans(prediction)
#view variable importance plot
mat <- xgb.importance (feature_names = colnames(train),model = gbdt)
xgb.plot.importance (importance_matrix = mat[1:30]) 
write.csv(sample_submission, "HousePrices.csv", row.names = F)This model gives a RMLSE(root mean log sq. error) of 0.1241. I trained a SVM and averaged the predictions. The resultant model gave me RMLSE of 0.1161 which is top 12% at the time of writing.