Peer Assessment II
1 Background
As a statistical consultant working for a real estate investment firm, your task is to develop a model to predict the selling price of a given home in Ames, Iowa. Your employer hopes to use this information to help assess whether the asking price of a house is higher or lower than the true value of the house. If the home is undervalued, it may be a good investment for the firm.
2 Training Data and relevant packages
In order to better assess the quality of the model you will produce, the data have been randomly divided into three separate pieces: a training data set, a testing data set, and a validation data set.
load("ames_train.Rdata")
load("ames_test.Rdata")
load("ames_validation.Rdata")Load Packages
library(statsr)
library(dplyr)## Warning: package 'dplyr' was built under R version 3.4.2library(BAS)## Warning: package 'BAS' was built under R version 3.4.2library(dplyr)
library(devtools)## Warning: package 'devtools' was built under R version 3.4.2library(statsr)
library(GGally)
library(statsr)
library(psych)
library(broom)
library(xtable)
library(ggplot2)
library(GGally)
library(psych)
library(dplyr)
library(gridExtra)
library(tidyr)
library(reshape)
library(data.table)## Warning: package 'data.table' was built under R version 3.4.2library(corrplot)## Warning: package 'corrplot' was built under R version 3.4.2library(labeling)
library(MASS)
library(Boruta) ## Warning: package 'Boruta' was built under R version 3.4.2## Warning: package 'ranger' was built under R version 3.4.2library(car) ## Warning: package 'car' was built under R version 3.4.2library(corrplot)
library(RColorBrewer)
library(bootstrap)
library(DAAG)## Warning: package 'DAAG' was built under R version 3.4.2library(crossval)## Warning: package 'crossval' was built under R version 3.4.2library(rpart)
library(rpart.plot)## Warning: package 'rpart.plot' was built under R version 3.4.2library(partykit)## Warning: package 'partykit' was built under R version 3.4.2library(BASS)## Warning: package 'BASS' was built under R version 3.4.2library(visreg)## Warning: package 'visreg' was built under R version 3.4.2Part 1 - Exploratory Data Analysis (EDA)
We will start with some high level EDA to visulize the make up and interactions of the features within our data sets, along with an anaylsis of NA values within the data sets. This intial assesment of NAs will lead us to a decision about how we NA values will be addressed in later stages.
Summarize Missing values
We initially start by loading our data sets based upon the provided selection criteria, along with a quick query verifying the nature of the missing values in our data set.
ames_train <- ames_train %>% filter(Sale.Condition == "Normal" & price < 500001 &
area < 4001)
vars <- names(ames_train) %in% c("price", "area", "MS.SubClass", "Lot.Area",
"Overall.Qual", "Overall.Cond", "Year.Built", "Year.Remod.Add", "Total.Bsmt.SF",
"Bsmt.Half.Bath", "Central.Air", "Bsmt.Full.Bath", "Half.Bath", "Full.Bath",
"Bedroom.AbvGr", "Kitchen.AbvGr", "TotRms.AbvGrd", "Fireplaces", "Garage.Cars",
"Garage.Area", "X1st.Flr.SF", "X2nd.Flr.SF", "Wood.Deck.SF", "Open.Porch.SF",
"Enclosed.Porch", "Screen.Porch", "Pool.Area", "Bsmt.Full.Bath")
train <- ames_train[vars]
ames_test <- ames_test %>% filter(Sale.Condition == "Normal" & price < 500001 &
area < 4001)
vars <- names(ames_test) %in% c("price", "area", "MS.SubClass", "Lot.Area",
"Overall.Qual", "Overall.Cond", "Year.Built", "Year.Remod.Add", "Total.Bsmt.SF",
"Bsmt.Half.Bath", "Central.Air", "Bsmt.Full.Bath", "Half.Bath", "Full.Bath",
"Bedroom.AbvGr", "Kitchen.AbvGr", "TotRms.AbvGrd", "Fireplaces", "Garage.Cars",
"Garage.Area", "X1st.Flr.SF", "X2nd.Flr.SF", "Wood.Deck.SF", "Open.Porch.SF",
"Enclosed.Porch", "Screen.Porch", "Pool.Area", "Bsmt.Full.Bath")
test <- ames_test[vars]
ames_validation <- ames_validation %>% filter(Sale.Condition == "Normal" & price <
500001 & area < 4001)
vars <- names(ames_validation) %in% c("price", "area", "MS.SubClass", "Lot.Area",
"Overall.Qual", "Overall.Cond", "Year.Built", "Year.Remod.Add", "Total.Bsmt.SF",
"Bsmt.Half.Bath", "Central.Air", "Bsmt.Full.Bath", "Half.Bath", "Full.Bath",
"Bedroom.AbvGr", "Kitchen.AbvGr", "TotRms.AbvGrd", "Fireplaces", "Garage.Cars",
"Garage.Area", "X1st.Flr.SF", "X2nd.Flr.SF", "Wood.Deck.SF", "Open.Porch.SF",
"Enclosed.Porch", "Screen.Porch", "Pool.Area", "Bsmt.Full.Bath")
validation <- ames_validation[vars]sum(is.na(ames_train)) / (nrow(ames_train) *ncol(ames_train))# Missing values % ## [1] 0.05813746colSums(sapply(ames_train, is.na))#Missing values by variable## PID area price MS.SubClass
## 0 0 0 0
## MS.Zoning Lot.Frontage Lot.Area Street
## 0 156 0 0
## Alley Lot.Shape Land.Contour Utilities
## 779 0 0 0
## Lot.Config Land.Slope Neighborhood Condition.1
## 0 0 0 0
## Condition.2 Bldg.Type House.Style Overall.Qual
## 0 0 0 0
## Overall.Cond Year.Built Year.Remod.Add Roof.Style
## 0 0 0 0
## Roof.Matl Exterior.1st Exterior.2nd Mas.Vnr.Type
## 0 0 0 0
## Mas.Vnr.Area Exter.Qual Exter.Cond Foundation
## 4 0 0 0
## Bsmt.Qual Bsmt.Cond Bsmt.Exposure BsmtFin.Type.1
## 20 20 20 20
## BsmtFin.SF.1 BsmtFin.Type.2 BsmtFin.SF.2 Bsmt.Unf.SF
## 0 20 0 0
## Total.Bsmt.SF Heating Heating.QC Central.Air
## 0 0 0 0
## Electrical X1st.Flr.SF X2nd.Flr.SF Low.Qual.Fin.SF
## 0 0 0 0
## Bsmt.Full.Bath Bsmt.Half.Bath Full.Bath Half.Bath
## 0 0 0 0
## Bedroom.AbvGr Kitchen.AbvGr Kitchen.Qual TotRms.AbvGrd
## 0 0 0 0
## Functional Fireplaces Fireplace.Qu Garage.Type
## 0 0 414 35
## Garage.Yr.Blt Garage.Finish Garage.Cars Garage.Area
## 36 35 0 0
## Garage.Qual Garage.Cond Paved.Drive Wood.Deck.SF
## 36 36 0 0
## Open.Porch.SF Enclosed.Porch X3Ssn.Porch Screen.Porch
## 0 0 0 0
## Pool.Area Pool.QC Fence Misc.Feature
## 0 830 653 804
## Misc.Val Mo.Sold Yr.Sold Sale.Type
## 0 0 0 0
## Sale.Condition
## 0Summary Statistics
head(describe(ames_train)) ## vars n mean sd median trimmed
## PID 1 832 714292099.48 188938380.01 535455575 711894738.22
## area 2 832 1446.62 486.67 1382 1402.46
## price 3 832 173656.22 69586.56 155000 165322.36
## MS.SubClass 4 832 57.90 43.36 50 50.14
## MS.Zoning* 5 832 6.00 0.80 6 6.07
## Lot.Frontage 6 676 68.49 24.13 69 67.51
## mad min max range skew kurtosis
## PID 12374565.38 526302030 1007100110 480798080 0.06 -1.99
## area 472.21 334 3672 3338 0.87 0.91
## price 48925.80 39300 500000 460700 1.34 2.31
## MS.SubClass 44.48 20 190 170 1.32 1.18
## MS.Zoning* 0.00 2 7 5 -2.65 9.30
## Lot.Frontage 16.31 21 313 292 2.03 16.54
## se
## PID 6550259.78
## area 16.87
## price 2412.48
## MS.SubClass 1.50
## MS.Zoning* 0.03
## Lot.Frontage 0.93tail(describe(ames_train))## vars n mean sd median trimmed mad min max
## Misc.Feature* 76 28 3.86 0.59 4 3.96 0.00 2 5
## Misc.Val 77 832 53.25 611.82 0 0.00 0.00 0 15500
## Mo.Sold 78 832 6.16 2.66 6 6.08 2.97 1 12
## Yr.Sold 79 832 2007.83 1.29 2008 2007.79 1.48 2006 2010
## Sale.Type* 80 832 9.68 1.54 10 10.00 0.00 1 10
## Sale.Condition* 81 832 5.00 0.00 5 5.00 0.00 5 5
## range skew kurtosis se
## Misc.Feature* 3 -2.06 4.63 0.11
## Misc.Val 15500 21.04 501.26 21.21
## Mo.Sold 11 0.25 -0.28 0.09
## Yr.Sold 4 0.07 -1.13 0.04
## Sale.Type* 9 -4.86 22.48 0.05
## Sale.Condition* 0 NaN NaN 0.00names(ames_train)## [1] "PID" "area" "price"
## [4] "MS.SubClass" "MS.Zoning" "Lot.Frontage"
## [7] "Lot.Area" "Street" "Alley"
## [10] "Lot.Shape" "Land.Contour" "Utilities"
## [13] "Lot.Config" "Land.Slope" "Neighborhood"
## [16] "Condition.1" "Condition.2" "Bldg.Type"
## [19] "House.Style" "Overall.Qual" "Overall.Cond"
## [22] "Year.Built" "Year.Remod.Add" "Roof.Style"
## [25] "Roof.Matl" "Exterior.1st" "Exterior.2nd"
## [28] "Mas.Vnr.Type" "Mas.Vnr.Area" "Exter.Qual"
## [31] "Exter.Cond" "Foundation" "Bsmt.Qual"
## [34] "Bsmt.Cond" "Bsmt.Exposure" "BsmtFin.Type.1"
## [37] "BsmtFin.SF.1" "BsmtFin.Type.2" "BsmtFin.SF.2"
## [40] "Bsmt.Unf.SF" "Total.Bsmt.SF" "Heating"
## [43] "Heating.QC" "Central.Air" "Electrical"
## [46] "X1st.Flr.SF" "X2nd.Flr.SF" "Low.Qual.Fin.SF"
## [49] "Bsmt.Full.Bath" "Bsmt.Half.Bath" "Full.Bath"
## [52] "Half.Bath" "Bedroom.AbvGr" "Kitchen.AbvGr"
## [55] "Kitchen.Qual" "TotRms.AbvGrd" "Functional"
## [58] "Fireplaces" "Fireplace.Qu" "Garage.Type"
## [61] "Garage.Yr.Blt" "Garage.Finish" "Garage.Cars"
## [64] "Garage.Area" "Garage.Qual" "Garage.Cond"
## [67] "Paved.Drive" "Wood.Deck.SF" "Open.Porch.SF"
## [70] "Enclosed.Porch" "X3Ssn.Porch" "Screen.Porch"
## [73] "Pool.Area" "Pool.QC" "Fence"
## [76] "Misc.Feature" "Misc.Val" "Mo.Sold"
## [79] "Yr.Sold" "Sale.Type" "Sale.Condition"Count of cases and features
dim(train) ## [1] 832 27dim(test)## [1] 813 27dim(validation) ## [1] 762 27Check NAs in trining data
CheckNA <- function( df , show = 1) {
na_count <- sapply(df, function(x) sum(is.na(x)))
data.na_count <- data.frame(na_count)
data.merged <- data.frame(cbind(c(row.names(data.na_count)), data.na_count[, 1]))
colnames(data.merged) <- c('feature', 'No_NA')
data.na <- data.merged[data.merged$No_NA != 0,]
if (show == 0) {
}
if(show == 1){
data.na
}
if(show == 2){
cScheme <- dim(data.na)[1]
ggplot(data = data.na, aes(x = data.na$feature, y = data.na$No_NA, fill = feature)) +
geom_bar(stat = "identity") +
xlab("feature") + ylab("Number of NA") +
ggtitle("Number of NA's in dataset") +
theme(axis.text.x = element_text(angle = 90, hjust = 1)) +
geom_text(aes(label = No_NA), vjust = 1.6, color = "black", position = position_dodge(0.9), size = 3) +
guides(FALSE) +
scale_fill_manual(values = colorRampPalette(c("gray","steelblue" ))(cScheme))
}
}# check which features contain NA values
CheckNA(ames_train, 2)
Visualizing Sales Price before and after transformation
sp1 <-ggplot(data = train, aes(x = (price), y = ..density..)) +
geom_histogram(fill = "green", colour="black", size = .2, bins = 30) +
ggtitle("Distribution of Price variabe w/out log transform ") + xlab("Price$") + geom_density()
sp2<- ggplot(data = train, aes(x =(log (price)), y = ..density..)) +
geom_histogram(fill = "blue", colour="black", size = .2, bins = 30) +
ggtitle("Distribution of Log Price") +
xlab("Log Price$ ") + geom_density()
grid.arrange(sp1,sp2) 
Ploting a correlation matrix for numeric variables
data(train, package="reshape")## Warning in data(train, package = "reshape"): data set 'train' not foundggpairs(data=train, # data.frame with variables
columns=1:8, # columns to plot, default to all.
title="Density Data", # title of the plot
colour = "sex"
) # aesthetics, ggplot2 style## Warning in warn_if_args_exist(list(...)): Extra arguments: "colour" are
## being ignored. If these are meant to be aesthetics, submit them using the
## 'mapping' variable within ggpairs with ggplot2::aes or ggplot2::aes_string.
This graph has been chosen, because it shows multiple information in a compact way. The main required information is to see the distribution of the given feature. It also shows the general correlation and some indication of outliners in a compact form. This plot is meant to be used to isolate some trends and potential problems as a guide for future analysis.
A key observation is that price is right skewed and not normally distributed.
sapply(train, class)## area price MS.SubClass Lot.Area Overall.Qual
## "integer" "integer" "integer" "integer" "integer"
## Overall.Cond Year.Built Year.Remod.Add Total.Bsmt.SF Central.Air
## "integer" "integer" "integer" "integer" "factor"
## X1st.Flr.SF X2nd.Flr.SF Bsmt.Full.Bath Bsmt.Half.Bath Full.Bath
## "integer" "integer" "integer" "integer" "integer"
## Half.Bath Bedroom.AbvGr Kitchen.AbvGr TotRms.AbvGrd Fireplaces
## "integer" "integer" "integer" "integer" "integer"
## Garage.Cars Garage.Area Wood.Deck.SF Open.Porch.SF Enclosed.Porch
## "integer" "integer" "integer" "integer" "integer"
## Screen.Porch Pool.Area
## "integer" "integer"ggcorr(train[,-1],
label = TRUE,
label_alpha = TRUE)## Warning in ggcorr(train[, -1], label = TRUE, label_alpha = TRUE): data in
## column(s) 'Central.Air' are not numeric and were ignored
The correlation plot indicates the 20 features with the strongest effect on housing prices.
The features Overall.Qual and area show the strongest correlation.
–
Part 2 - Development and assessment of an initial model, following a semi-guided process of analysis
Section 2.1 Initial Model
Using the the EDA and the codebook as reference: size, quality and condition should be major drivers for the sales price of a house.
Initial Model Selection
price ~ area + Overall.Qual + Overall.Cond + Kitchen.Qual + Bsmt.Qual + Exter.Qual + Year.Built + Garage.Area + Lot.Area + Fireplace.Qu
Next we show a summary of our initial model.
df.train <- ames_train %>% dplyr::select(price, area, Overall.Qual,
Overall.Cond, Kitchen.Qual,
Year.Built, Bsmt.Qual,
Exter.Qual, Garage.Area,
Lot.Area, Fireplace.Qu)
initial.model <- as.formula(price ~ area + Overall.Qual +
Overall.Cond + Year.Built +
Exter.Qual + Bsmt.Qual +
Lot.Area + Kitchen.Qual +
Fireplace.Qu + Garage.Area)
initial.lm.model <- lm(initial.model , data=(na.omit(df.train)))
summary(initial.lm.model)##
## Call:
## lm(formula = initial.model, data = (na.omit(df.train)))
##
## Residuals:
## Min 1Q Median 3Q Max
## -78264 -16766 -1300 13895 134726
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -7.296e+05 1.990e+05 -3.667 0.000279 ***
## area 5.880e+01 3.485e+00 16.875 < 2e-16 ***
## Overall.Qual 1.210e+04 1.903e+03 6.359 5.67e-10 ***
## Overall.Cond 7.982e+03 1.770e+03 4.509 8.64e-06 ***
## Year.Built 3.901e+02 9.785e+01 3.986 8.00e-05 ***
## Exter.QualFa -8.588e+04 1.881e+04 -4.565 6.71e-06 ***
## Exter.QualGd -3.340e+04 9.110e+03 -3.666 0.000280 ***
## Exter.QualTA -5.054e+04 1.055e+04 -4.791 2.36e-06 ***
## Bsmt.QualFa -1.050e+04 2.841e+04 -0.370 0.711821
## Bsmt.QualGd -3.413e+04 5.619e+03 -6.074 2.95e-09 ***
## Bsmt.QualTA -3.277e+04 7.346e+03 -4.461 1.07e-05 ***
## Lot.Area 7.892e-01 9.826e-02 8.032 1.14e-14 ***
## Kitchen.QualFa -3.022e+04 2.759e+04 -1.095 0.273976
## Kitchen.QualGd -2.531e+04 6.800e+03 -3.723 0.000226 ***
## Kitchen.QualTA -3.127e+04 7.882e+03 -3.967 8.67e-05 ***
## Fireplace.QuFa 5.237e+03 1.131e+04 0.463 0.643718
## Fireplace.QuGd 4.936e+03 9.651e+03 0.511 0.609302
## Fireplace.QuPo 1.353e+04 1.193e+04 1.133 0.257728
## Fireplace.QuTA 1.152e+03 9.719e+03 0.118 0.905740
## Garage.Area 5.782e+01 9.413e+00 6.142 2.00e-09 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 26390 on 391 degrees of freedom
## Multiple R-squared: 0.8807, Adjusted R-squared: 0.8749
## F-statistic: 151.9 on 19 and 391 DF, p-value: < 2.2e-16Summary estimates should only be trusted if the conditions for the regression are reasonable.
Evaluate conditions for Regression
There is a linear relationship between any numerical predictor variables and the response variable.
The residuals are nearly normal distributed
The residuals display constant variability
The residuals are independent
We first will examine whether the numerical variables included in the model are linearly related to the response variable (price) by examining the distribution of the residuals
checki = rstandard(initial.lm.model)
qqnorm(checki)
The residuals are scattered randomly around 0 in most cases except for the ones in the higher price segment. In general the residuals exhibit “heteroscedasticity”, meaning that the residuals get larger as the prediction moves from small to large.
Next, we check whether the residuals display a nearly normal distribution centred around 0.
par(mfrow = c(1,2))
hist(initial.lm.model$residuals, col = 'steelblue')
qqnorm(initial.lm.model$residuals, col = 'darkgreen')
qqline(initial.lm.model$residuals, col = 'red')
The results of the histogram of the residuals shows a normal distribution around 0, which is slightly left skewed. This might be driven by a outliners (see figure below). The Q-Q plot also indicates some skewness in the tails, but there are no major deviations. So we can conclude that the conditions for this model are reasonable.
par(mfrow = c(1,2))
ggplot(data = NULL, aes(x = initial.lm.model$fitted, y = initial.lm.model$residuals)) +
geom_point(col = 'darkgreen') +
geom_hline(yintercept = 0, linetype = 'dashed', color = 'red')
The results show that the residuals are equally variable for low and medium values of the predicted values, i.e., residuals have a constant variability, while in higher price segments the model tends to over estimate.
We can also perform a hierarchical analysis of variance table by using the anova function: An ANOVA table shows the possible associations between the independent and dependent variables.
anova(initial.lm.model)## Analysis of Variance Table
##
## Response: price
## Df Sum Sq Mean Sq F value Pr(>F)
## area 1 1.1534e+12 1.1534e+12 1655.9096 < 2.2e-16 ***
## Overall.Qual 1 5.6725e+11 5.6725e+11 814.3740 < 2.2e-16 ***
## Overall.Cond 1 1.8608e+04 1.8608e+04 0.0000 0.995879
## Year.Built 1 5.0742e+10 5.0742e+10 72.8485 3.132e-16 ***
## Exter.Qual 3 1.0674e+11 3.5580e+10 51.0806 < 2.2e-16 ***
## Bsmt.Qual 3 4.2067e+10 1.4022e+10 20.1314 3.749e-12 ***
## Lot.Area 1 4.8870e+10 4.8870e+10 70.1606 9.907e-16 ***
## Kitchen.Qual 3 1.1508e+10 3.8358e+09 5.5069 0.001031 **
## Fireplace.Qu 4 3.4871e+09 8.7177e+08 1.2516 0.288617
## Garage.Area 1 2.6281e+10 2.6281e+10 37.7301 2.000e-09 ***
## Residuals 391 2.7235e+11 6.9655e+08
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1With regards to inference for the model, the p-value of the model’s F-statistic indicates that the model as a whole is significant, except for Overall.Cond. We need to consider removing this feature.
| Section 2.2 Model Selection |
| We will now consider three different model selection types based on our initial model and verify if they will arrive at the same parsimonious model. |
| We will run three different model selection types: |
| P-Value (find the summary statistics at the beginning of this section 2.1) |
| Akaike information criterion |
| Boruta |
| ```r # Model selection using Akaike information criterion (AIC) |
| lm.model <- lm(initial.model, data=(na.omit(df.train))) |
| model.AIC <- stepAIC(lm.model, k = 2, trace = FALSE) model.AIC$anova ``` |
## Stepwise Model Path ## Analysis of Deviance Table ## ## Initial Model: ## price ~ area + Overall.Qual + Overall.Cond + Year.Built + Exter.Qual + ## Bsmt.Qual + Lot.Area + Kitchen.Qual + Fireplace.Qu + Garage.Area ## ## Final Model: ## price ~ area + Overall.Qual + Overall.Cond + Year.Built + Exter.Qual + ## Bsmt.Qual + Lot.Area + Kitchen.Qual + Garage.Area ## ## ## Step Df Deviance Resid. Df Resid. Dev AIC ## 1 391 272349688176 8388.133 ## 2 - Fireplace.Qu 4 2935847340 395 275285535516 8384.540 |
r # Model selection using Boruta set.seed(13) boruta.model <- Boruta(initial.model, (na.omit(df.train)), maxRuns = 100, doTrace=0) print(boruta.model) |
## Boruta performed 10 iterations in 2.212126 secs. ## 10 attributes confirmed important: area, Bsmt.Qual, Exter.Qual, ## Fireplace.Qu, Garage.Area and 5 more; ## No attributes deemed unimportant. |
| ```r Boruta.Final.model <- getConfirmedFormula(boruta.model) |
| plot(boruta.model) ``` |
 |
| ```r boruta.median <- data.frame(attStats(boruta.model)[2]) boruta.result <- cbind(rownames(boruta.median), boruta.median) colnames(boruta.result) <- c(‘features’, ‘medianImpact’) |
| boruta.result <- boruta.result %>% arrange(desc(medianImpact)) |
| # add the decision of boruta selection, if a feature is significant to the model |
| boruta.result$decision <- attStats(boruta.model)[‘decision’] boruta.result[1:10,] ``` |
## features medianImpact decision ## 1 area 33.710125 Confirmed ## 2 Garage.Area 22.295413 Confirmed ## 3 Overall.Qual 20.904388 Confirmed ## 4 Lot.Area 19.194994 Confirmed ## 5 Year.Built 15.888855 Confirmed ## 6 Kitchen.Qual 15.732111 Confirmed ## 7 Exter.Qual 15.489750 Confirmed ## 8 Bsmt.Qual 15.097198 Confirmed ## 9 Overall.Cond 7.338089 Confirmed ## 10 Fireplace.Qu 5.102601 Confirmed |
| What we see from the analysis is, that all 3 model selection methods end up with the same result. The initial model provides p-values, which indicate significance for all chosen features. |
| Section 2.3 Initial Model Residuals |
| One way to assess the performance of a model is to examine the model’s residuals. Below, you’ll find a residual plot as well as the applied regression line, which indicates the linearity of the data. |
r ggplot(data = NULL, aes(x = initial.lm.model$fitted, y = initial.lm.model$residuals)) + geom_point(col = 'darkgreen') + geom_smooth(method = "lm", formula = y ~ splines::bs(x, 3), se = TRUE) + geom_hline(yintercept = 0, linetype = 'dashed', color = 'red') |
 |
| The plot shows that the residuals are equally variable for medium values of the predicted values, i.e., residuals have a constant variability, while in lower and higher price segments the model tends to over estimate. The regression line shows the general trend of the distribution. It also exhibits “heteroscedasticity”, meaning that the residuals get larger as the prediction moves from small to large. |
| There are ways to resolve this. The most successful solution to overcome heteroscedasticity is to transform a variable, or heteroscedasticity often indicates that an important variable is missing. We will use the outcome of this curve later on, when we decide if we need to apply any type of interaction or transformation. |
Section 2.3 Initial Model Residuals
Because the aim is to reduce the prediction error so for a good prediction the residual should be minimized.
Comparison has to carried out on lm model(model.full,model.AIC) Model residuals analysis is the most important measure to find the goodness of fit.The residuals should not be correlated with another variable.Adjacent residuals should not be autocorrelated.They should have consistency.These values cause outliers. There have to be very little outliers.The residuals should have a normal distribution. Negative values indicate leakage information. Missing values and inadequate subset could be possibly casusing curvature.
If the distribution becomes better with bayesian method or not. The RMSE for full model and AIC is same. Normality and Homoscedicity is also plotted.
Durbin Watson test is to be performed to test the residual correlations.
crPlots(lm.model)
plot(lm.model)## Warning: not plotting observations with leverage one:
## 101, 325
## Warning: not plotting observations with leverage one:
## 101, 325
plot(model.AIC)## Warning: not plotting observations with leverage one:
## 101, 325
## Warning: not plotting observations with leverage one:
## 101, 325
Normality test of residuals
plot(resid(model.AIC),main="AIC Histogram of Normality",xlab='standard residuals',ylab='Frequency')
Colinearity(multiple relationship between variables) ( Speaks to section 3.3)
We now will investigate if we need to apply any interaction variables. For that purpose we will investigate on how much multicollinearity (correlation between predictors) exists in our regression model via Variance inflation factors (VIF). Multicollinearity is problematic because it can increase the variance of the regression coefficients, making them unstable and difficult to interpret. Variance inflation factors (VIF) measure how much the variance of the estimated regression coefficients are inflated as compared to when the predictor variables are not linearly related.
Variance inflation factor is calculated via the following function: \[VIF_{k} = \frac{1}{1 - R^2}\]
Use the following guidelines to interpret the VIF: VIF Status of predictors VIF = 1 Not correlated 1 < VIF < 5 Moderately correlated VIF > 5 to 10 Highly correlated
vif(model.AIC)## area Overall.Qual Overall.Cond Year.Built Exter.QualFa
## 1.6075 3.2400 1.4814 3.3711 1.4852
## Exter.QualGd Exter.QualTA Bsmt.QualFa Bsmt.QualGd Bsmt.QualTA
## 11.5200 15.8250 1.1537 4.5713 7.3879
## Lot.Area Kitchen.QualFa Kitchen.QualGd Kitchen.QualTA Garage.Area
## 1.0624 1.0865 6.4363 8.6747 1.8244The figures above indicates that all of our features are moderately correlated. Therefore, we decide not to include any variable interactions.
Homoscedicity test of residuals
plot(model.AIC)## Warning: not plotting observations with leverage one:
## 101, 325
## Warning: not plotting observations with leverage one:
## 101, 325
Autocorrelation of residuals
durbinWatsonTest(model.AIC)## lag Autocorrelation D-W Statistic p-value
## 1 -0.02851302 2.055466 0.592
## Alternative hypothesis: rho != 0Section 2.4 Initial Model RMSE
Initial.model<-lm(log(price)~log(area)+Lot.Area+Overall.Qual+Overall.Cond +Year.Built+Total.Bsmt.SF+ Bsmt.Full.Bath + Fireplaces+Kitchen.AbvGr+
Garage.Area,data=(na.omit(train)))
predict.Initial <- exp(predict(Initial.model, train))
resid.Initial <- train$price - predict.Initial
rmse.Initial<- sqrt(mean(resid.Initial^2))
rmse.Initial## [1] 19788.97Section 2.5 Overfitting
No model is without short comings. We have to identify those and adapt the model in order to minimize them.This way we can improve the model. This process is repeated a number and critical values are measured and compared till satisfication. Find out if the model is:
Doing well on training set.
Poor performance with out-of-sample.
Compare performance on both in-sample and out-of-sample data sets.
To do this overfitting dataset ames_test has to be compared with ames_train. Here we are going to compare test and train models for overfitting. Use your model from above to generate predictions for the housing prices in the test data set. Accuracy of predicted prices against actual prices. The difference in coverage prbabilities are also compared. The RMSE error is also giving indication for the over/under estimation.
predict.Initial <- exp(predict(Initial.model,
newdata=test,interval = "prediction"))
coverage.prob.Initial <- mean(test$price > predict.Initial[,"lwr"] &
test$price < predict.Initial[,"upr"])
coverage.prob.Initial## [1] 0.9507995cp_test=coverage.prob.InitialSummary of inital model section:
For predicting home prices I used stepwise method for both training and test datasets
Predicting prices(Train dataset)-Bayesian Model-AIC
model.bic <- stepAIC(Initial.model,k =log(nrow(train)))## Start: AIC=-3566.29
## log(price) ~ log(area) + Lot.Area + Overall.Qual + Overall.Cond +
## Year.Built + Total.Bsmt.SF + Bsmt.Full.Bath + Fireplaces +
## Kitchen.AbvGr + Garage.Area
##
## Df Sum of Sq RSS AIC
## <none> 10.470 -3566.3
## - Kitchen.AbvGr 1 0.3570 10.827 -3545.1
## - Fireplaces 1 0.3945 10.865 -3542.2
## - Lot.Area 1 0.5170 10.987 -3532.9
## - Bsmt.Full.Bath 1 0.8929 11.363 -3504.9
## - Garage.Area 1 0.9144 11.385 -3503.4
## - Total.Bsmt.SF 1 2.0160 12.486 -3426.5
## - Overall.Cond 1 2.7843 13.255 -3376.8
## - Overall.Qual 1 3.2720 13.742 -3346.8
## - Year.Built 1 3.7057 14.176 -3320.9
## - log(area) 1 7.2731 17.743 -3134.2pred_price <-predict(model.bic)
P = predict(model.bic)
pred.test= data.frame(predicted = P )
mean_price=mean(P)
print(mean_price)## [1] 11.9928actual_price=mean(log(ames_train$price))
print(actual_price)## [1] 11.9928Predicting prices (Test dataset)-Bayesian Model-BIC
model.init <- stepAIC(Initial.model, k =log(nrow(test)))## Start: AIC=-3566.54
## log(price) ~ log(area) + Lot.Area + Overall.Qual + Overall.Cond +
## Year.Built + Total.Bsmt.SF + Bsmt.Full.Bath + Fireplaces +
## Kitchen.AbvGr + Garage.Area
##
## Df Sum of Sq RSS AIC
## <none> 10.470 -3566.5
## - Kitchen.AbvGr 1 0.3570 10.827 -3545.3
## - Fireplaces 1 0.3945 10.865 -3542.5
## - Lot.Area 1 0.5170 10.987 -3533.1
## - Bsmt.Full.Bath 1 0.8929 11.363 -3505.2
## - Garage.Area 1 0.9144 11.385 -3503.6
## - Total.Bsmt.SF 1 2.0160 12.486 -3426.7
## - Overall.Cond 1 2.7843 13.255 -3377.1
## - Overall.Qual 1 3.2720 13.742 -3347.0
## - Year.Built 1 3.7057 14.176 -3321.1
## - log(area) 1 7.2731 17.743 -3134.4pred_price <-predict(model.init)
P = predict(model.init)
pred.test= data.frame(predicted = P )
mean_price=mean(P)
print(mean_price)## [1] 11.9928actual_price=mean(log(test$price))
actual_price## [1] 12.02297Part 3 Development of a Final Model
Having experimented with various regression methods I can conclude that the best regression model can be achieved with stepwise regsubsetting and K-fold methods. Bayesian “BAS” is also a very suitable method with rich and easy diagnostics. The selection of variable can be achieved by comparing the information criteria,Cp,adjusted R-squared,RMSE,AIC/BIC.Confirmation of maximum variables till no further performance and prediction improvement are made sure. Boruta and regsubsetting provides these verifications.
Section 3.1 Final Model: bayesian method and stepwise.
Bayesian Method for Regression
Final.model <- bas.lm(log(price) ~log(Total.Bsmt.SF + 1) + log(area)+ Bedroom.AbvGr + Year.Built + log(X1st.Flr.SF) +
log(Lot.Area) + Overall.Cond+Overall.Qual+TotRms.AbvGrd+
Bsmt.Full.Bath+Garage.Cars+Garage.Area+Year.Remod.Add+
Kitchen.AbvGr+Fireplaces+MS.SubClass+Half.Bath+Full.Bath,
data =train, prior = "BIC",modelprior=uniform())
Final.model##
## Call:
## bas.lm(formula = log(price) ~ log(Total.Bsmt.SF + 1) + log(area) + Bedroom.AbvGr + Year.Built + log(X1st.Flr.SF) + log(Lot.Area) + Overall.Cond + Overall.Qual + TotRms.AbvGrd + Bsmt.Full.Bath + Garage.Cars + Garage.Area + Year.Remod.Add + Kitchen.AbvGr + Fireplaces + MS.SubClass + Half.Bath + Full.Bath, data = train, prior = "BIC", modelprior = uniform())
##
##
## Marginal Posterior Inclusion Probabilities:
## Intercept log(Total.Bsmt.SF + 1) log(area)
## 1.00000 0.99999 1.00000
## Bedroom.AbvGr Year.Built log(X1st.Flr.SF)
## 0.62405 1.00000 1.00000
## log(Lot.Area) Overall.Cond Overall.Qual
## 1.00000 1.00000 1.00000
## TotRms.AbvGrd Bsmt.Full.Bath Garage.Cars
## 0.08543 1.00000 0.13888
## Garage.Area Year.Remod.Add Kitchen.AbvGr
## 0.93368 0.98571 0.98284
## Fireplaces MS.SubClass Half.Bath
## 0.99999 0.03807 0.03411
## Full.Bath
## 0.05115summary(Final.model)## P(B != 0 | Y) model 1 model 2 model 3
## Intercept 1.00000000 1.0000 1.0000000 1.0000000
## log(Total.Bsmt.SF + 1) 0.99999458 1.0000 1.0000000 1.0000000
## log(area) 1.00000000 1.0000 1.0000000 1.0000000
## Bedroom.AbvGr 0.62405134 1.0000 0.0000000 1.0000000
## Year.Built 1.00000000 1.0000 1.0000000 1.0000000
## log(X1st.Flr.SF) 0.99999912 1.0000 1.0000000 1.0000000
## log(Lot.Area) 1.00000000 1.0000 1.0000000 1.0000000
## Overall.Cond 1.00000000 1.0000 1.0000000 1.0000000
## Overall.Qual 1.00000000 1.0000 1.0000000 1.0000000
## TotRms.AbvGrd 0.08542856 0.0000 0.0000000 1.0000000
## Bsmt.Full.Bath 1.00000000 1.0000 1.0000000 1.0000000
## Garage.Cars 0.13888327 0.0000 0.0000000 0.0000000
## Garage.Area 0.93368434 1.0000 1.0000000 1.0000000
## Year.Remod.Add 0.98571228 1.0000 1.0000000 1.0000000
## Kitchen.AbvGr 0.98283900 1.0000 1.0000000 1.0000000
## Fireplaces 0.99998878 1.0000 1.0000000 1.0000000
## MS.SubClass 0.03806871 0.0000 0.0000000 0.0000000
## Half.Bath 0.03411016 0.0000 0.0000000 0.0000000
## Full.Bath 0.05114819 0.0000 0.0000000 0.0000000
## BF NA 1.0000 0.6802189 0.1328129
## PostProbs NA 0.4001 0.2721000 0.0531000
## R2 NA 0.9177 0.9169000 0.9179000
## dim NA 14.0000 13.0000000 15.0000000
## logmarg NA -993.8561 -994.2414310 -995.8749041
## model 4 model 5
## Intercept 1.00000000 1.00000000
## log(Total.Bsmt.SF + 1) 1.00000000 1.00000000
## log(area) 1.00000000 1.00000000
## Bedroom.AbvGr 1.00000000 1.00000000
## Year.Built 1.00000000 1.00000000
## log(X1st.Flr.SF) 1.00000000 1.00000000
## log(Lot.Area) 1.00000000 1.00000000
## Overall.Cond 1.00000000 1.00000000
## Overall.Qual 1.00000000 1.00000000
## TotRms.AbvGrd 0.00000000 0.00000000
## Bsmt.Full.Bath 1.00000000 1.00000000
## Garage.Cars 1.00000000 1.00000000
## Garage.Area 1.00000000 0.00000000
## Year.Remod.Add 1.00000000 1.00000000
## Kitchen.AbvGr 1.00000000 1.00000000
## Fireplaces 1.00000000 1.00000000
## MS.SubClass 0.00000000 0.00000000
## Half.Bath 0.00000000 0.00000000
## Full.Bath 0.00000000 0.00000000
## BF 0.08125174 0.07646485
## PostProbs 0.03250000 0.03060000
## R2 0.91780000 0.91720000
## dim 15.00000000 14.00000000
## logmarg -996.36629343 -996.42701444image(Final.model, rotate = F)
coef(Final.model)##
## Marginal Posterior Summaries of Coefficients:
##
## Using BMA
##
## Based on the top 262144 models
## post mean post SD post p(B != 0)
## Intercept 1.199e+01 3.790e-03 1.000e+00
## log(Total.Bsmt.SF + 1) 2.140e-02 3.852e-03 1.000e+00
## log(area) 3.777e-01 2.880e-02 1.000e+00
## Bedroom.AbvGr -1.173e-02 1.061e-02 6.241e-01
## Year.Built 3.223e-03 2.224e-04 1.000e+00
## log(X1st.Flr.SF) 1.112e-01 1.808e-02 1.000e+00
## log(Lot.Area) 9.660e-02 9.301e-03 1.000e+00
## Overall.Cond 4.848e-02 4.266e-03 1.000e+00
## Overall.Qual 8.192e-02 4.895e-03 1.000e+00
## TotRms.AbvGrd 6.042e-04 2.530e-03 8.543e-02
## Bsmt.Full.Bath 7.077e-02 8.203e-03 1.000e+00
## Garage.Cars 4.044e-03 1.186e-02 1.389e-01
## Garage.Area 1.421e-04 4.717e-05 9.337e-01
## Year.Remod.Add 1.049e-03 3.004e-04 9.857e-01
## Kitchen.AbvGr -8.481e-02 2.506e-02 9.828e-01
## Fireplaces 3.981e-02 7.289e-03 1.000e+00
## MS.SubClass -1.866e-06 2.724e-05 3.807e-02
## Half.Bath 5.780e-05 1.982e-03 3.411e-02
## Full.Bath 5.263e-04 3.433e-03 5.115e-02Final Model by using Bayesian Method
AIC.bas <- bas.lm(log(price) ~log(Garage.Area + 1) +
log(Total.Bsmt.SF + 1) + Garage.Cars + log(area) +
Full.Bath + Half.Bath +
Bedroom.AbvGr + Year.Built + log(X1st.Flr.SF) +
log(X2nd.Flr.SF + 1) +log(Lot.Area) + Central.Air + Overall.Cond+Overall.Qual, data =na.omit(train),
prior = "AIC",modelprior=uniform())
AIC.bas##
## Call:
## bas.lm(formula = log(price) ~ log(Garage.Area + 1) + log(Total.Bsmt.SF + 1) + Garage.Cars + log(area) + Full.Bath + Half.Bath + Bedroom.AbvGr + Year.Built + log(X1st.Flr.SF) + log(X2nd.Flr.SF + 1) + log(Lot.Area) + Central.Air + Overall.Cond + Overall.Qual, data = na.omit(train), prior = "AIC", modelprior = uniform())
##
##
## Marginal Posterior Inclusion Probabilities:
## Intercept log(Garage.Area + 1) log(Total.Bsmt.SF + 1)
## 1.0000 0.4978 1.0000
## Garage.Cars log(area) Full.Bath
## 1.0000 1.0000 0.2923
## Half.Bath Bedroom.AbvGr Year.Built
## 0.2899 1.0000 1.0000
## log(X1st.Flr.SF) log(X2nd.Flr.SF + 1) log(Lot.Area)
## 0.8803 0.4115 1.0000
## Central.AirY Overall.Cond Overall.Qual
## 0.9942 1.0000 1.0000summary(AIC.bas)## P(B != 0 | Y) model 1 model 2
## Intercept 1.0000000 1.0000 1.0000000
## log(Garage.Area + 1) 0.4978436 0.0000 1.0000000
## log(Total.Bsmt.SF + 1) 1.0000000 1.0000 1.0000000
## Garage.Cars 0.9999945 1.0000 1.0000000
## log(area) 1.0000000 1.0000 1.0000000
## Full.Bath 0.2923140 0.0000 0.0000000
## Half.Bath 0.2899258 0.0000 0.0000000
## Bedroom.AbvGr 0.9999998 1.0000 1.0000000
## Year.Built 1.0000000 1.0000 1.0000000
## log(X1st.Flr.SF) 0.8803172 1.0000 1.0000000
## log(X2nd.Flr.SF + 1) 0.4115000 0.0000 0.0000000
## log(Lot.Area) 1.0000000 1.0000 1.0000000
## Central.AirY 0.9942209 1.0000 1.0000000
## Overall.Cond 1.0000000 1.0000 1.0000000
## Overall.Qual 1.0000000 1.0000 1.0000000
## BF NA 1.0000 0.9909467
## PostProbs NA 0.1467 0.1454000
## R2 NA 0.9047 0.9049000
## dim NA 11.0000 12.0000000
## logmarg NA -1018.7154 -1018.7244550
## model 3 model 4 model 5
## Intercept 1.0000000 1.000000 1.0000000
## log(Garage.Area + 1) 0.0000000 1.000000 1.0000000
## log(Total.Bsmt.SF + 1) 1.0000000 1.000000 1.0000000
## Garage.Cars 1.0000000 1.000000 1.0000000
## log(area) 1.0000000 1.000000 1.0000000
## Full.Bath 0.0000000 0.000000 1.0000000
## Half.Bath 0.0000000 0.000000 0.0000000
## Bedroom.AbvGr 1.0000000 1.000000 1.0000000
## Year.Built 1.0000000 1.000000 1.0000000
## log(X1st.Flr.SF) 1.0000000 1.000000 1.0000000
## log(X2nd.Flr.SF + 1) 1.0000000 1.000000 0.0000000
## log(Lot.Area) 1.0000000 1.000000 1.0000000
## Central.AirY 1.0000000 1.000000 1.0000000
## Overall.Cond 1.0000000 1.000000 1.0000000
## Overall.Qual 1.0000000 1.000000 1.0000000
## BF 0.5008923 0.482127 0.4373706
## PostProbs 0.0735000 0.070700 0.0642000
## R2 0.9047000 0.905000 0.9049000
## dim 12.0000000 13.000000 13.0000000
## logmarg -1019.4067246 -1019.444908 -1019.5423349image(AIC.bas, rotate = F)
coef(AIC.bas)##
## Marginal Posterior Summaries of Coefficients:
##
## Using BMA
##
## Based on the top 16384 models
## post mean post SD post p(B != 0)
## Intercept 11.9927955 0.0040637 1.0000000
## log(Garage.Area + 1) -0.0031993 0.0045690 0.4978436
## log(Total.Bsmt.SF + 1) 0.0251912 0.0040171 1.0000000
## Garage.Cars 0.0444416 0.0095886 0.9999945
## log(area) 0.4574010 0.0570836 1.0000000
## Full.Bath -0.0015900 0.0069031 0.2923140
## Half.Bath 0.0014271 0.0066043 0.2899258
## Bedroom.AbvGr -0.0397790 0.0068988 0.9999998
## Year.Built 0.0036647 0.0002217 1.0000000
## log(X1st.Flr.SF) 0.1059955 0.0521764 0.8803172
## log(X2nd.Flr.SF + 1) -0.0025436 0.0047593 0.4115000
## log(Lot.Area) 0.1171650 0.0094822 1.0000000
## Central.AirY 0.0746779 0.0223200 0.9942209
## Overall.Cond 0.0529705 0.0043663 1.0000000
## Overall.Qual 0.0864533 0.0050286 1.0000000Final model using multiple linear regression(lm)method.
Final.lm <- lm(log(price) ~log(Total.Bsmt.SF + 1) + log(area)+ Bedroom.AbvGr + Year.Built + log(X1st.Flr.SF) +
log(Lot.Area) + Overall.Cond+Overall.Qual,
data =train)
predict.Final <- exp(predict(Final.lm, train))
resid.Final <- train$price - predict.Final
rmse.Final <- sqrt(mean(resid.Final^2))
rmse.Final## [1] 22160.99Cross validation of Final model.(Stepwise methods)
step <- stepAIC(Final.lm, direction="both")## Start: AIC=-3520.27
## log(price) ~ log(Total.Bsmt.SF + 1) + log(area) + Bedroom.AbvGr +
## Year.Built + log(X1st.Flr.SF) + log(Lot.Area) + Overall.Cond +
## Overall.Qual
##
## Df Sum of Sq RSS AIC
## <none> 11.835 -3520.3
## - Bedroom.AbvGr 1 0.4934 12.329 -3488.3
## - log(Total.Bsmt.SF + 1) 1 0.7207 12.556 -3473.1
## - log(X1st.Flr.SF) 1 0.8307 12.666 -3465.8
## - log(Lot.Area) 1 2.5645 14.400 -3359.1
## - Overall.Cond 1 2.6967 14.532 -3351.5
## - Overall.Qual 1 4.5543 16.390 -3251.4
## - log(area) 1 5.6083 17.444 -3199.6
## - Year.Built 1 7.1167 18.952 -3130.6summary(Final.lm)##
## Call:
## lm(formula = log(price) ~ log(Total.Bsmt.SF + 1) + log(area) +
## Bedroom.AbvGr + Year.Built + log(X1st.Flr.SF) + log(Lot.Area) +
## Overall.Cond + Overall.Qual, data = train)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.80336 -0.06908 0.00160 0.07343 0.39921
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -2.7512763 0.4139264 -6.647 5.45e-11 ***
## log(Total.Bsmt.SF + 1) 0.0285325 0.0040306 7.079 3.11e-12 ***
## log(area) 0.4518322 0.0228800 19.748 < 2e-16 ***
## Bedroom.AbvGr -0.0410143 0.0070021 -5.857 6.79e-09 ***
## Year.Built 0.0042744 0.0001921 22.246 < 2e-16 ***
## log(X1st.Flr.SF) 0.1399698 0.0184166 7.600 8.05e-14 ***
## log(Lot.Area) 0.1260728 0.0094410 13.354 < 2e-16 ***
## Overall.Cond 0.0572853 0.0041833 13.694 < 2e-16 ***
## Overall.Qual 0.0902493 0.0050714 17.796 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.1199 on 823 degrees of freedom
## Multiple R-squared: 0.8999, Adjusted R-squared: 0.8989
## F-statistic: 925.1 on 8 and 823 DF, p-value: < 2.2e-16plot(resid(Final.lm))
plot(Final.lm)
visreg(Final.lm)
anova(Final.lm)## Analysis of Variance Table
##
## Response: log(price)
## Df Sum Sq Mean Sq F value Pr(>F)
## log(Total.Bsmt.SF + 1) 1 18.390 18.390 1278.79 < 2.2e-16 ***
## log(area) 1 55.259 55.259 3842.49 < 2.2e-16 ***
## Bedroom.AbvGr 1 4.494 4.494 312.49 < 2.2e-16 ***
## Year.Built 1 14.586 14.586 1014.23 < 2.2e-16 ***
## log(X1st.Flr.SF) 1 3.024 3.024 210.26 < 2.2e-16 ***
## log(Lot.Area) 1 1.870 1.870 130.02 < 2.2e-16 ***
## Overall.Cond 1 4.250 4.250 295.52 < 2.2e-16 ***
## Overall.Qual 1 4.554 4.554 316.69 < 2.2e-16 ***
## Residuals 823 11.836 0.014
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Section 3.5 Model Testing
The model proposed in section 3.1was used as initial model to run initial predictions on the test data set. The comparison of the RMSE by using both training- and test data, while predicting house prices indicated that the model was not overfitting.
Changes have taken place towards removing certain non value add features (e.g. Pool.Area and Pool.QC). I also went back to change features (e.g. Sale.Condition etc.), by clustering parameter values. Abnormal has been set to 1, while all others have been set to 0. This also improved the performance. Then I focused on the non - linearity of the model. I applied quadratic and log transformation to certain (e.g. Lot.Area, BsmtFin.SF.1, X1st.Flr.SF etc.) as indicated above. All this measures helped to improve the performance.
The evaluation on the model has mainly been performed via the Root Mean Squared Error.
Part 4 Final Model Assessment
Section 4.1 Final Model Residual
Final.lm <- lm(log(price) ~log(Total.Bsmt.SF + 1) + log(area)+ Bedroom.AbvGr + Year.Built + log(X1st.Flr.SF) +
log(Lot.Area) + Overall.Cond+Overall.Qual,
data =na.omit(train),pior="BIC")## Warning: In lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) :
## extra argument 'pior' will be disregardedpredict.Final <- exp(predict(Final.lm, train))
resid.Final <- train$price - predict.Final
plot(resid(Final.lm))
plot(Final.lm)
model.resid<-Final.lm$residuals
plot(model.resid, main="Residuals",pch=20)
abline(0,0, lwd=2, col="blueviolet")
qqnorm(residuals(Final.lm))
qqline(residuals(Final.lm))
BIC.bas <- bas.lm(log(price) ~log(Garage.Area + 1) +
log(Total.Bsmt.SF + 1) + Garage.Cars + log(area) +
Full.Bath + Half.Bath +
Bedroom.AbvGr + Year.Built + log(X1st.Flr.SF) +
log(X2nd.Flr.SF + 1) +
log(Lot.Area) + Overall.Cond+Overall.Qual,
data =na.omit(train), prior="BIC",
modelprior=uniform())
summary(BIC.bas)## P(B != 0 | Y) model 1 model 2
## Intercept 1.00000000 1.0000 1.0000000
## log(Garage.Area + 1) 0.03855625 0.0000 0.0000000
## log(Total.Bsmt.SF + 1) 1.00000000 1.0000 1.0000000
## Garage.Cars 0.99996052 1.0000 1.0000000
## log(area) 1.00000000 1.0000 1.0000000
## Full.Bath 0.04897608 0.0000 0.0000000
## Half.Bath 0.04306068 0.0000 0.0000000
## Bedroom.AbvGr 0.99999409 1.0000 1.0000000
## Year.Built 1.00000000 1.0000 1.0000000
## log(X1st.Flr.SF) 0.85638295 1.0000 0.0000000
## log(X2nd.Flr.SF + 1) 0.18067164 0.0000 1.0000000
## log(Lot.Area) 1.00000000 1.0000 1.0000000
## Overall.Cond 1.00000000 1.0000 1.0000000
## Overall.Qual 1.00000000 1.0000 1.0000000
## BF NA 1.0000 0.1766875
## PostProbs NA 0.7160 0.1265000
## R2 NA 0.9033 0.9029000
## dim NA 10.0000 10.0000000
## logmarg NA -1047.2501 -1048.9834280
## model 3 model 4 model 5
## Intercept 1.000000e+00 1.000000e+00 1.000000e+00
## log(Garage.Area + 1) 0.000000e+00 0.000000e+00 0.000000e+00
## log(Total.Bsmt.SF + 1) 1.000000e+00 1.000000e+00 1.000000e+00
## Garage.Cars 1.000000e+00 1.000000e+00 1.000000e+00
## log(area) 1.000000e+00 1.000000e+00 1.000000e+00
## Full.Bath 1.000000e+00 0.000000e+00 0.000000e+00
## Half.Bath 0.000000e+00 1.000000e+00 0.000000e+00
## Bedroom.AbvGr 1.000000e+00 1.000000e+00 1.000000e+00
## Year.Built 1.000000e+00 1.000000e+00 1.000000e+00
## log(X1st.Flr.SF) 1.000000e+00 1.000000e+00 1.000000e+00
## log(X2nd.Flr.SF + 1) 0.000000e+00 0.000000e+00 1.000000e+00
## log(Lot.Area) 1.000000e+00 1.000000e+00 1.000000e+00
## Overall.Cond 1.000000e+00 1.000000e+00 1.000000e+00
## Overall.Qual 1.000000e+00 1.000000e+00 1.000000e+00
## BF 5.142395e-02 4.658921e-02 4.518892e-02
## PostProbs 3.680000e-02 3.340000e-02 3.240000e-02
## R2 9.034000e-01 9.034000e-01 9.034000e-01
## dim 1.100000e+01 1.100000e+01 1.100000e+01
## logmarg -1.050218e+03 -1.050316e+03 -1.050347e+03image(BIC.bas, rotate = F)
coef(BIC.bas)##
## Marginal Posterior Summaries of Coefficients:
##
## Using BMA
##
## Based on the top 8192 models
## post mean post SD post p(B != 0)
## Intercept 1.199e+01 4.090e-03 1.000e+00
## log(Garage.Area + 1) -9.198e-05 9.950e-04 3.856e-02
## log(Total.Bsmt.SF + 1) 2.750e-02 3.981e-03 1.000e+00
## Garage.Cars 4.166e-02 7.849e-03 1.000e+00
## log(area) 4.489e-01 5.434e-02 1.000e+00
## Full.Bath -4.966e-04 3.356e-03 4.898e-02
## Half.Bath 3.437e-04 2.858e-03 4.306e-02
## Bedroom.AbvGr -3.850e-02 6.915e-03 1.000e+00
## Year.Built 3.919e-03 2.041e-04 1.000e+00
## log(X1st.Flr.SF) 1.132e-01 5.060e-02 8.564e-01
## log(X2nd.Flr.SF + 1) -1.921e-03 4.518e-03 1.807e-01
## log(Lot.Area) 1.174e-01 9.494e-03 1.000e+00
## Overall.Cond 5.799e-02 4.121e-03 1.000e+00
## Overall.Qual 8.630e-02 5.056e-03 1.000e+00plot(BIC.bas)
train.ZS = bas.lm(log(price) ~ log(Garage.Area + 1) +
log(Total.Bsmt.SF + 1) + Garage.Cars + log(area) +
Full.Bath + Half.Bath +
Bedroom.AbvGr + Year.Built + log(X1st.Flr.SF) +
log(X2nd.Flr.SF + 1) +
log(Lot.Area) + Overall.Cond+Overall.Qual,
data =na.omit(train),prior="ZS-null",
modelprior=uniform(), initprobs="eplogp") ## Warning in bas.lm(log(price) ~ log(Garage.Area + 1) + log(Total.Bsmt.SF
## + : We recommend using the implementation using the Jeffreys-Zellner-Siow
## prior (prior='JZS') which uses numerical integration rahter than the Laplace
## approximationplot(train.ZS, ask=F) 
train.ZS ##
## Call:
## bas.lm(formula = log(price) ~ log(Garage.Area + 1) + log(Total.Bsmt.SF + 1) + Garage.Cars + log(area) + Full.Bath + Half.Bath + Bedroom.AbvGr + Year.Built + log(X1st.Flr.SF) + log(X2nd.Flr.SF + 1) + log(Lot.Area) + Overall.Cond + Overall.Qual, data = na.omit(train), prior = "ZS-null", modelprior = uniform(), initprobs = "eplogp")
##
##
## Marginal Posterior Inclusion Probabilities:
## Intercept log(Garage.Area + 1) log(Total.Bsmt.SF + 1)
## 1.00000 0.03365 1.00000
## Garage.Cars log(area) Full.Bath
## 0.99996 1.00000 0.04259
## Half.Bath Bedroom.AbvGr Year.Built
## 0.03749 0.99999 1.00000
## log(X1st.Flr.SF) log(X2nd.Flr.SF + 1) log(Lot.Area)
## 0.75559 0.27658 1.00000
## Overall.Cond Overall.Qual
## 1.00000 1.00000options(width = 80)
summary(train.ZS)## P(B != 0 | Y) model 1 model 2 model 3
## Intercept 1.00000000 1.0000 1.0000000 1.0000000
## log(Garage.Area + 1) 0.03365489 0.0000 0.0000000 0.0000000
## log(Total.Bsmt.SF + 1) 1.00000000 1.0000 1.0000000 1.0000000
## Garage.Cars 0.99995788 1.0000 1.0000000 1.0000000
## log(area) 1.00000000 1.0000 1.0000000 1.0000000
## Full.Bath 0.04258669 0.0000 0.0000000 0.0000000
## Half.Bath 0.03749060 0.0000 0.0000000 0.0000000
## Bedroom.AbvGr 0.99999331 1.0000 1.0000000 1.0000000
## Year.Built 1.00000000 1.0000 1.0000000 1.0000000
## log(X1st.Flr.SF) 0.75559417 1.0000 0.0000000 0.0000000
## log(X2nd.Flr.SF + 1) 0.27657745 0.0000 1.0000000 1.0000000
## log(Lot.Area) 1.00000000 1.0000 1.0000000 1.0000000
## Overall.Cond 1.00000000 1.0000 1.0000000 1.0000000
## Overall.Qual 1.00000000 1.0000 1.0000000 1.0000000
## BF NA 1.0000 0.1807785 0.1807785
## PostProbs NA 0.6339 0.1146000 0.1146000
## R2 NA 0.9033 0.9029000 0.9029000
## dim NA 10.0000 10.0000000 10.0000000
## logmarg NA 934.3534 932.6428674 932.6428674
## model 4 model 5
## Intercept 1.00000000 1.00000000
## log(Garage.Area + 1) 0.00000000 0.00000000
## log(Total.Bsmt.SF + 1) 1.00000000 1.00000000
## Garage.Cars 1.00000000 1.00000000
## log(area) 1.00000000 1.00000000
## Full.Bath 1.00000000 0.00000000
## Half.Bath 0.00000000 1.00000000
## Bedroom.AbvGr 1.00000000 1.00000000
## Year.Built 1.00000000 1.00000000
## log(X1st.Flr.SF) 1.00000000 1.00000000
## log(X2nd.Flr.SF + 1) 0.00000000 0.00000000
## log(Lot.Area) 1.00000000 1.00000000
## Overall.Cond 1.00000000 1.00000000
## Overall.Qual 1.00000000 1.00000000
## BF 0.05015929 0.04550812
## PostProbs 0.03180000 0.02880000
## R2 0.90340000 0.90340000
## dim 11.00000000 11.00000000
## logmarg 931.36079871 931.26348567image(train.ZS, rotate=F)
coef.ZS = coef(train.ZS)
plot(coef.ZS, subset=c(5:6), ask=F)
plot(coef.ZS, ask=FALSE)
confint(coef.ZS)## 2.5% 97.5% beta
## Intercept 11.984812762 12.000946744 1.199280e+01
## log(Garage.Area + 1) 0.000000000 0.000000000 -8.014107e-05
## log(Total.Bsmt.SF + 1) 0.019806791 0.035448308 2.754503e-02
## Garage.Cars 0.026122331 0.057088978 4.156013e-02
## log(area) 0.379751807 0.591814912 4.619211e-01
## Full.Bath 0.000000000 0.000000000 -4.313148e-04
## Half.Bath 0.000000000 0.000000000 2.982645e-04
## Bedroom.AbvGr -0.052718789 -0.025406463 -3.853678e-02
## Year.Built 0.003485515 0.004296364 3.904278e-03
## log(X1st.Flr.SF) 0.000000000 0.161074781 9.979002e-02
## log(X2nd.Flr.SF + 1) -0.014112471 0.000000000 -3.155182e-03
## log(Lot.Area) 0.098807861 0.136119703 1.175414e-01
## Overall.Cond 0.049667868 0.065888525 5.795340e-02
## Overall.Qual 0.076431685 0.096273712 8.627507e-02
## attr(,"Probability")
## [1] 0.95
## attr(,"class")
## [1] "confint.bas"plot(confint(coef.ZS, parm=2:9))
## NULLmuhat.BMA = fitted(train.ZS, estimator="BMA")
BMA = predict(train.ZS, estimator="BMA")
par(mar=c(9, 9, 3, 3))
plot(muhat.BMA, BMA$fit,
pch=16,
xlab=expression(hat(mu[i])), ylab=expression(hat(Y[i])))
abline(0,1)
BPM = predict(train.ZS, estimator="BPM")
(train.ZS$namesx[attr(BPM$fit, 'model') +1])[-1]## [1] "log(Total.Bsmt.SF + 1)" "Garage.Cars" "log(area)"
## [4] "Bedroom.AbvGr" "Year.Built" "log(X1st.Flr.SF)"
## [7] "log(X2nd.Flr.SF + 1)" "log(Lot.Area)" "Overall.Cond"
## [10] "Overall.Qual"#new.pred = predict(Final.ZS, newdata=test, estimator="MPM")
Final.ZS = bas.lm(price ~ .,
data=test,
prior="ZS-null",
modelprior=uniform(),
method = "MCMC", MCMC.iterations = 10^6) ## Warning in bas.lm(price ~ ., data = test, prior = "ZS-null", modelprior =
## uniform(), : We recommend using the implementation using the Jeffreys-Zellner-
## Siow prior (prior='JZS') which uses numerical integration rahter than the
## Laplace approximationBPM = predict(Final.ZS, estimator="BPM", se.fit=TRUE)
Final.conf.fit = confint(BPM, parm="mean")
Final.conf.pred = confint(BPM, parm="pred")
cbind(Final.conf.fit, Final.conf.pred)## 2.5% 97.5% mean 2.5% 97.5% pred
## [1,] 289225.009 300953.59 295089.30 250712.643 339465.96 295089.30
## [2,] 228493.042 238387.05 233440.05 189175.267 277704.82 233440.05
## [3,] 109256.027 117959.76 113607.89 69405.671 157810.12 113607.89
## [4,] 152114.304 159897.58 156005.94 111846.657 200165.23 156005.94
## [5,] 251191.070 261816.96 256504.01 212196.846 300811.18 256504.01
## [6,] 197529.103 208795.75 203162.43 158815.702 247509.15 203162.43
## [7,] 106033.774 127069.02 116551.40 71323.991 161778.80 116551.40
## [8,] 223207.799 236646.84 229927.32 185429.569 274425.07 229927.32
## [9,] 245096.436 256264.10 250680.27 206339.802 295020.73 250680.27
## [10,] 158315.595 172562.87 165439.23 120878.667 209999.80 165439.23
## [11,] 51766.475 62883.95 57325.21 12987.899 101662.53 57325.21
## [12,] 230276.896 237545.40 233911.15 189773.797 278048.49 233911.15
## [13,] 293610.980 305327.80 299469.39 255093.512 343845.27 299469.39
## [14,] 220505.423 231788.29 226146.86 181799.102 270494.62 226146.86
## [15,] 164808.591 175548.99 170178.79 125864.721 214492.86 170178.79
## [16,] 153962.910 162315.65 158139.28 113953.991 202324.57 158139.28
## [17,] 129442.353 141079.26 135260.81 90890.183 179631.43 135260.81
## [18,] 145615.150 156031.62 150823.38 106528.649 195118.12 150823.38
## [19,] 130462.658 138786.70 134624.68 90440.741 178808.61 134624.68
## [20,] 180459.525 194238.89 187349.21 142825.446 231872.97 187349.21
## [21,] 296846.527 304750.65 300798.59 256633.937 344963.24 300798.59
## [22,] 187296.080 194889.22 191092.65 146941.641 235243.66 191092.65
## [23,] 130739.567 138230.30 134484.93 90338.295 178631.57 134484.93
## [24,] 193626.528 204838.30 199232.41 154889.163 243575.66 199232.41
## [25,] 143558.850 152199.02 147878.93 103679.829 192078.04 147878.93
## [26,] 173653.970 183022.77 178338.37 134102.170 222574.57 178338.37
## [27,] 156077.450 167772.44 161924.95 117550.505 206299.39 161924.95
## [28,] 149190.246 159370.95 154280.60 109999.571 198561.63 154280.60
## [29,] 125180.730 141714.46 133447.60 88690.040 178205.15 133447.60
## [30,] 152195.348 174395.30 163295.32 117928.957 208661.69 163295.32
## [31,] 91716.003 104382.48 98049.24 53608.182 142490.30 98049.24
## [32,] 174336.744 181661.23 177998.99 133859.326 222138.65 177998.99
## [33,] 114852.536 125804.09 120328.31 76001.323 164655.30 120328.31
## [34,] 188923.224 196840.52 192881.87 148716.630 237047.12 192881.87
## [35,] 228951.283 236369.63 232660.45 188516.874 276804.04 232660.45
## [36,] 178514.299 189266.20 183890.25 139575.483 228205.02 183890.25
## [37,] 104744.225 112754.11 108749.17 64579.751 152918.58 108749.17
## [38,] 146892.269 157334.67 152113.47 107817.207 196409.73 152113.47
## [39,] 182075.905 188129.42 185102.66 141011.178 229194.15 185102.66
## [40,] 279476.154 288005.17 283740.66 239546.955 327934.37 283740.66
## [41,] 196787.530 205437.11 201112.32 156912.754 245311.88 201112.32
## [42,] 114120.909 123053.65 118587.28 74373.635 162800.92 118587.28
## [43,] 134684.208 141422.38 138053.29 93936.987 182169.60 138053.29
## [44,] 189592.017 200908.20 195250.11 150900.228 239599.99 195250.11
## [45,] 206985.119 216615.60 211800.36 167550.113 256050.61 211800.36
## [46,] 113380.596 122070.76 117725.68 73524.124 161927.24 117725.68
## [47,] 266900.631 275002.82 270951.73 226778.101 315125.35 270951.73
## [48,] 104796.442 118950.88 111873.66 67320.491 156426.83 111873.66
## [49,] 171022.679 186124.23 178573.46 133942.615 223204.30 178573.46
## [50,] 135686.051 158568.10 147127.08 101676.061 192578.09 147127.08
## [51,] 120388.003 129900.60 125144.30 80900.429 169388.17 125144.30
## [52,] 101455.023 111727.23 106591.13 62304.814 150877.44 106591.13
## [53,] 160823.226 171843.37 166333.30 122002.058 210664.53 166333.30
## [54,] 292453.432 302424.32 297438.88 253169.786 341707.97 297438.88
## [55,] 96745.844 111160.07 103952.96 59378.969 148526.94 103952.96
## [56,] 158114.963 168684.95 163399.96 119096.133 207703.78 163399.96
## [57,] 128212.369 136434.25 132323.31 88144.156 176502.46 132323.31
## [58,] 116512.976 126811.53 121662.25 77374.411 165950.09 121662.25
## [59,] 124111.677 132369.59 128240.63 84059.800 172421.47 128240.63
## [60,] 227249.300 235808.08 231528.69 187333.543 275723.83 231528.69
## [61,] 121927.666 129431.34 125679.50 81532.316 169826.69 125679.50
## [62,] 322641.654 341661.03 332151.34 287147.654 377155.03 332151.34
## [63,] 254885.619 266962.14 260923.88 216523.897 305323.86 260923.88
## [64,] 61651.664 71541.91 66596.79 22332.218 110861.36 66596.79
## [65,] 108395.608 119116.21 113755.91 69443.038 158068.78 113755.91
## [66,] 185130.181 192966.07 189048.13 144886.513 233209.74 189048.13
## [67,] 69234.402 81601.95 75418.18 30998.170 119838.18 75418.18
## [68,] 236370.012 248209.34 242289.68 197905.667 286673.68 242289.68
## [69,] 215905.057 228816.65 222360.85 177902.164 266819.54 222360.85
## [70,] 73692.558 90295.34 81993.95 37230.003 126757.90 81993.95
## [71,] 254663.333 263049.15 258856.24 214669.388 303043.10 258856.24
## [72,] 170710.602 178081.34 174395.97 130254.382 218537.56 174395.97
## [73,] 96164.632 107798.58 101981.61 57611.178 146352.04 101981.61
## [74,] 139077.584 146844.12 142960.85 98802.299 187119.40 142960.85
## [75,] 253965.082 269117.09 261541.08 216905.968 306176.20 261541.08
## [76,] 138893.309 158098.94 148496.13 103472.667 193519.59 148496.13
## [77,] 134377.791 143173.54 138775.66 94568.888 182982.44 138775.66
## [78,] 147589.355 158988.33 153288.84 108933.663 197644.02 153288.84
## [79,] 245455.081 254482.27 249968.67 205750.236 294187.11 249968.67
## [80,] 107392.814 120330.09 113861.45 69400.896 158322.01 113861.45
## [81,] 120885.250 129684.68 125284.96 81078.005 169491.92 125284.96
## [82,] 117884.131 127527.72 122705.93 78454.967 166956.89 122705.93
## [83,] 109749.211 118706.81 114228.01 70013.111 158442.91 114228.01
## [84,] 113395.348 127785.99 120590.67 76018.588 165162.75 120590.67
## [85,] 145015.861 154613.02 149814.44 105566.005 194062.88 149814.44
## [86,] 153931.706 162087.23 158009.47 113833.389 202185.55 158009.47
## [87,] 129235.020 137785.78 133510.40 89315.640 177705.16 133510.40
## [88,] 108151.264 123473.09 115812.18 71162.572 160461.79 115812.18
## [89,] 244992.564 255154.11 250073.33 205793.407 294353.26 250073.33
## [90,] 208059.705 217700.42 212880.06 168629.258 257130.87 212880.06
## [91,] 120130.996 131939.16 126035.08 81653.148 170417.01 126035.08
## [92,] 139574.183 147467.96 143521.07 99356.883 187685.26 143521.07
## [93,] 137549.643 148609.79 143079.72 98745.990 187413.45 143079.72
## [94,] 249435.112 262331.70 255883.40 211425.803 300341.00 255883.40
## [95,] 112063.395 131468.96 121766.18 76721.290 166811.06 121766.18
## [96,] 221981.843 228205.50 225093.67 180996.264 269191.08 225093.67
## [97,] 120031.175 131024.99 125528.08 81198.477 169857.68 125528.08
## [98,] 133212.796 141492.08 137352.44 93170.606 181534.27 137352.44
## [99,] 308169.584 323475.92 315822.75 271174.472 360471.03 315822.75
## [100,] 119467.846 128676.30 124072.07 79844.291 168299.85 124072.07
## [101,] 134582.558 146568.74 140575.65 96181.786 184969.51 140575.65
## [102,] 86238.901 103897.54 95068.22 50203.371 139933.07 95068.22
## [103,] 269410.264 281291.28 275350.77 230963.979 319737.57 275350.77
## [104,] 121971.414 134455.57 128213.49 83785.333 172641.65 128213.49
## [105,] 127952.921 136158.50 132055.71 87877.316 176234.11 132055.71
## [106,] 250300.806 262299.28 256300.04 211905.351 300694.73 256300.04
## [107,] 179633.189 190011.59 184822.39 140529.891 229114.89 184822.39
## [108,] 174224.538 181089.05 177656.79 133535.617 221777.97 177656.79
## [109,] 271115.079 279872.90 275493.99 231289.097 319698.88 275493.99
## [110,] 123265.751 132832.55 128049.15 83802.356 172295.94 128049.15
## [111,] 201668.853 210893.63 206281.24 162052.609 250509.87 206281.24
## [112,] 163719.425 172950.69 168335.06 124106.086 212564.03 168335.06
## [113,] 204078.692 212496.22 208287.45 164099.091 252475.82 208287.45
## [114,] 209273.597 224890.69 217082.14 172406.968 261757.32 217082.14
## [115,] 26663.220 43923.68 35293.45 -9532.643 80119.54 35293.45
## [116,] 148731.756 158610.47 153671.11 109407.189 197935.04 153671.11
## [117,] 168382.939 175541.42 171962.18 127829.326 216095.03 171962.18
## [118,] 191094.934 203459.41 197277.17 152857.381 241696.96 197277.17
## [119,] 108252.109 119738.58 113995.34 69634.523 158356.17 113995.34
## [120,] 203000.077 210953.96 206977.02 162810.133 251143.91 206977.02
## [121,] 202526.962 213089.21 207808.08 163504.721 252111.45 207808.08
## [122,] 260583.231 268550.80 264567.01 220399.512 308734.52 264567.01
## [123,] 258134.474 266322.10 262228.29 218050.725 306405.85 262228.29
## [124,] 202829.834 210970.27 206900.05 162724.670 251075.44 206900.05
## [125,] 117041.369 128814.13 122927.75 78548.170 167307.33 122927.75
## [126,] 108863.049 120278.50 114570.78 70214.537 158927.02 114570.78
## [127,] 133122.428 140786.82 136954.63 92800.537 181108.71 136954.63
## [128,] 236836.092 243866.30 240351.20 196223.498 284478.89 240351.20
## [129,] 162532.380 169780.87 166156.62 122020.098 210293.15 166156.62
## [130,] 106207.958 116316.54 111262.25 66985.353 155539.15 111262.25
## [131,] 179107.409 186712.97 182910.19 138758.646 227061.74 182910.19
## [132,] 138255.240 148281.93 143268.59 98996.344 187540.83 143268.59
## [133,] 262034.991 270501.86 266268.42 222077.704 310459.14 266268.42
## [134,] 136896.075 150031.48 143463.78 98988.699 187938.85 143463.78
## [135,] 207745.398 217776.83 212761.12 168488.605 257033.63 212761.12
## [136,] 104539.601 112586.60 108563.10 64392.000 152734.21 108563.10
## [137,] 189716.460 197918.35 193817.40 149639.179 237995.63 193817.40
## [138,] 123612.384 131641.63 127627.01 83456.712 171797.30 127627.01
## [139,] 179738.763 186325.37 183032.06 138921.481 227142.65 183032.06
## [140,] 216484.604 222783.19 219633.90 175533.832 263733.96 219633.90
## [141,] 225882.666 232379.91 229131.29 185024.016 273238.56 229131.29
## [142,] 152801.687 158722.06 155761.87 111674.909 199848.84 155761.87
## [143,] 354398.528 367636.95 361017.74 316535.028 405500.45 361017.74
## [144,] 152596.653 159910.78 156253.71 112114.482 200392.95 156253.71
## [145,] 317966.885 328362.65 323164.77 278871.249 367458.29 323164.77
## [146,] 169803.033 178498.20 174150.62 129948.815 218352.42 174150.62
## [147,] 56548.087 66769.94 61659.02 17375.617 105942.41 61659.02
## [148,] 156671.936 168513.75 162592.84 118208.669 206977.02 162592.84
## [149,] 83573.876 104959.50 94266.69 48998.222 139535.15 94266.69
## [150,] 234891.272 249488.47 242189.87 197601.002 286778.74 242189.87
## [151,] 305758.164 321015.42 313386.79 268742.712 358030.87 313386.79
## [152,] 232820.430 241844.34 237332.39 193114.114 281550.66 237332.39
## [153,] 266532.664 277135.86 271834.26 227528.454 316140.07 271834.26
## [154,] 108231.509 118182.24 113206.87 68938.917 157474.83 113206.87
## [155,] 185857.805 196230.88 191044.34 146752.154 235336.53 191044.34
## [156,] 124048.482 133578.13 128813.31 84568.519 173058.10 128813.31
## [157,] 157806.756 168049.07 162927.91 118643.333 207212.49 162927.91
## [158,] 271953.299 279960.48 275956.89 231787.595 320126.18 275956.89
## [159,] 148986.230 161195.54 155090.88 110681.825 199499.94 155090.88
## [160,] 212203.345 218962.51 215582.93 171465.817 259700.04 215582.93
## [161,] 311308.339 335976.81 323642.58 277958.540 369326.61 323642.58
## [162,] 136287.947 149263.60 142775.78 98312.423 187239.13 142775.78
## [163,] 166496.598 174898.07 170697.33 126509.733 214884.93 170697.33
## [164,] 132646.714 145166.37 138906.54 94475.887 183337.20 138906.54
## [165,] 93702.012 104028.54 98865.27 54575.806 143154.74 98865.27
## [166,] 101716.149 115997.88 108857.02 64293.693 153420.34 108857.02
## [167,] 271585.660 283952.40 277769.03 233349.080 322188.98 277769.03
## [168,] 195893.463 204220.58 200057.02 155872.941 244241.10 200057.02
## [169,] 252088.602 262237.05 257162.82 212883.647 301442.00 257162.82
## [170,] 178987.271 190200.43 184593.85 140250.515 228937.19 184593.85
## [171,] 105581.315 113091.43 109336.37 65188.913 153483.83 109336.37
## [172,] 98168.509 114267.23 106217.87 61499.977 150935.76 106217.87
## [173,] 152185.120 162258.95 157222.04 112947.118 201496.95 157222.04
## [174,] 171970.178 180591.03 176280.60 132082.441 220478.77 176280.60
## [175,] 179815.751 189744.25 184780.00 140513.292 229046.71 184780.00
## [176,] 194848.432 203301.88 199075.15 154885.077 243265.23 199075.15
## [177,] 318897.181 335397.53 327147.36 282392.879 371901.83 327147.36
## [178,] 198997.650 207435.22 203216.43 159027.115 247405.75 203216.43
## [179,] 164178.266 172638.82 168408.54 124218.125 212598.96 168408.54
## [180,] 176856.333 184987.91 180922.12 136747.149 225097.10 180922.12
## [181,] 151473.381 159665.33 155569.35 111391.591 199747.12 155569.35
## [182,] 78825.378 86578.77 82702.08 38544.103 126860.05 82702.08
## [183,] 218948.439 228068.52 223508.48 179285.277 267731.69 223508.48
## [184,] 109530.172 119620.48 114575.32 70299.470 158851.18 114575.32
## [185,] 297084.244 306479.16 301781.70 257544.116 346019.28 301781.70
## [186,] 167486.593 178605.37 173045.98 128708.584 217383.37 173045.98
## [187,] 117324.577 127299.17 122311.88 78042.576 166581.18 122311.88
## [188,] 225004.434 233461.83 229233.13 185042.866 273423.40 229233.13
## [189,] 167374.802 176128.78 171751.79 127547.087 215956.49 171751.79
## [190,] 166974.094 178715.12 172844.61 128467.129 217222.09 172844.61
## [191,] 153895.611 162873.03 158384.32 114168.416 202600.23 158384.32
## [192,] 171086.197 178897.96 174992.08 130831.534 219152.62 174992.08
## [193,] 138779.972 148010.46 143395.22 99166.288 187624.15 143395.22
## [194,] 173799.035 182361.33 178080.18 133884.866 222275.50 178080.18
## [195,] 177528.307 189231.26 183379.78 139004.817 227754.75 183379.78
## [196,] 145615.150 156031.62 150823.38 106528.649 195118.12 150823.38
## [197,] 127151.700 135293.46 131222.58 87047.138 175398.03 131222.58
## [198,] 205928.665 217968.14 211948.40 167550.938 256345.87 211948.40
## [199,] 250404.495 260509.27 255456.88 211180.204 299733.56 255456.88
## [200,] 254484.815 265298.19 259891.50 215572.994 304210.01 259891.50
## [201,] 139860.700 150325.21 145092.95 100795.389 189390.52 145092.95
## [202,] 250074.884 270766.19 260420.54 215232.813 305608.26 260420.54
## [203,] 131407.799 141354.48 136381.14 92113.411 180648.87 136381.14
## [204,] 135476.432 145287.63 140382.03 96121.862 184642.20 140382.03
## [205,] 138980.837 147357.53 143169.19 98982.762 187355.61 143169.19
## [206,] 281804.519 290790.64 286297.58 242081.230 330513.92 286297.58
## [207,] 145243.368 156775.17 151009.27 106645.507 195373.03 151009.27
## [208,] 206104.630 213935.69 210020.16 165858.760 254181.56 210020.16
## [209,] 145229.167 154261.56 149745.37 105526.661 193964.07 149745.37
## [210,] 217964.313 229229.37 223596.84 179250.217 267943.47 223596.84
## [211,] 274851.698 285899.64 280375.67 236042.700 324708.64 280375.67
## [212,] 292396.340 301940.05 297168.19 252922.647 341413.74 297168.19
## [213,] 135674.856 156773.37 146224.11 100989.342 191458.89 146224.11
## [214,] 211726.469 224345.35 218035.91 173598.235 262473.58 218035.91
## [215,] 184873.593 191177.92 188025.76 143925.485 232126.03 188025.76
## [216,] 136083.506 146984.06 141533.78 97209.937 185857.63 141533.78
## [217,] 147175.075 155927.51 151551.29 107346.666 195755.92 151551.29
## [218,] 172107.335 180992.79 176550.06 132338.801 220761.32 176550.06
## [219,] 229935.267 242323.61 236129.44 191707.983 280550.89 236129.44
## [220,] 248706.670 259130.97 253918.82 209623.625 298214.02 253918.82
## [221,] 209323.879 218168.26 213746.07 169536.867 257955.27 213746.07
## [222,] 206952.926 218669.53 212811.23 168435.362 257187.09 212811.23
## [223,] 160761.653 169484.52 165123.09 120919.920 209326.25 165123.09
## [224,] 137977.923 151009.52 144493.72 100026.279 188961.16 144493.72
## [225,] 144033.679 151572.65 147803.16 103654.474 191951.85 147803.16
## [226,] 83046.757 91087.89 87067.32 42896.486 131238.16 87067.32
## [227,] 115828.346 126311.08 121069.71 76771.069 165368.35 121069.71
## [228,] 253501.850 261346.68 257424.27 213262.256 301586.28 257424.27
## [229,] 117715.276 128377.12 123046.20 78736.871 167355.53 123046.20
## [230,] 199361.462 206841.03 203101.25 158955.082 247247.41 203101.25
## [231,] 182816.027 190930.89 186873.46 142699.253 231047.67 186873.46
## [232,] 17835.259 41683.13 29759.19 -15815.779 75334.16 29759.19
## [233,] 231624.142 241367.97 236496.06 192239.606 280752.51 236496.06
## [234,] 185561.904 196100.70 190831.30 146529.336 235133.27 190831.30
## [235,] 110471.198 126138.27 118304.73 73625.183 162984.28 118304.73
## [236,] 100354.657 123151.79 111753.22 66312.876 157193.57 111753.22
## [237,] 143532.047 151326.45 147429.25 103269.473 191589.03 147429.25
## [238,] 160490.968 171105.56 165798.27 121491.774 210104.76 165798.27
## [239,] 197734.366 206443.85 202089.11 157886.602 246291.61 202089.11
## [240,] 187139.493 194871.68 191005.59 146848.544 235162.63 191005.59
## [241,] 266138.608 274364.52 270251.56 226072.223 314430.91 270251.56
## [242,] 287519.689 298535.26 293027.47 248696.519 337358.43 293027.47
## [243,] 222018.380 231771.64 226895.01 182638.042 271151.98 226895.01
## [244,] 132209.999 143419.50 137814.75 93471.645 182157.86 137814.75
## [245,] 113050.931 121491.78 117271.36 73081.881 161460.83 117271.36
## [246,] 292125.263 301925.64 297025.45 252765.880 341285.02 297025.45
## [247,] 161397.359 171167.39 166282.37 122024.479 210540.27 166282.37
## [248,] 169626.396 183212.15 176419.27 131910.386 220928.16 176419.27
## [249,] 146398.699 155333.85 150866.27 106652.508 195080.04 150866.27
## [250,] 223992.965 232889.21 228441.09 184229.283 272652.89 228441.09
## [251,] 205074.801 216371.58 210723.19 166374.550 255071.84 210723.19
## [252,] 298074.405 307982.34 303028.37 258762.816 347293.93 303028.37
## [253,] 168862.306 178441.76 173652.03 129404.555 217899.51 173652.03
## [254,] 268135.845 276402.37 272269.11 228087.869 316450.34 272269.11
## [255,] 182126.595 189742.36 185934.48 141782.493 230086.46 185934.48
## [256,] 203973.248 211400.15 207686.70 163542.760 251830.64 207686.70
## [257,] 131243.434 138470.16 134856.80 90721.162 178992.43 134856.80
## [258,] 190208.320 198243.82 194226.07 150055.490 238396.65 194226.07
## [259,] 198648.099 218242.48 208445.29 163379.975 253510.61 208445.29
## [260,] 154154.644 165603.65 159879.15 115520.746 204237.55 159879.15
## [261,] 125902.347 145702.75 135802.55 90714.726 180890.37 135802.55
## [262,] 158956.254 175782.39 167369.32 122584.533 212154.12 167369.32
## [263,] 232321.249 242027.10 237174.17 192919.812 281428.54 237174.17
## [264,] 164415.071 175051.56 169733.32 125425.513 214041.12 169733.32
## [265,] 179583.942 187012.68 183298.31 139154.291 227442.33 183298.31
## [266,] 193572.456 207081.01 200326.73 155823.723 244829.75 200326.73
## [267,] 162385.787 171634.23 167010.01 122780.140 211239.88 167010.01
## [268,] 185531.068 196068.92 190799.99 146498.083 235101.91 190799.99
## [269,] 190841.505 197818.83 194330.17 150204.567 238455.77 194330.17
## [270,] 290756.334 333109.22 311932.78 263113.320 360752.24 311932.78
## [271,] 145895.307 155440.53 150667.92 106422.291 194913.55 150667.92
## [272,] 179768.380 186963.63 183366.00 139231.655 227500.35 183366.00
## [273,] 148548.115 158980.33 153764.22 109468.563 198059.88 153764.22
## [274,] 133260.582 145340.13 139300.36 94900.168 183700.54 139300.36
## [275,] 182686.181 189548.44 186117.31 141996.220 230238.40 186117.31
## [276,] 124115.547 132622.93 128369.24 84176.571 172561.90 128369.24
## [277,] 196125.993 207426.67 201776.33 157427.439 246125.22 201776.33
## [278,] 83069.097 106123.42 94596.26 49123.484 140069.03 94596.26
## [279,] 147016.596 158186.44 152601.52 108260.915 196942.12 152601.52
## [280,] 223284.085 229820.27 226552.18 182443.469 270660.89 226552.18
## [281,] 69429.244 83410.18 76419.71 31880.242 120959.18 76419.71
## [282,] 130913.911 140533.19 135723.55 91473.913 179973.19 135723.55
## [283,] 171235.610 178255.27 174745.44 130618.161 218872.72 174745.44
## [284,] 226299.247 233395.02 229847.14 185716.814 273977.46 229847.14
## [285,] 244692.362 255112.55 249902.46 205607.504 294197.41 249902.46
## [286,] 146910.818 154640.03 150775.43 106618.513 194932.34 150775.43
## [287,] 207832.254 216043.43 211937.84 167759.188 256116.50 211937.84
## [288,] 99517.681 111126.41 105322.05 60953.268 149690.82 105322.05
## [289,] 225448.607 240244.86 232846.73 188241.464 277452.00 232846.73
## [290,] 250922.555 258184.04 254553.30 210416.239 298690.36 254553.30
## [291,] 192732.454 204203.77 198468.11 154108.272 242827.96 198468.11
## [292,] 183083.130 193750.27 188416.70 144107.053 232726.34 188416.70
## [293,] 80963.432 92923.39 86943.41 42551.318 131335.50 86943.41
## [294,] 239043.957 247530.34 243287.15 199095.494 287478.81 243287.15
## [295,] 167092.155 180853.70 173972.93 129450.544 218495.32 173972.93
## [296,] 134738.824 146436.03 140587.43 96212.840 184962.01 140587.43
## [297,] 187465.251 197251.89 192358.57 148099.759 236617.38 192358.57
## [298,] 238904.621 248164.01 243534.32 199303.875 287764.76 243534.32
## [299,] 111750.624 120185.18 115967.90 71778.725 160157.08 115967.90
## [300,] 214744.114 222987.71 218865.91 174685.745 263046.07 218865.91
## [301,] 171599.501 178719.37 175159.43 131028.142 219290.73 175159.43
## [302,] 168623.065 177404.03 173013.55 128807.508 217219.59 173013.55
## [303,] 177797.471 186040.65 181919.06 137738.916 226099.21 181919.06
## [304,] 104899.631 113519.41 109209.52 65011.411 153407.63 109209.52
## [305,] 171036.401 179116.36 175076.38 130903.772 219248.99 175076.38
## [306,] 321122.082 332012.46 326567.27 282244.053 370890.49 326567.27
## [307,] 135071.425 155373.59 145222.51 100078.933 190366.08 145222.51
## [308,] 140641.998 157786.79 149214.39 104399.399 194029.39 149214.39
## [309,] 287211.912 304661.86 295936.89 251092.456 340781.32 295936.89
## [310,] 232956.361 244802.05 238879.21 194494.773 283263.64 238879.21
## [311,] 310103.102 324361.68 317232.39 272670.921 361793.86 317232.39
## [312,] 168376.177 179052.79 173714.48 129404.267 218024.70 173714.48
## [313,] 197631.928 206525.99 202078.96 157867.265 246290.65 202078.96
## [314,] 104536.697 112284.05 108410.38 64252.667 152568.08 108410.38
## [315,] 383581.985 406369.62 394975.80 349536.648 440414.96 394975.80
## [316,] 345981.778 358686.02 352333.90 307890.147 396777.65 352333.90
## [317,] 128080.660 139456.57 133768.61 89414.916 178122.31 133768.61
## [318,] 157273.420 166568.78 161921.10 117688.774 206153.43 161921.10
## [319,] 189833.693 197312.46 193573.08 149426.949 237719.21 193573.08
## [320,] 291241.857 300685.92 295963.89 251723.687 340204.09 295963.89
## [321,] 108012.656 128494.09 118253.37 73089.560 163417.19 118253.37
## [322,] 210567.056 219491.19 215029.12 170815.914 259242.33 215029.12
## [323,] 166873.655 176632.29 171752.97 127495.707 216010.24 171752.97
## [324,] 140344.435 147606.40 143975.42 99838.336 188112.50 143975.42
## [325,] 227778.344 237418.07 232598.21 188347.457 276848.96 232598.21
## [326,] 175714.688 183453.99 179584.34 135426.982 223741.69 179584.34
## [327,] 191784.159 209645.25 200714.70 155829.825 245599.58 200714.70
## [328,] 206470.919 216785.33 211628.12 167339.362 255916.89 211628.12
## [329,] 202889.844 210669.47 206779.66 162620.531 250938.78 206779.66
## [330,] 300153.216 309880.25 305016.73 260761.206 349272.26 305016.73
## [331,] 174564.415 188307.39 181435.90 136914.952 225956.86 181435.90
## [332,] 99353.959 109330.03 104341.99 60072.609 148611.38 104341.99
## [333,] 198578.124 207154.95 202866.54 158670.514 247062.56 202866.54
## [334,] 51585.745 60260.43 55923.09 11722.293 100123.88 55923.09
## [335,] 137464.008 145751.79 141607.90 97425.667 185790.13 141607.90
## [336,] 202248.685 210693.01 206470.85 162281.204 250660.49 206470.85
## [337,] 141201.363 148781.71 144991.54 100841.078 189142.00 144991.54
## [338,] 75863.487 88981.67 82422.58 37948.772 126896.38 82422.58
## [339,] 150526.918 158460.28 154493.60 110327.633 198659.56 154493.60
## [340,] 149681.264 159538.33 154609.80 110347.079 198872.52 154609.80
## [341,] 102662.346 112844.55 107753.45 63472.333 152034.56 107753.45
## [342,] 213330.945 221583.47 217457.21 173276.627 261637.79 217457.21
## [343,] 182112.203 189216.32 185664.26 141533.606 229794.92 185664.26
## [344,] 150572.830 162796.22 156684.52 112274.497 201094.55 156684.52
## [345,] 209048.267 217992.73 213520.50 169306.262 257734.73 213520.50
## [346,] 118332.801 126864.49 122598.65 78404.809 166792.48 122598.65
## [347,] 121494.295 131034.72 126264.51 82019.137 170509.87 126264.51
## [348,] 284076.670 294519.50 289298.09 245001.801 333594.37 289298.09
## [349,] 166866.060 179671.09 173268.58 128817.591 217719.56 173268.58
## [350,] 209143.580 218001.46 213572.52 169362.641 257782.40 213572.52
## [351,] 103791.018 113371.19 108581.11 64333.589 152828.62 108581.11
## [352,] 116184.168 125000.58 120592.37 76384.568 164800.18 120592.37
## [353,] 201587.638 209956.48 205772.06 161586.009 249958.11 205772.06
## [354,] 143982.734 154555.01 149268.87 104964.909 193572.83 149268.87
## [355,] 199722.267 206426.23 203074.25 158959.244 247189.25 203074.25
## [356,] 249143.823 258045.56 253594.69 209382.609 297806.77 253594.69
## [357,] 144468.211 153017.12 148742.66 104547.995 192937.33 148742.66
## [358,] 133239.032 143032.48 138135.75 93876.567 182394.94 138135.75
## [359,] 143315.378 153210.21 148262.80 103997.971 192527.62 148262.80
## [360,] 81474.086 95002.85 88238.47 43733.923 132743.02 88238.47
## [361,] 155761.562 165559.96 160660.76 116401.298 204920.22 160660.76
## [362,] 158863.688 168794.11 163828.90 119562.083 208095.72 163828.90
## [363,] 151886.749 162546.61 157216.68 112907.472 201525.89 157216.68
## [364,] 176401.118 186907.89 181654.50 137354.438 225954.57 181654.50
## [365,] 71656.512 83856.23 77756.37 33347.970 122164.77 77756.37
## [366,] 222236.882 229422.19 225829.54 181695.593 269963.48 225829.54
## [367,] 188032.756 194940.33 191486.54 147363.685 235609.40 191486.54
## [368,] 266565.392 277284.76 271925.07 227612.277 316237.87 271925.07
## [369,] 243940.153 253951.93 248946.04 204674.642 293217.44 248946.04
## [370,] 258799.817 272438.66 265619.24 221106.295 310132.19 265619.24
## [371,] 122551.701 132380.05 127465.88 83204.756 171727.00 127465.88
## [372,] 249779.389 257944.50 253861.95 209685.424 298038.47 253861.95
## [373,] 243370.681 250453.58 246912.13 202782.325 291041.93 246912.13
## [374,] 246242.467 256692.60 251467.54 207170.819 295764.25 251467.54
## [375,] 220704.050 229162.10 224933.08 180742.778 269123.37 224933.08
## [376,] 249221.101 259385.84 254303.47 210023.357 298583.58 254303.47
## [377,] 174711.629 186875.88 180793.75 136387.786 225199.72 180793.75
## [378,] 206204.455 214169.62 210187.04 166019.643 254354.43 210187.04
## [379,] 180490.394 186885.67 183688.03 139584.487 227791.58 183688.03
## [380,] 183338.894 189676.82 186507.86 142406.380 230609.33 186507.86
## [381,] 227873.527 236113.98 231993.75 187813.733 276173.77 231993.75
## [382,] 107008.654 115367.67 111188.16 67002.576 155373.75 111188.16
## [383,] 172138.795 180524.37 176331.58 132144.738 220518.43 176331.58
## [384,] 188109.899 196247.77 192178.84 148003.571 236354.10 192178.84
## [385,] 223293.546 232478.35 227885.95 183659.395 272112.50 227885.95
## [386,] 167120.759 178172.52 172646.64 128313.433 216979.84 172646.64
## [387,] 117794.194 129496.15 123645.17 79270.270 168020.07 123645.17
## [388,] 151429.475 159089.59 155259.53 111105.631 199413.44 155259.53
## [389,] 63947.399 76163.05 70055.22 25645.729 114464.72 70055.22
## [390,] 135265.977 150023.57 142644.77 98042.704 187246.84 142644.77
## [391,] 97947.591 119098.96 108523.28 63282.335 153764.22 108523.28
## [392,] 141545.224 152883.71 147214.47 102863.162 191565.77 147214.47
## [393,] 182955.629 192669.22 187812.43 143557.639 232067.21 187812.43
## [394,] 265649.839 274908.88 270279.36 226048.935 314509.78 270279.36
## [395,] 68426.104 89207.73 78816.92 33618.833 124015.00 78816.92
## [396,] 218021.961 228513.53 223267.75 178968.580 267566.91 223267.75
## [397,] 233916.921 244632.50 239274.71 194962.142 283587.28 239274.71
## [398,] 224005.862 238279.87 231142.87 186580.162 275705.57 231142.87
## [399,] 173405.215 182536.64 177970.93 133747.139 222194.72 177970.93
## [400,] 113934.782 125826.96 119880.87 75493.330 164268.41 119880.87
## [401,] 105055.656 116131.10 110593.38 66258.695 154928.06 110593.38
## [402,] 263808.886 274792.45 269300.67 224971.698 313629.63 269300.67
## [403,] 264186.537 276128.44 270157.49 225766.609 314548.36 270157.49
## [404,] 155094.670 172048.35 163571.51 118774.694 208368.32 163571.51
## [405,] 50605.897 68372.89 59489.40 14613.853 104364.94 59489.40
## [406,] 193766.772 201589.18 197677.98 153516.962 241838.99 197677.98
## [407,] 185469.818 200002.81 192736.31 148152.686 237319.94 192736.31
## [408,] 64402.025 77385.75 70893.89 26429.948 115357.83 70893.89
## [409,] 173476.766 184223.17 178849.97 134535.533 223164.40 178849.97
## [410,] 205030.668 213738.76 209384.72 165182.278 253587.15 209384.72
## [411,] 134675.593 145198.91 139937.25 95636.206 184238.30 139937.25
## [412,] 144741.509 168994.37 156867.94 111239.571 202496.31 156867.94
## [413,] 114558.410 124361.81 119460.11 75200.372 163719.85 119460.11
## [414,] 241216.717 248992.27 245104.49 200945.545 289263.44 245104.49
## [415,] 178190.781 184782.44 181486.61 137375.839 225597.38 181486.61
## [416,] 138842.241 146150.65 142496.44 98357.447 186635.44 142496.44
## [417,] 136887.407 149499.06 143193.23 98756.072 187630.39 143193.23
## [418,] 260819.749 269557.03 265188.39 220984.511 309392.27 265188.39
## [419,] 126224.108 145070.56 135647.33 90661.837 180632.83 135647.33
## [420,] 223000.304 237968.27 230484.29 185864.699 275103.88 230484.29
## [421,] 102886.101 110077.77 106481.94 62347.735 150616.14 106481.94
## [422,] 127826.778 136325.80 132076.29 87884.027 176268.55 132076.29
## [423,] 207365.764 214833.85 211099.81 166954.130 255245.49 211099.81
## [424,] 197220.066 205735.81 201477.94 157284.869 245671.00 201477.94
## [425,] 272752.411 284530.25 278641.33 234261.411 323021.25 278641.33
## [426,] 13744.141 35436.93 24590.54 -20714.453 69895.53 24590.54
## [427,] 172135.195 178581.72 175358.46 131253.047 219463.87 175358.46
## [428,] 233366.675 241982.09 237674.38 193476.487 281872.28 237674.38
## [429,] 101562.578 119142.38 110352.48 65495.369 155209.58 110352.48
## [430,] 96302.640 106037.78 101170.21 56914.237 145426.18 101170.21
## [431,] 135331.370 145187.74 140259.55 95996.874 184522.23 140259.55
## [432,] 117924.079 127369.03 122646.56 78406.307 166886.80 122646.56
## [433,] 183863.859 198993.80 191428.83 146795.584 236062.07 191428.83
## [434,] 108092.485 144151.24 126121.86 78582.866 173660.86 126121.86
## [435,] 332168.623 343745.56 337957.09 293590.390 382323.79 337957.09
## [436,] 170622.982 184015.96 177319.47 132825.193 221813.75 177319.47
## [437,] 166741.560 174184.69 170463.12 126318.499 214607.75 170463.12
## [438,] 208739.915 217376.22 213058.07 168859.151 257256.98 213058.07
## [439,] 126705.888 139514.32 133110.10 88658.874 177561.33 133110.10
## [440,] 134625.010 143275.52 138950.26 94750.654 183149.88 138950.26
## [441,] 127264.142 136117.07 131690.61 87480.976 175900.24 131690.61
## [442,] 189651.665 198231.35 193941.51 149745.347 238137.67 193941.51
## [443,] 174659.974 181159.82 177909.90 133802.530 222017.26 177909.90
## [444,] 271263.069 279439.20 275351.14 231174.105 319528.17 275351.14
## [445,] 311032.663 321558.48 316295.57 271994.374 360596.77 316295.57
## [446,] 107328.528 118662.80 112995.66 68644.629 157346.70 112995.66
## [447,] 182046.143 190767.40 186406.77 142203.684 230609.86 186406.77
## [448,] 100717.917 111921.31 106319.61 61976.895 150662.33 106319.61
## [449,] 259573.638 270560.21 265066.93 220737.771 309396.08 265066.93
## [450,] 234439.115 243325.93 238882.52 194671.195 283093.85 238882.52
## [451,] 117356.745 127111.36 122234.05 77977.008 166491.10 122234.05
## [452,] 162608.858 173336.86 167972.86 123659.540 212286.18 167972.86
## [453,] 205284.669 220663.79 212974.23 168319.698 257628.76 212974.23
## [454,] 101105.262 108446.36 104775.81 60635.460 148916.17 104775.81
## [455,] 148193.052 158717.74 153455.39 109154.265 197756.52 153455.39
## [456,] 84125.197 99558.94 91842.07 47182.829 136501.31 91842.07
## [457,] 102478.791 113748.14 108113.47 63766.568 152460.36 108113.47
## [458,] 198743.354 206156.94 202450.15 158306.767 246593.53 202450.15
## [459,] 146456.152 156852.45 151654.30 107360.753 195947.85 151654.30
## [460,] 182038.421 190157.59 186098.01 141923.601 230272.41 186098.01
## [461,] 162258.958 173050.64 167654.80 123337.612 211971.98 167654.80
## [462,] 161830.606 173727.10 167778.85 123391.021 212166.68 167778.85
## [463,] 296641.026 307616.58 302128.80 257800.331 346457.28 302128.80
## [464,] 184683.829 198523.05 191603.44 147075.035 236131.84 191603.44
## [465,] 228828.124 239146.31 233987.22 189698.235 278276.20 233987.22
## [466,] 130328.123 138598.38 134463.25 90281.842 178644.66 134463.25
## [467,] 93142.055 106306.07 99724.06 55246.871 144201.26 99724.06
## [468,] 191505.924 200200.36 195853.14 151651.377 240054.91 195853.14
## [469,] 258159.314 266996.90 262578.11 218369.243 306786.97 262578.11
## [470,] 152286.365 161402.38 156844.37 112621.378 201067.37 156844.37
## [471,] 219307.328 228426.33 223866.83 179643.679 268089.98 223866.83
## [472,] 234651.436 241960.80 238306.12 194167.082 282445.15 238306.12
## [473,] 144887.922 160710.41 152799.16 108105.926 197492.40 152799.16
## [474,] 168322.142 178179.31 173250.73 128988.003 217513.45 173250.73
## [475,] 216052.155 224805.14 220428.65 176223.992 264633.30 220428.65
## [476,] 202132.692 210449.20 206290.95 162107.365 250474.53 206290.95
## [477,] 119812.920 131331.29 125572.10 81209.213 169934.99 125572.10
## [478,] 94226.158 109215.80 101720.98 57099.569 146342.39 101720.98
## [479,] 289594.068 303286.21 296440.14 251923.102 340957.17 296440.14
## [480,] 167504.348 184841.22 176172.78 131339.320 221006.25 176172.78
## [481,] 242255.174 251682.85 246969.01 202729.685 291208.34 246969.01
## [482,] 189938.328 197859.04 193898.68 149733.287 238064.08 193898.68
## [483,] 262900.458 272486.15 267693.31 223445.491 311941.12 267693.31
## [484,] 211602.848 220663.45 216133.15 171913.004 260353.30 216133.15
## [485,] 131433.959 137690.97 134562.47 90463.880 178661.05 134562.47
## [486,] 112949.547 123308.94 118129.24 73837.855 162420.63 118129.24
## [487,] 258340.449 268645.73 263493.09 219204.859 307781.32 263493.09
## [488,] 232373.178 244611.23 238492.20 194081.166 282903.24 238492.20
## [489,] 164911.887 173323.42 169117.65 124929.575 213305.73 169117.65
## [490,] 183013.106 189300.28 186156.70 142057.036 230256.35 186156.70
## [491,] 181038.913 191495.57 186267.24 141970.141 230564.34 186267.24
## [492,] 176196.850 186318.96 181257.91 136980.237 225535.58 181257.91
## [493,] 255919.077 264132.81 260025.94 215847.168 304204.72 260025.94
## [494,] 225196.262 232378.39 228787.33 184653.514 272921.14 228787.33
## [495,] 268618.752 292579.90 280599.33 235009.503 326189.15 280599.33
## [496,] 235384.099 241416.81 238400.45 194309.683 282491.23 238400.45
## [497,] 200543.088 207486.68 204014.88 159890.614 248139.15 204014.88
## [498,] 91462.830 101241.90 96352.37 52093.974 140610.76 96352.37
## [499,] 157848.849 165848.37 161848.61 117679.664 206017.56 161848.61
## [500,] 130661.460 139466.54 135064.00 90856.761 179271.24 135064.00
## [501,] 216950.766 223964.53 220457.65 176330.604 264584.69 220457.65
## [502,] 138255.240 148281.93 143268.59 98996.344 187540.83 143268.59
## [503,] 145013.445 154278.62 149646.03 105415.289 193876.78 149646.03
## [504,] 180521.441 190992.27 185756.86 141458.919 230054.80 185756.86
## [505,] 268438.734 275215.58 271827.16 227709.371 315944.95 271827.16
## [506,] 125739.779 134221.12 129980.45 85789.036 174171.86 129980.45
## [507,] 119585.924 134340.79 126963.36 82361.515 171565.20 126963.36
## [508,] 251095.141 261463.07 256279.10 211987.217 300570.99 256279.10
## [509,] 127228.596 140933.12 134080.86 89562.871 178598.85 134080.86
## [510,] 231025.103 241683.76 236354.43 192045.295 280663.57 236354.43
## [511,] 154980.980 162774.16 158877.57 114717.846 203037.29 158877.57
## [512,] 218977.530 231445.04 225211.28 180784.294 269638.27 225211.28
## [513,] 134839.055 143802.62 139320.84 95105.637 183536.04 139320.84
## [514,] 257660.714 269403.79 263532.25 219154.636 307909.87 263532.25
## [515,] 188696.336 197830.89 193263.62 149039.664 237487.57 193263.62
## [516,] 130224.460 139071.07 134647.76 90438.450 178857.08 134647.76
## [517,] 170471.998 176736.87 173604.43 129505.569 217703.30 173604.43
## [518,] 145710.018 160130.69 152920.35 108345.847 197494.86 152920.35
## [519,] 246587.317 256365.17 251476.24 207217.916 295734.57 251476.24
## [520,] 147099.847 156286.47 151693.16 107466.511 195919.80 151693.16
## [521,] 57436.587 72117.60 64777.10 20181.347 109372.85 64777.10
## [522,] 153062.766 165919.86 159491.31 115036.573 203946.05 159491.31
## [523,] 190914.067 197921.85 194417.96 150291.150 238544.76 194417.96
## [524,] 158414.040 166715.61 162564.82 118381.944 206747.70 162564.82
## [525,] 193297.134 201123.19 197210.16 153048.984 241371.34 197210.16
## [526,] 165579.997 175790.46 170685.23 126402.486 214967.97 170685.23
## [527,] 107590.950 118280.51 112935.73 68624.732 157246.73 112935.73
## [528,] 200195.918 209829.97 205012.94 160762.502 249263.38 205012.94
## [529,] 241114.619 248429.69 244772.15 200632.881 288911.43 244772.15
## [530,] 250120.482 259043.47 254581.98 210368.825 298795.13 254581.98
## [531,] 104847.283 113837.68 109342.48 65125.918 153559.05 109342.48
## [532,] 253745.621 261436.25 257590.94 213435.708 301746.17 257590.94
## [533,] 132595.350 140012.78 136304.06 92160.521 180447.61 136304.06
## [534,] 78850.951 90152.08 84501.51 40152.594 128850.43 84501.51
## [535,] 259260.775 272515.30 265888.04 221404.126 310371.95 265888.04
## [536,] 203566.960 214012.25 208789.60 164493.174 253086.04 208789.60
## [537,] 113761.224 125668.28 119714.75 75326.216 164103.29 119714.75
## [538,] 174015.677 190544.54 182280.11 137523.003 227037.22 182280.11
## [539,] 137446.985 146637.80 142042.39 97815.528 186269.26 142042.39
## [540,] 208844.930 217150.64 212997.79 168814.713 257180.86 212997.79
## [541,] 34991.570 50042.15 42516.86 -2109.676 87143.40 42516.86
## [542,] 55045.138 73968.75 64506.95 19513.351 109500.54 64506.95
## [543,] 209047.676 220032.82 214540.25 170211.182 258869.32 214540.25
## [544,] 144291.199 151966.71 148128.95 103974.383 192283.52 148128.95
## [545,] 137172.077 148080.02 142626.05 98301.750 186950.35 142626.05
## [546,] 219434.980 228279.13 223857.05 179647.864 268066.25 223857.05
## [547,] 28565.122 39300.26 33932.69 -10381.060 78246.44 33932.69
## [548,] 9564.137 26421.70 17992.92 -26794.828 62780.66 17992.92
## [549,] 156219.008 164095.49 160157.25 115993.833 204320.67 160157.25
## [550,] 145507.953 157579.77 151543.86 107144.201 195943.53 151543.86
## [551,] 259211.950 266629.40 262920.68 218777.133 307064.22 262920.68
## [552,] 50330.581 67817.17 59073.88 14225.877 103921.87 59073.88
## [553,] 232559.914 242152.67 237356.29 193108.094 281604.49 237356.29
## [554,] 134605.439 149888.94 142247.19 97600.865 186893.51 142247.19
## [555,] 132027.169 143619.66 137823.42 93455.700 182191.13 137823.42
## [556,] 142355.223 149248.79 145802.01 101679.699 189924.32 145802.01
## [557,] 182090.661 190328.40 186209.53 142029.637 230389.42 186209.53
## [558,] 178875.854 185322.10 182098.97 137993.575 226204.37 182098.97
## [559,] 272936.301 282044.34 277490.32 233267.737 321712.90 277490.32
## [560,] 173339.418 181035.41 177187.41 133031.952 221342.88 177187.41
## [561,] 117539.680 127351.34 122445.51 78185.312 166705.70 122445.51
## [562,] 133757.609 145439.56 139598.58 95225.002 183972.17 139598.58
## [563,] 271095.952 280908.15 276002.05 231741.823 320262.27 276002.05
## [564,] 291645.132 301009.73 296327.43 252091.452 340563.41 296327.43
## [565,] 257943.427 267848.43 262895.93 218630.535 307161.32 262895.93
## [566,] 130422.745 138775.22 134598.98 90413.704 178784.26 134598.98
## [567,] 195158.907 206680.52 200919.71 156556.614 245282.81 200919.71
## [568,] 204141.267 225496.34 214818.80 169553.942 260083.66 214818.80
## [569,] 144252.587 156572.45 150412.52 105995.825 194829.21 150412.52
## [570,] 53653.140 67989.19 60821.17 16253.482 105388.85 60821.17
## [571,] 225169.557 232018.92 228594.24 184473.651 272714.83 228594.24
## [572,] 169303.030 181552.44 175427.73 131015.914 219839.55 175427.73
## [573,] 30343.748 52214.11 41278.93 -4047.398 86605.26 41278.93
## [574,] 126581.941 136393.95 131487.94 87227.730 175748.16 131487.94
## [575,] 212743.787 221097.15 216920.47 172735.149 261105.79 216920.47
## [576,] 98310.798 108554.14 103432.47 59147.831 147717.11 103432.47
## [577,] 158136.292 167352.50 162744.40 118516.210 206972.58 162744.40
## [578,] 121173.976 132577.91 126875.94 82520.447 171231.44 126875.94
## [579,] 272558.678 282793.47 277676.08 233391.931 321960.22 277676.08
## [580,] 102267.493 118621.19 110444.34 65703.322 155185.36 110444.34
## [581,] 180281.797 190578.80 185430.30 141142.549 229718.05 185430.30
## [582,] 127095.952 137027.56 132061.75 87794.871 176328.64 132061.75
## [583,] 191065.460 199530.07 195297.76 151107.152 239488.38 195297.76
## [584,] 317759.977 336261.04 327010.51 282060.868 371960.15 327010.51
## [585,] 143840.840 153474.44 148657.64 104407.226 192908.06 148657.64
## [586,] 193927.442 200086.20 197006.82 152911.693 241101.95 197006.82
## [587,] 117418.542 128127.29 122772.92 78460.760 167085.07 122772.92
## [588,] 89329.911 98815.05 94072.48 49830.085 138314.88 94072.48
## [589,] 209996.247 218326.42 214161.33 169977.110 258345.56 214161.33
## [590,] 181414.348 190396.91 185905.63 141689.464 230121.80 185905.63
## [591,] 190962.278 203211.47 197086.87 152675.068 241498.68 197086.87
## [592,] 154740.973 163862.42 159301.70 115078.423 203524.97 159301.70
## [593,] 195819.640 207432.87 201626.26 157257.183 245995.33 201626.26
## [594,] 117605.456 129188.41 123396.93 79029.838 167764.02 123396.93
## [595,] 253144.188 261674.60 257409.40 213215.620 301603.17 257409.40
## [596,] 136645.166 146187.56 141416.36 97170.888 185661.84 141416.36
## [597,] 181975.819 191827.29 186901.56 142639.150 231163.96 186901.56
## [598,] 116111.358 124619.15 120365.25 76172.569 164557.94 120365.25
## [599,] 153012.851 161386.68 157199.76 113013.476 201386.05 157199.76
## [600,] 249984.182 258085.68 254034.93 209861.337 298208.52 254034.93
## [601,] 153948.341 165112.22 159530.28 115190.051 203870.51 159530.28
## [602,] 159011.527 170148.49 164580.01 120241.474 208918.55 164580.01
## [603,] 171855.606 191673.93 181764.77 136674.978 226854.56 181764.77
## [604,] 129180.425 145717.08 137448.75 92690.928 182206.58 137448.75
## [605,] 131682.687 139334.72 135508.70 91355.152 179662.26 135508.70
## [606,] 110159.594 121385.25 115772.42 71428.295 160116.55 115772.42
## [607,] 128617.756 137244.49 132931.12 88732.673 177129.57 132931.12
## [608,] 134870.342 150038.49 142454.42 97817.930 187090.90 142454.42
## [609,] 41603.126 54196.30 47899.72 3463.865 92335.57 47899.72
## [610,] 142786.591 157075.67 149931.13 105367.217 194495.04 149931.13
## [611,] 106169.075 121002.48 113585.78 68977.424 158194.13 113585.78
## [612,] 225493.139 241386.75 233439.94 188740.396 278139.49 233439.94
## [613,] 181205.061 190412.51 185808.79 141581.056 230036.52 185808.79
## [614,] 203880.557 211413.91 207647.24 163498.785 251795.69 207647.24
## [615,] 95451.874 104690.36 100071.12 55841.769 144300.46 100071.12
## [616,] 251788.690 259872.42 255830.56 211657.777 300003.34 255830.56
## [617,] 48682.932 70400.34 59541.64 14233.700 104849.58 59541.64
## [618,] 194354.846 212121.72 203238.28 158362.751 248113.81 203238.28
## [619,] 129526.230 139555.19 134540.71 90268.339 178813.08 134540.71
## [620,] 71306.825 81315.69 76311.26 32040.025 120582.49 76311.26
## [621,] 126219.353 135191.95 130705.65 86489.991 174921.31 130705.65
## [622,] 176274.780 187867.90 182071.34 137703.583 226439.10 182071.34
## [623,] 236310.103 245175.87 240742.99 196532.712 284953.26 240742.99
## [624,] 269229.630 289220.27 279224.95 234116.148 324333.76 279224.95
## [625,] 51459.370 68190.05 59824.71 15048.861 104600.56 59824.71
## [626,] 66889.293 77296.80 72093.05 27798.839 116387.25 72093.05
## [627,] 191047.327 202752.68 196900.00 152524.878 241275.12 196900.00
## [628,] 266919.453 277155.82 272037.64 227753.400 316321.87 272037.64
## [629,] 171626.298 178679.83 175153.07 131024.437 219281.69 175153.07
## [630,] 133393.316 149961.66 141677.49 96916.730 186438.24 141677.49
## [631,] 152308.371 160168.69 156238.53 112075.830 200401.23 156238.53
## [632,] 249840.204 314230.74 282035.47 227524.619 336546.33 282035.47
## [633,] 260222.898 272652.64 266437.77 222013.423 310862.11 266437.77
## [634,] 306596.474 316361.63 311479.05 267221.425 355736.67 311479.05
## [635,] 138072.894 148766.55 143419.72 99108.476 187730.96 143419.72
## [636,] 191129.415 198953.21 195041.31 150880.236 239202.39 195041.31
## [637,] 127012.011 135728.41 131370.21 87167.363 175573.06 131370.21
## [638,] 167788.091 175286.79 171537.44 127390.466 215684.42 171537.44
## [639,] 115488.179 127739.80 121613.99 77202.016 166025.96 121613.99
## [640,] 166880.089 176146.68 171513.38 127282.566 215744.20 171513.38
## [641,] 236486.700 244741.18 240613.94 196433.267 284794.61 240613.94
## [642,] 143897.926 153904.89 148901.41 104630.283 193172.54 148901.41
## [643,] 156792.858 166198.91 161495.89 117257.709 205734.06 161495.89
## [644,] 112238.235 121223.37 116730.80 72514.507 160947.10 116730.80
## [645,] 133986.572 144610.17 139298.37 94991.339 183605.40 139298.37
## [646,] 141905.786 153155.98 147530.88 103185.201 191876.56 147530.88
## [647,] 168359.758 179037.42 173698.59 129388.309 218008.87 173698.59
## [648,] 190762.921 197378.26 194070.59 149958.932 238182.25 194070.59
## [649,] 156789.453 167603.52 162196.49 117877.939 206515.04 162196.49
## [650,] 155140.588 163159.85 159150.22 114980.376 203320.06 159150.22
## [651,] 68356.973 80040.96 74198.97 29825.250 118572.68 74198.97
## [652,] 288518.443 298542.68 293530.56 249258.457 337802.66 293530.56
## [653,] 256833.416 267426.41 262129.91 217824.712 306435.11 262129.91
## [654,] 156446.778 166815.50 161631.14 117339.205 205923.07 161631.14
## [655,] 185008.948 192764.01 188886.48 144728.431 233044.52 188886.48
## [656,] 106636.179 114863.44 110749.81 66570.406 154929.21 110749.81
## [657,] 167168.911 180239.56 173704.23 129233.927 218174.54 173704.23
## [658,] 189465.931 198328.40 193897.16 149687.056 238107.27 193897.16
## [659,] 91267.269 99753.39 95510.33 51318.687 139701.97 95510.33
## [660,] 172027.608 184305.61 178166.61 133752.814 222580.40 178166.61
## [661,] 152337.532 162167.62 157252.58 112991.358 201513.79 157252.58
## [662,] 202529.547 212335.46 207432.51 163172.628 251692.38 207432.51
## [663,] 122189.692 131749.63 126969.66 82723.239 171216.08 126969.66
## [664,] 105593.538 116936.81 111265.17 66913.564 155616.78 111265.17
## [665,] 276413.659 296455.00 286434.33 241319.901 331548.76 286434.33
## [666,] 150618.269 159401.68 155009.97 110803.811 199216.14 155009.97
## [667,] 144746.668 154591.39 149669.03 105406.997 193931.06 149669.03
## [668,] 93291.302 107879.67 100585.49 55997.339 145173.63 100585.49
## [669,] 176112.411 193393.60 184753.00 139924.914 229581.09 184753.00
## [670,] 128438.041 137695.68 133066.86 88836.509 177297.21 133066.86
## [671,] 193888.370 202977.00 198432.69 154211.102 242654.27 198432.69
## [672,] 113130.817 123995.44 118563.13 74241.490 162884.77 118563.13
## [673,] 152078.644 170826.98 161452.81 116477.566 206428.06 161452.81
## [674,] 139194.462 152014.33 145604.40 101152.343 190056.45 145604.40
## [675,] 80516.532 98209.78 89363.16 44494.898 134231.41 89363.16
## [676,] 137035.352 145421.65 141228.50 97041.623 185415.38 141228.50
## [677,] 112691.971 122365.09 117528.53 73275.959 161781.10 117528.53
## [678,] 245942.829 255132.66 250537.74 206310.930 294764.56 250537.74
## [679,] 223251.797 231687.98 227469.89 183280.634 271659.14 227469.89
## [680,] 133582.210 141942.68 137762.44 93576.788 181948.10 137762.44
## [681,] 118986.960 134150.36 126568.66 81932.575 171204.74 126568.66
## [682,] 134215.777 144142.07 139178.92 94912.337 183445.51 139178.92
## [683,] 306949.241 319096.20 313022.72 268617.936 357427.51 313022.72
## [684,] 104267.612 112411.01 108339.31 64163.790 152514.83 108339.31
## [685,] 165953.409 174498.69 170226.05 126031.557 214420.54 170226.05
## [686,] 95074.350 110842.27 102958.31 58269.892 147646.73 102958.31
## [687,] 184591.579 191724.18 188157.88 144026.075 232289.69 188157.88
## [688,] 102872.919 110355.03 106613.98 62467.704 150760.25 106613.98
## [689,] 128449.184 137310.52 132879.85 88669.802 177089.90 132879.85
## [690,] 172455.056 181596.68 177025.87 132801.552 221250.18 177025.87
## [691,] 120049.859 130449.39 125249.62 80955.883 169543.36 125249.62
## [692,] 259979.890 268628.22 264304.06 220104.552 308503.56 264304.06
## [693,] 240472.628 250645.42 245559.03 201278.452 289839.60 245559.03
## [694,] 295185.334 304647.44 299916.39 255675.221 344157.55 299916.39
## [695,] 151038.990 158622.36 154830.68 110680.086 198981.27 154830.68
## [696,] 106245.056 119470.80 112857.93 68376.159 157339.70 112857.93
## [697,] 202499.511 208715.75 205607.63 161510.484 249704.77 205607.63
## [698,] 149370.116 163354.68 156362.40 111822.641 200902.15 156362.40
## [699,] 114553.708 122759.29 118656.50 74478.102 162834.89 118656.50
## [700,] 201802.185 213781.31 207791.75 163398.361 252185.13 207791.75
## [701,] 166540.988 184823.07 175682.03 130754.795 220609.26 175682.03
## [702,] 197677.268 204777.68 201227.47 157096.966 245357.98 201227.47
## [703,] 201423.328 209148.54 205285.94 161129.198 249442.67 205285.94
## [704,] 180934.632 190540.71 185737.67 141488.750 229986.59 185737.67
## [705,] 213670.325 221251.22 217460.77 173310.288 261611.26 217460.77
## [706,] 172096.909 179118.85 175607.88 131480.511 219735.25 175607.88
## [707,] 199636.635 218460.37 209048.51 164065.385 254031.62 209048.51
## [708,] 128959.043 137790.30 133374.67 89166.125 177583.22 133374.67
## [709,] 240745.920 250353.39 245549.66 201300.660 289798.65 245549.66
## [710,] 198412.302 204807.40 201609.85 157506.312 245713.39 201609.85
## [711,] 114588.632 123713.93 119151.28 74927.810 163374.76 119151.28
## [712,] 155123.150 172735.02 163929.08 119068.830 208789.34 163929.08
## [713,] 52866.021 63006.46 57936.24 13657.521 102214.96 57936.24
## [714,] 163438.667 173012.08 168225.37 123978.224 212472.53 168225.37
## [715,] 55609.218 76055.91 65832.56 20672.685 110992.44 65832.56
## [716,] 164245.276 174536.21 169390.74 125103.346 213678.14 169390.74
## [717,] 51696.783 67098.97 59397.88 14741.357 104054.39 59397.88
## [718,] 133090.298 142139.36 137614.83 93395.275 181834.39 137614.83
## [719,] 235745.668 244513.52 240129.59 195924.203 284334.98 240129.59
## [720,] 162535.287 173404.61 167969.95 123648.020 212291.88 167969.95
## [721,] 175981.152 186243.87 181112.51 136826.750 225398.27 181112.51
## [722,] 175510.913 183642.32 179576.62 135401.649 223751.58 179576.62
## [723,] 167900.306 178965.04 173432.67 129098.659 217766.69 173432.67
## [724,] 390030.476 406472.80 398251.64 353502.501 443000.78 398251.64
## [725,] 238305.505 245564.90 241935.20 197798.228 286072.18 241935.20
## [726,] 160117.072 167495.09 163806.08 119664.190 207947.97 163806.08
## [727,] 96792.901 106744.82 101768.86 57500.835 146036.88 101768.86
## [728,] 137278.020 145716.17 141497.10 97307.749 185686.44 141497.10
## [729,] 167839.181 177019.21 172429.20 128202.892 216655.50 172429.20
## [730,] 298697.481 309224.71 303961.10 259659.817 348262.38 303961.10
## [731,] 184371.628 190702.78 187537.20 143435.972 231638.44 187537.20
## [732,] 275014.807 285367.11 280190.96 235899.987 324481.93 280190.96
## [733,] 201395.129 208137.77 204766.45 160649.973 248882.93 204766.45
## [734,] 161706.714 170228.40 165967.56 121774.205 210160.91 165967.56
## [735,] 249473.326 258423.70 253948.51 209733.977 298163.05 253948.51
## [736,] 129260.429 147427.98 138344.21 93428.587 183259.82 138344.21
## [737,] 146876.511 157318.63 152097.57 107801.328 196393.82 152097.57
## [738,] 104622.247 115744.18 110183.21 65845.619 154520.80 110183.21
## [739,] 121673.217 129998.14 125835.68 81651.703 170019.66 125835.68
## [740,] 73918.870 87773.51 80846.19 36316.586 125375.79 80846.19
## [741,] 121265.138 129546.04 125405.59 81223.680 169587.50 125405.59
## [742,] 224692.367 233100.73 228896.55 184708.621 273084.48 228896.55
## [743,] 106691.776 115715.59 111203.69 66985.419 155421.95 111203.69
## [744,] 105961.217 117787.25 111874.23 67491.112 156257.36 111874.23
## [745,] 145621.209 156037.12 150829.16 106534.462 195123.87 150829.16
## [746,] 245780.517 255526.30 250653.41 206396.850 294909.96 250653.41
## [747,] 120023.291 129712.03 124867.66 80614.237 169121.09 124867.66
## [748,] 76940.168 88386.84 82663.50 38305.252 127021.75 82663.50
## [749,] 134837.437 145337.64 140087.54 95787.861 184387.21 140087.54
## [750,] 270274.282 278923.87 274599.08 230399.509 318798.64 274599.08
## [751,] 33778.069 50392.09 42085.08 -2679.908 86850.07 42085.08
## [752,] 106269.373 119159.61 112714.49 68257.352 157171.63 112714.49
## [753,] 165834.612 172581.02 169207.81 125091.193 213324.44 169207.81
## [754,] 103703.891 119901.23 111802.56 67075.764 156529.35 111802.56
## [755,] 275098.554 283863.02 279480.79 235275.565 323686.01 279480.79
## [756,] 103173.696 112819.81 107996.75 63745.654 152247.85 107996.75
## [757,] 169538.518 176807.09 173172.81 129035.453 217310.16 173172.81
## [758,] 138867.198 150450.16 144658.68 100291.584 189025.77 144658.68
## [759,] 210375.611 218476.67 214426.14 170252.568 258599.71 214426.14
## [760,] 143892.839 154509.92 149201.38 104894.741 193508.02 149201.38
## [761,] 95286.688 105851.28 100568.99 56265.482 144872.49 100568.99
## [762,] 258573.705 278369.05 268471.38 223384.108 313558.64 268471.38
## [763,] 164728.914 173428.01 169078.46 124876.465 213280.45 169078.46
## [764,] 190957.636 199912.46 195435.05 151220.290 239649.81 195435.05
## [765,] 115772.439 125703.44 120737.94 76471.092 165004.79 120737.94
## [766,] 264589.334 272091.08 268340.21 224193.104 312487.31 268340.21
## [767,] 166609.553 178390.48 172500.02 128119.893 216880.14 172500.02
## [768,] 109869.898 123362.59 116616.24 72114.436 161118.05 116616.24
## [769,] 204969.465 213313.07 209141.27 164956.411 253326.13 209141.27
## [770,] 138056.243 147142.83 142599.54 98378.058 186821.02 142599.54
## [771,] 76336.245 93910.94 85123.59 40266.985 129980.20 85123.59
## [772,] 159159.949 168916.84 164038.39 119781.223 208295.56 164038.39
## [773,] 215061.255 223389.96 219225.61 175041.450 263409.76 219225.61
## [774,] 169298.472 179235.75 174267.11 129999.907 218534.31 174267.11
## [775,] 123111.134 133651.32 128381.23 84079.178 172683.28 128381.23
## [776,] 154686.441 171750.69 163218.57 118411.257 208025.88 163218.57
## [777,] 228154.782 238431.66 233293.22 189006.641 277579.80 233293.22
## [778,] 91909.606 100814.02 96361.81 52149.597 140574.03 96361.81
## [779,] 222312.523 229483.43 225897.97 181764.617 270031.33 225897.97
## [780,] 207185.413 215041.88 211113.65 166951.119 255276.17 211113.65
## [781,] 131664.826 149885.23 140775.03 95854.057 185696.00 140775.03
## [782,] 194513.358 202411.21 198462.29 154297.914 242626.66 198462.29
## [783,] 192938.980 201377.25 197158.11 152968.762 241347.47 197158.11
## [784,] 104309.649 114791.86 109550.75 65252.143 153849.37 109550.75
## [785,] 93062.228 107619.30 100340.77 55755.176 144926.36 100340.77
## [786,] 278490.006 286831.54 282660.77 238476.010 326845.53 282660.77
## [787,] 192623.811 199662.30 196143.06 152015.029 240271.09 196143.06
## [788,] 261562.446 274755.98 268159.21 223679.833 312638.59 268159.21
## [789,] 124146.831 132505.30 128326.07 84140.507 172511.63 128326.07
## [790,] 124743.862 147739.85 136241.86 90776.467 181707.25 136241.86
## [791,] 180521.441 190992.27 185756.86 141458.919 230054.80 185756.86
## [792,] 152947.316 168136.54 160541.93 115903.650 205180.21 160541.93
## [793,] 239322.235 246903.93 243113.08 198962.563 287263.60 243113.08
## [794,] 186992.642 193648.17 190320.40 146207.234 234433.57 190320.40
## [795,] 167704.717 174678.53 171191.62 127066.161 215317.08 171191.62
## [796,] 80485.608 93589.11 87037.36 42564.633 131510.08 87037.36
## [797,] 140585.030 148480.13 144532.58 100368.331 188696.83 144532.58
## [798,] 93593.797 109187.00 101390.40 56717.309 146063.48 101390.40
## [799,] 236782.742 245430.38 241106.56 196907.089 285306.03 241106.56
## [800,] 144061.870 152489.39 148275.63 104086.791 192464.47 148275.63
## [801,] 143421.650 150154.74 146788.20 102672.083 190904.31 146788.20
## [802,] 148404.215 158658.45 153531.33 109246.064 197816.60 153531.33
## [803,] 188798.767 199794.19 194296.48 149966.775 238626.18 194296.48
## [804,] 224558.140 232909.67 228733.91 184548.672 272919.14 228733.91
## [805,] 192905.819 200317.94 196611.88 152468.558 240755.20 196611.88
## [806,] 94861.791 105463.10 100162.44 55856.748 144468.14 100162.44
## [807,] 247677.759 257413.35 252545.56 208289.560 296801.55 252545.56
## [808,] 237016.824 247082.66 242049.74 197775.277 286324.20 242049.74
## [809,] 138255.240 148281.93 143268.59 98996.344 187540.83 143268.59
## [810,] 74688.183 85685.79 80186.99 35857.148 124516.83 80186.99
## [811,] 293376.416 302745.41 298060.91 253824.704 342297.12 298060.91
## [812,] 212874.971 219112.92 215993.95 171896.034 260091.86 215993.95
## [813,] 162734.654 172324.79 167529.72 123281.667 211777.78 167529.72plot(Final.conf.fit)
## NULLdiagnostics(Final.ZS, type="model", pch=16)
The RMSE for final model has been computed and compared.
predict.Final <- exp(predict(Final.lm, train))
resid.Final <- train$price - predict.Final
rmse.Final <- sqrt(mean(resid.Final^2))
rmse.Final## [1] 22160.99The RMSE of the final model is lower as compared to BIC/AIC but higher than model with transformed variables. Initial model has lowest rsme.
Section 4.3 Final Model Evaluation
What are some strengths and weaknesses of your model?
The final model contains only 8 variables giving largest Adjusted R-Squared,least possible RMSE and AIC/BIC lowest. The residuals are minimum and outliers still exist in the model as it appears in plots. There is no one magic procedure to remove all the overfitings and underfittings.By using ‘BAS’ with “BIC”as prior model is taking care overfiting and it seems to me the best model selection.
Final Model Validation
validata=validation
val.ZS = bas.lm(log(price) ~ log(Garage.Area + 1) +
log(Total.Bsmt.SF + 1) + Garage.Cars + log(area) +
Full.Bath + Half.Bath +
Bedroom.AbvGr + Year.Built + log(X1st.Flr.SF) +
log(X2nd.Flr.SF + 1) +
log(Lot.Area) + Overall.Cond+Overall.Qual,
data =na.omit(validata),prior="ZS-null",
modelprior=uniform(), initprobs="eplogp") ## Warning in bas.lm(log(price) ~ log(Garage.Area + 1) + log(Total.Bsmt.SF
## + : We recommend using the implementation using the Jeffreys-Zellner-Siow
## prior (prior='JZS') which uses numerical integration rahter than the Laplace
## approximationplot(val.ZS, ask=F) 
0.0.0.15.13 Concluding Assessments
library(BMS)
vars1 <- names(train) %in% c("price","area","Lot.Area","Overall.Qual", "Overall.Cond","Year.Built","Total.Bsmt.SF", "Bedroom.AbvGr","X2nd.Flr.SF")
traindata=train[vars1]
vars2 <- names(test) %in% c("price","area","Lot.Area","Overall.Qual", "Overall.Cond","Year.Built","Total.Bsmt.SF", "Bedroom.AbvGr","X2nd.Flr.SF")
testdata=test[vars2]
vars3 <- names(validation) %in% c("price","area","Lot.Area","Overall.Qual", "Overall.Cond","Year.Built","Total.Bsmt.SF", "Bedroom.AbvGr","X2nd.Flr.SF")
validata=validation[vars3]
mflstr = bms(traindata, burn=100000, iter=200000, g="BRIC", mprior="uniform", nmodel=2000, mcmc="bd", user.int=FALSE)
mflste = bms(testdata, burn=100000, iter=200000, g="BRIC", mprior="uniform", nmodel=2000, mcmc="bd", user.int=FALSE)
mflsval = bms(validata, burn=100000, iter=200000, g="BRIC", mprior="uniform", nmodel=2000, mcmc="bd", user.int=FALSE)
print(mflstr);(mflste);(mflsval)## PIP Post Mean Post SD Cond.Pos.Sign Idx
## price 1.000000 3.555203e-03 0.0001638236 1 1
## Overall.Cond 1.000000 -3.847965e+01 5.9079071730 0 4
## Year.Built 1.000000 -3.007891e+00 0.2703954138 0 5
## Total.Bsmt.SF 1.000000 2.459073e-01 0.0257142346 1 6
## X2nd.Flr.SF 1.000000 5.544106e-01 0.0215860276 1 7
## Bedroom.AbvGr 1.000000 1.185450e+02 9.0468382871 1 8
## Lot.Area 0.035385 -6.852755e-06 0.0001207770 0 2
## Overall.Qual 0.033065 -2.930775e-02 1.4092502398 0 3
##
## Mean no. regressors Draws Burnins Time
## "6.0685" "2e+05" "1e+05" "16.17693 secs"
## No. models visited Modelspace 2^K % visited % Topmodels
## "5259" "256" "2054" "100"
## Corr PMP No. Obs. Model Prior g-Prior
## "1.0000" "832" "uniform / 4" "BRIC"
## Shrinkage-Stats
## "Av=0.9988"## PIP Post Mean Post SD Cond.Pos.Sign Idx
## price 1.0000 3.935761e-03 0.0001823628 1 1
## Overall.Cond 1.0000 -4.054045e+01 6.3281176895 0 4
## Year.Built 1.0000 -3.048471e+00 0.2938322083 0 5
## Total.Bsmt.SF 1.0000 2.031323e-01 0.0266269906 1 6
## X2nd.Flr.SF 1.0000 5.088599e-01 0.0226250018 1 7
## Bedroom.AbvGr 1.0000 1.048275e+02 9.6798845628 1 8
## Lot.Area 0.0997 1.264265e-04 0.0004627588 1 2
## Overall.Qual 0.0558 -4.783492e-01 2.7857077220 0 3
##
## Mean no. regressors Draws Burnins Time
## "6.1555" "2e+05" "1e+05" "10.98863 secs"
## No. models visited Modelspace 2^K % visited % Topmodels
## "11842" "256" "4626" "100"
## Corr PMP No. Obs. Model Prior g-Prior
## "1.0000" "813" "uniform / 4" "BRIC"
## Shrinkage-Stats
## "Av=0.9988"## PIP Post Mean Post SD Cond.Pos.Sign Idx
## price 1.00000 0.003638346 2.456909e-04 1 1
## Lot.Area 1.00000 0.010065558 1.551229e-03 1 2
## Overall.Cond 1.00000 -42.799177464 6.918678e+00 0 4
## Year.Built 1.00000 -2.890698410 3.304465e-01 0 5
## Total.Bsmt.SF 1.00000 0.198006978 2.645506e-02 1 6
## X2nd.Flr.SF 1.00000 0.546395956 2.558911e-02 1 7
## Bedroom.AbvGr 1.00000 107.557779741 1.038400e+01 1 8
## Overall.Qual 0.27016 -5.313183568 9.940035e+00 0 3
##
## Mean no. regressors Draws Burnins Time
## "7.2702" "2e+05" "1e+05" "11.38665 secs"
## No. models visited Modelspace 2^K % visited % Topmodels
## "19999" "256" "7812" "100"
## Corr PMP No. Obs. Model Prior g-Prior
## "1.0000" "762" "uniform / 4" "BRIC"
## Shrinkage-Stats
## "Av=0.9987"–
The final model evaluation states a quite nice picture as you can see from the table above. The Root Mean Squared Error (RMSE), which is at around 21’810 Dollar, while calculating the RMSE based on training data. The RMSE slightly drops With the test data (Dollar - 18’630), but using the real-life (validation) data, our model shows slight rise of the RMSE (Dollar - 18’817). In general, the lower the RMSE, the better the model fit. Therefore, our final model is a better fitted for validation data.
Under- and Overvalued houses
Which houses will be Under- and Overvalued by our model. To answer this question we need to have a look at the Residuals vs Fitted plot. The plot can be separated into 2 areas - lower 25% and medium to higher (75%) sections. Houses in the lower price segment (25%) will slightly be over valued. This is obvious, since we have a slight non linearity in this area. Housing prices in the mid and high range should be pretty precise, since we have almost a straight regression line.
– Part 5 Conclusion
The data set at hand was used to gather deep insight on the key characteristics that can influence sales prices of a homes in Ames, Iowa. The stepwise approach in analysing data thoroughly, deriving information from it towards data transformation to reach a decent result at the end, is quite a journey.
Feature engineering, cleaning and recoding of the data proofed to be one of the major tasks on the route to select the best model. I also experienced that this is one of the most time consuming parts in designing a model. The Model selection process by using multiple approaches gave me a deep insight in the diverse selection methods.
To me it was helpful to start with an initial model to get a feeling, respectively get a reference point to start with. Constantly measuring the improvement of the model via adjusted R22, Root Mean Squared Error or visual representations as Residual vs. Fitted plots is vital to understand the behaviour of each change on the model.
Key take away: Fetting the best model wasn’t primarily driven by the model I used. It was rather driven by managing outliners, high leverage points, feature engineering and transformation. In those areas lies the real potential to get a good model. Following a stepwise approach (improving and constantly measuring) by using the mentioned approaches in cooperation with the Bayesian Model Averaging approach, we were able to find a robust model.