LBB RM
Adrian Djaya Prathama
2024-02-19
1 Data Preparation
1.1 Load Library
library(dplyr)
library(GGally)
library(MLmetrics)
library(tidyverse)
library(car)
library(lmtest)
library(inspectdf)
library(performance)
options(scipen = 100, max.print = 1e+08)1.2 Read Data
Read and save the data it into house data frame
house <- read.csv("data_input/USA_Housing.csv")Checking data structure
glimpse(house)## Rows: 5,000
## Columns: 7
## $ Avg..Area.Income <dbl> 79545.46, 79248.64, 61287.07, 63345.24, 5…
## $ Avg..Area.House.Age <dbl> 5.682861, 6.002900, 5.865890, 7.188236, 5…
## $ Avg..Area.Number.of.Rooms <dbl> 7.009188, 6.730821, 8.512727, 5.586729, 7…
## $ Avg..Area.Number.of.Bedrooms <dbl> 4.09, 3.09, 5.13, 3.26, 4.23, 4.04, 3.41,…
## $ Area.Population <dbl> 23086.80, 40173.07, 36882.16, 34310.24, 2…
## $ Price <dbl> 1059033.6, 1505890.9, 1058988.0, 1260616.…
## $ Address <chr> "208 Michael Ferry Apt. 674\nLaurabury, N…Deleting unused columns
house_clean <- house %>% select(-c(Address))2 Data Exploration
Checking is any missing values on the data
colSums(is.na(house_clean))## Avg..Area.Income Avg..Area.House.Age
## 0 0
## Avg..Area.Number.of.Rooms Avg..Area.Number.of.Bedrooms
## 0 0
## Area.Population Price
## 0 0No missing value on the data
Checking the correlation between Price as the target
with all the other variables
ggcorr(house_clean, label = T, label_size = 2.9, hjust = 1, layout.exp = 2)
From the graph above we could see that Avg..Area.Income
has the strongest correlation with Price
3 Modeling
3.1 Creating Machine Learning Model
Because Price has the strongest correlation with
Avg..Area.Income, so we will create the model with
Price as the target and Avg..Area.Income as
the predictor variable.
house_lm <- lm(Price ~ Avg..Area.Income,
data = house_clean)
summary(house_lm)##
## Call:
## lm(formula = Price ~ Avg..Area.Income, data = house_clean)
##
## Residuals:
## Min 1Q Median 3Q Max
## -965112 -187163 -2365 183084 983680
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -221579.4782 25000.1306 -8.863 <0.0000000000000002 ***
## Avg..Area.Income 21.1955 0.3602 58.844 <0.0000000000000002 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 271400 on 4998 degrees of freedom
## Multiple R-squared: 0.4093, Adjusted R-squared: 0.4091
## F-statistic: 3463 on 1 and 4998 DF, p-value: < 0.00000000000000022From the summary above, we can see the Adjusted R-squared is 0.4091
which means 40.91% of the price was explained by
Avg..Area.Income and the rest from other variables.
Next we will create a model using Price as the target
with all the variables in the data frame as the predictor
house_lm1 <- lm(Price ~ .,
data = house_clean)
summary(house_lm1)##
## Call:
## lm(formula = Price ~ ., data = house_clean)
##
## Residuals:
## Min 1Q Median 3Q Max
## -337020 -70236 320 69175 361870
##
## Coefficients:
## Estimate Std. Error t value
## (Intercept) -2637299.0333 17157.8092 -153.708
## Avg..Area.Income 21.5780 0.1343 160.656
## Avg..Area.House.Age 165637.0269 1443.4130 114.754
## Avg..Area.Number.of.Rooms 120659.9488 1605.1604 75.170
## Avg..Area.Number.of.Bedrooms 1651.1391 1308.6712 1.262
## Area.Population 15.2007 0.1442 105.393
## Pr(>|t|)
## (Intercept) <0.0000000000000002 ***
## Avg..Area.Income <0.0000000000000002 ***
## Avg..Area.House.Age <0.0000000000000002 ***
## Avg..Area.Number.of.Rooms <0.0000000000000002 ***
## Avg..Area.Number.of.Bedrooms 0.207
## Area.Population <0.0000000000000002 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 101200 on 4994 degrees of freedom
## Multiple R-squared: 0.918, Adjusted R-squared: 0.9179
## F-statistic: 1.119e+04 on 5 and 4994 DF, p-value: < 0.00000000000000022From the summary, we can see there is 1 variables that doesn’t
significantly impact the house price. So next we will use Step-Wise
Regression with direction backward to create a new model
with the best variables.
house_back <- step(house_lm1, direction = "backward")## Start: AIC=115249.9
## Price ~ Avg..Area.Income + Avg..Area.House.Age + Avg..Area.Number.of.Rooms +
## Avg..Area.Number.of.Bedrooms + Area.Population
##
## Df Sum of Sq RSS AIC
## - Avg..Area.Number.of.Bedrooms 1 16287981430 51114959547695 115250
## <none> 51098671566265 115250
## - Avg..Area.Number.of.Rooms 1 57816321293270 108914992859535 119032
## - Area.Population 1 113653120691812 164751792258077 121101
## - Avg..Area.House.Age 1 134739428405496 185838099971761 121704
## - Avg..Area.Income 1 264092032671228 315190704237493 124345
##
## Step: AIC=115249.5
## Price ~ Avg..Area.Income + Avg..Area.House.Age + Avg..Area.Number.of.Rooms +
## Area.Population
##
## Df Sum of Sq RSS AIC
## <none> 51114959547695 115250
## - Avg..Area.Number.of.Rooms 1 74764641482450 125879601030145 119754
## - Area.Population 1 113657374142563 164772333690258 121100
## - Avg..Area.House.Age 1 134791004859689 185905964407384 121703
## - Avg..Area.Income 1 264409832482435 315524792030129 124348summary(house_back)##
## Call:
## lm(formula = Price ~ Avg..Area.Income + Avg..Area.House.Age +
## Avg..Area.Number.of.Rooms + Area.Population, data = house_clean)
##
## Residuals:
## Min 1Q Median 3Q Max
## -338419 -70058 132 69074 362025
##
## Coefficients:
## Estimate Std. Error t value
## (Intercept) -2637560.6725 17157.5724 -153.73
## Avg..Area.Income 21.5827 0.1343 160.74
## Avg..Area.House.Age 165657.8724 1443.4039 114.77
## Avg..Area.Number.of.Rooms 121598.1646 1422.6077 85.48
## Area.Population 15.1961 0.1442 105.39
## Pr(>|t|)
## (Intercept) <0.0000000000000002 ***
## Avg..Area.Income <0.0000000000000002 ***
## Avg..Area.House.Age <0.0000000000000002 ***
## Avg..Area.Number.of.Rooms <0.0000000000000002 ***
## Area.Population <0.0000000000000002 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 101200 on 4995 degrees of freedom
## Multiple R-squared: 0.918, Adjusted R-squared: 0.9179
## F-statistic: 1.398e+04 on 4 and 4995 DF, p-value: < 0.00000000000000022From the summary above, we can see that the Adjusted R-squared both
have the same numbers between before and
afterStep-Wise Regression. Both have 0.9179 which means
that both model was 91.79% explained by the variables.
Based on 3 models that we created, the best model that we could use is the model with all variables and the step-wise model.
4 Model Evaluation
4.1 Mean Absolute Percentage Error
MAPE for model with Price target with all variables
MAPE(y_pred = house_clean$Price,
y_true = predict(house_lm1, house_clean)) *100## [1] 7.390759MAPE for step-wise model
MAPE(y_pred = house_clean$Price,
y_true = predict(house_back, house_clean)) *100## [1] 7.387608From calculation above, step-wise model is a better model by 0.003 points. This means both models are good enough to be used because both models value number is close to 0.
4.2 Normality Test
Normality test with all Variables
hist(house_lm1$residuals)
shapiro.test(house_lm1$residuals)##
## Shapiro-Wilk normality test
##
## data: house_lm1$residuals
## W = 0.99955, p-value = 0.3056Normality test with Step-Wise model
hist(house_back$residuals)
shapiro.test(house_back$residuals)##
## Shapiro-Wilk normality test
##
## data: house_back$residuals
## W = 0.99957, p-value = 0.341Based on the Shapiro test above, both models has p-value > alpha . Which means H0 is accepted, residual distributed normally for both models.
4.3 Homoscedasticity
Homoscedasticity test with all Variable
plot(house_clean$Price, house_lm1$residuals)
abline(h = 0, col = "red")
bptest(house_lm1)##
## studentized Breusch-Pagan test
##
## data: house_lm1
## BP = 10.314, df = 5, p-value = 0.0668Homoscedasticity test with Step-Wise model
plot(house_clean$Price, house_back$residuals)
abline(h = 0, col = "red")
bptest(house_back)##
## studentized Breusch-Pagan test
##
## data: house_back
## BP = 10.385, df = 4, p-value = 0.03441Based on Homoscedasticity test above, p-value for model with all variables > alpha (0.05), which means H0 is accepted. Data error spread constantly (Homoscedasticity). But p-value for Step-Wise model < alpha (0.05), which means H1 is accepted. Data error spread creating a pattern (Heteroscedasticity)
4.4 Multicolinearity
Multicolinearity test model with all variables
vif(house_lm1)## Avg..Area.Income Avg..Area.House.Age
## 1.001159 1.000577
## Avg..Area.Number.of.Rooms Avg..Area.Number.of.Bedrooms
## 1.273535 1.274413
## Area.Population
## 1.001266Multicolinearity test with Step-Wise model
vif(house_back)## Avg..Area.Income Avg..Area.House.Age Avg..Area.Number.of.Rooms
## 1.000390 1.000445 1.000214
## Area.Population
## 1.000619Based on Multicolinearity test, both models variable values is less than 10. Which means all the variables are independent and there is no correlation between each of the variables from both model.
5 Conclusion
Based on the test above, model with all variable is better and
useable than Step-Wise model. Although
Avg..Area.Number.of.Bedrooms variable doesn’t have any
significant impact to the intercept, but from this case it could make a
huge different based on the Homoscedasticity test.