This data set involves cars and various information about them. The data set car_data includes the variables: name, year, selling price, present price, kilometers driven, fuel type, seller type, transmission, and owner. The total sample size is 301. This analysis seeks to run a multi-linear regression, analyze conditions, transform the response variable with a box-cox, perform a goodness of fit test, build candidate models and select a final model. The final model will then be Bootstrapped and compared. In this particular instance the response variable will be selling price and the predictor variables are year, Present Price, and kilometers.
Below the full model is provided, along with a residual analysis along with discretizing Fuel Type.
library(kableExtra)
car_data <- read.csv("https://raw.githubusercontent.com/Kyle-Weber/STA321/main/www/car%20data.csv", header = TRUE)
car <- car_data[, -1]
price <- car$Selling_Price
year <- car$Year
#Discretizing continuous variable Fuel_Type
data <- data.frame(
Fuel_Type = c("Petrol", "Diesel", "Petrol", "Petrol", "Diesel")
)
# create a new variable based on the Fuel_Type variable
data$Fuel_Type_Discrete <- ifelse(data$Fuel_Type == "Diesel", 1, 0)
#Printing out full model and residual analysis
full.model = lm(price ~ ., data = car)
kable(summary(full.model)$coef, caption ="Statistics of Regression Coefficients")
## Warning in summary.lm(full.model): essentially perfect fit: summary may be
## unreliable
| Estimate | Std. Error | t value | Pr(>|t|) | |
|---|---|---|---|---|
| (Intercept) | 0 | 0 | -3.363786e+00 | 0.0008719 |
| Year | 0 | 0 | 3.384092e+00 | 0.0008122 |
| Selling_Price | 1 | 0 | 3.969546e+16 | 0.0000000 |
| Present_Price | 0 | 0 | -1.571441e+00 | 0.1171669 |
| Kms_Driven | 0 | 0 | 7.070469e-01 | 0.4801031 |
| Fuel_TypeDiesel | 0 | 0 | 1.722070e-02 | 0.9862723 |
| Fuel_TypePetrol | 0 | 0 | -1.594567e-01 | 0.8734197 |
| Seller_TypeIndividual | 0 | 0 | 5.253902e-01 | 0.5997120 |
| TransmissionManual | 0 | 0 | -6.726510e-01 | 0.5017031 |
| Owner | 0 | 0 | -7.921000e-02 | 0.9369200 |
par(mfrow=c(2,2))
plot(full.model)
In this chunk, a new model is created without the categorical or irrelevant variables.
my_model <- lm(price ~ year + Kms_Driven + Present_Price, data = car)
kable(summary(my_model)$coef, caption ="Statistics of Regression Coefficients")
| Estimate | Std. Error | t value | Pr(>|t|) | |
|---|---|---|---|---|
| (Intercept) | -968.4712735 | 93.6812401 | -10.3379425 | 0.0000000 |
| year | 0.4813043 | 0.0464931 | 10.3521639 | 0.0000000 |
| Kms_Driven | -0.0000012 | 0.0000035 | -0.3440474 | 0.7310541 |
| Present_Price | 0.5256219 | 0.0135265 | 38.8586658 | 0.0000000 |
par(mfrow=c(2,2))
plot(my_model)
In this case, the data is not considered normal and does not pass the conditions, so a boxcox transformation is needed in order to try and correct the data.
library(MASS)
par(pty = "s", mfrow = c(2, 2), oma=c(.1,.1,.1,.1), mar=c(4, 0, 2, 0))
boxcox(price ~ Kms_Driven + Year + Present_Price, data = car, lambda = seq(0, 1, length = 10),
xlab=expression(paste(lambda, ": Kms_Driven")))
boxcox(price ~ Kms_Driven + Year + Present_Price, data = car, lambda = seq(-0.5, 1, length = 10),
xlab=expression(paste(lambda, ": Kms_Driven")))
boxcox(price ~ log(1+Year) + Kms_Driven + Present_Price, data = car, lambda = seq(-0.5, 1, length = 10),
xlab=expression(paste(lambda, ": log-year")))
boxcox(price ~ log(1+Year) + Kms_Driven + log(Present_Price), data = car, lambda = seq(-0.5, 1, length = 10),
xlab=expression(paste(lambda, ": log-year, log-Present_Price")))
A square root transformation is performed, in this case the transformation has helped the Q-Q plot quite a bit. The residuals are now fitting better on the line then before.
library(kableExtra)
sqrt.price.log.dist <- lm((price)^0.5 ~ Year + Kms_Driven + Present_Price,
data = car_data)
kable(summary(sqrt.price.log.dist)$coef, caption = "sqrt-transformed model")
| Estimate | Std. Error | t value | Pr(>|t|) | |
|---|---|---|---|---|
| (Intercept) | -231.3385898 | 23.6393323 | -9.7861728 | 0.0000000 |
| Year | 0.1154252 | 0.0117320 | 9.8385114 | 0.0000000 |
| Kms_Driven | 0.0000006 | 0.0000009 | 0.6916139 | 0.4897201 |
| Present_Price | 0.1020296 | 0.0034133 | 29.8922196 | 0.0000000 |
par(mfrow = c(2,2))
plot(sqrt.price.log.dist)
A log-transformation is also used to try and normalize the residuals, but it is less effective than the square root transformation.
log.price = lm(log(price) ~ year + Kms_Driven + Present_Price, data = car)
kable(summary(log.price)$coef, caption = "log-transformed model")
| Estimate | Std. Error | t value | Pr(>|t|) | |
|---|---|---|---|---|
| (Intercept) | -280.9530932 | 38.6550334 | -7.2682150 | 0.0000000 |
| year | 0.1395664 | 0.0191841 | 7.2751007 | 0.0000000 |
| Kms_Driven | 0.0000007 | 0.0000015 | 0.4701361 | 0.6386028 |
| Present_Price | 0.1054324 | 0.0055813 | 18.8901433 | 0.0000000 |
par(mfrow = c(2,2))
plot(log.price)
This displays the goodness of fit measurements for the three models, the full model, the square root transformation, and the log transformation. The full model has the highest R-squared value with a value of 1 a perfect fit, pottentianly indicated overfitting. The next best model presented is the square root transformation. The square root transformation outperforms the log transformation in the fact that it has lower sum of squares(SSE), Akaike information criterion (AIC), and simulation-based calibration (SBC) values while also having a higher R^2 value.
select=function(m){
e = m$resid
n0 = length(e)
SSE=(m$df)*(summary(m)$sigma)^2
R.sq=summary(m)$r.squared
R.adj=summary(m)$adj.r
MSE=(summary(m)$sigma)^2
Cp=(SSE/MSE)-(n0-2*(n0-m$df))
AIC=n0*log(SSE)-n0*log(n0)+2*(n0-m$df)
SBC=n0*log(SSE)-n0*log(n0)+(log(n0))*(n0-m$df)
X=model.matrix(m)
H=X%*%solve(t(X)%*%X)%*%t(X)
d=e/(1-diag(H))
PRESS=t(d)%*%d
tbl = as.data.frame(cbind(SSE=SSE, R.sq=R.sq, R.adj = R.adj, Cp = Cp, AIC = AIC, SBC = SBC, PRD = PRESS))
names(tbl)=c("SSE", "R.sq", "R.adj", "Cp", "AIC", "SBC", "PRESS")
tbl
}
output.sum = rbind(select(full.model), select(sqrt.price.log.dist), select(log.price))
## Warning in summary.lm(m): essentially perfect fit: summary may be unreliable
## Warning in summary.lm(m): essentially perfect fit: summary may be unreliable
## Warning in summary.lm(m): essentially perfect fit: summary may be unreliable
## Warning in summary.lm(m): essentially perfect fit: summary may be unreliable
row.names(output.sum) = c("full.model", "sqrt.price.log.dist", "log.price")
kable(output.sum, caption = "Goodness-of-fit Measures of Candidate Models")
| SSE | R.sq | R.adj | Cp | AIC | SBC | PRESS | |
|---|---|---|---|---|---|---|---|
| full.model | 0.00000 | 1.0000000 | 1.0000000 | 10 | -20947.7353 | -20910.6642 | 0.00000 |
| sqrt.price.log.dist | 73.97152 | 0.7775812 | 0.7753345 | 4 | -414.4324 | -399.6040 | 98.94122 |
| log.price | 197.79088 | 0.5922889 | 0.5881706 | 4 | -118.3899 | -103.5614 | 256.49424 |
The final model chosen is the square root transformation.
kable(summary(sqrt.price.log.dist)$coef, caption = "Inferential Statistics of Final Model")
| Estimate | Std. Error | t value | Pr(>|t|) | |
|---|---|---|---|---|
| (Intercept) | -231.3385898 | 23.6393323 | -9.7861728 | 0.0000000 |
| Year | 0.1154252 | 0.0117320 | 9.8385114 | 0.0000000 |
| Kms_Driven | 0.0000006 | 0.0000009 | 0.6916139 | 0.4897201 |
| Present_Price | 0.1020296 | 0.0034133 | 29.8922196 | 0.0000000 |
In this section two different methods for bootstrapping the final model are performed. The first bootstrap method chosen is the bootstrap residuals (btr) method. In the chunk below, two normal-density curves were placed on each of the histograms.
sqrt.price.log.dist <- lm(sqrt(price) ~ Kms_Driven + Present_Price + year, data = car)
cmtrx <- coef(sqrt.price.log.dist)
model.resid = sqrt.price.log.dist$residuals
##
B=1000
num.p = dim(model.matrix(sqrt.price.log.dist))[2] # number of parameters
samp.n = dim(model.matrix(sqrt.price.log.dist))[1] # sample size
btr.mtrx = matrix(rep(0, num.p*B), ncol=num.p) # zero matrix to store boot coefs
for (i in 1:B){
## Bootstrap response values
bt.sqrt.price = sqrt.price.log.dist$fitted.values +
sample(sqrt.price.log.dist$residuals, samp.n, replace = TRUE) # bootstrap residuals
# replace sqrt(PriceUnitArea) with bootstrap sqrt.price
car$bt.sqrt.price = bt.sqrt.price # send the boot response to the data
btr.model = lm(bt.sqrt.price ~ Kms_Driven + Present_Price + year, data = car) # b
btr.mtrx[i,] = btr.model$coefficients
}
coef.mtrx <- btr.mtrx
boot.hist = function(bt.coef.mtrx, var.id, var.nm){
## bt.coef.mtrx = matrix for storing bootstrap estimates of coefficients
## var.id = variable ID (1, 2, ..., k+1)
## var.nm = variable name on the hist title, must be the string in the double quotes
## Bootstrap sampling distribution of the estimated coefficients
x1.1 <- seq(min(bt.coef.mtrx[,var.id]), max(bt.coef.mtrx[,var.id]), length=301 )
y1.1 <- dnorm(x1.1, mean(bt.coef.mtrx[,var.id]), sd(bt.coef.mtrx[,var.id]))
# height of the histogram - use it to make a nice-looking histogram.
highestbar = max(hist(bt.coef.mtrx[,var.id], plot = FALSE)$density)
ylimit <- max(c(y1.1,highestbar))
hist(bt.coef.mtrx[,var.id], probability = TRUE, main = var.nm, xlab="",
col = "azure1",ylim=c(0,ylimit), border="lightseagreen")
lines(x = x1.1, y = y1.1, col = "red3") # normal density curve
lines(density(bt.coef.mtrx[,var.id], adjust=2), col="blue") # loess curve
}
par(mfrow=c(2,3)) # histograms of bootstrap coefs
boot.hist(bt.coef.mtrx=coef.mtrx, var.id=1, var.nm ="Intercept" )
boot.hist(bt.coef.mtrx=coef.mtrx, var.id=2, var.nm ="Kms_Driven" )
boot.hist(bt.coef.mtrx=coef.mtrx, var.id=3, var.nm ="Present_Price" )
boot.hist(bt.coef.mtrx=coef.mtrx, var.id=4, var.nm ="Year" )
A Bootstrap Regression Coefficient Matrix is also provided below.
num.p = dim(btr.mtrx)[2] # number of parameters
btr.ci = NULL
btr.wd = NULL
for (i in 1:num.p){
lci.025 = round(quantile(btr.mtrx[, i], 0.025, type = 2), 8)
uci.975 = round(quantile(btr.mtrx[, i], 0.975, type = 2), 8)
btr.wd[i] = uci.975 - lci.025
btr.ci[i] = paste("[", round(lci.025, 4), ", ", round(uci.975, 4), "]")
}
# print the regression coefficient matrix with 95% residual bootstrap CI
cmtrx <- summary(sqrt.price.log.dist)$coef
kable(as.data.frame(cbind(sqrt.price.log.dist = formatC(cmtrx,4,format="f"),
btr.ci.95=btr.ci)),
caption = "Regression Coefficient Matrix with 95% Residual Bootstrap CI",
align = "c", digits = 4)
| Estimate | Std. Error | t value | Pr(>|t|) | btr.ci.95 | |
|---|---|---|---|---|---|
| (Intercept) | -231.3386 | 23.6393 | -9.7862 | 0.0000 | [ -279.4027 , -185.7892 ] |
| Kms_Driven | 0.0000 | 0.0000 | 0.6916 | 0.4897 | [ 0 , 0 ] |
| Present_Price | 0.1020 | 0.0034 | 29.8922 | 0.0000 | [ 0.0953 , 0.1079 ] |
| year | 0.1154 | 0.0117 | 9.8385 | 0.0000 | [ 0.0928 , 0.1393 ] |
The second method is Bootstrap cases (btc). In the chunk below, two normal-density curves are again placed on each of the histograms.
car_data <- read.csv("https://raw.githubusercontent.com/Kyle-Weber/STA321/main/www/car%20data.csv", header = TRUE)
cmtrx <- coef(sqrt.price.log.dist)
# fit the model
sqrt.price.log.dist = lm(sqrt(price) ~ Kms_Driven + Present_Price + year, data = car)
# set the number of bootstrap replicates
B = 1000
# get the number of parameters in the model and sample size
num.p = dim(model.frame(sqrt.price.log.dist))[2]
smpl.n = dim(model.frame(sqrt.price.log.dist))[1]
# zero matrix to store bootstrap coefficients
coef.mtrx = matrix(rep(0, B*num.p), ncol = num.p)
# loop through bootstrap replicates
for (i in 1:B) {
# generate bootstrap sample
bootc.id = sample(1:smpl.n, smpl.n, replace = TRUE)
# fit the model to the bootstrap sample
sqrt.price.log.dist.btc = lm(sqrt(price) ~ Kms_Driven + Present_Price + year, data = car[bootc.id, ])
# extract coefficients from the bootstrap regression model
coef.mtrx[i, ] = coef(sqrt.price.log.dist.btc)
}
boot.hist = function(cmtrx, bt.coef.mtrx, var.id, var.nm){
## bt.coef.mtrx = matrix for storing bootstrap estimates of coefficients
## var.id = variable ID (1, 2, ..., k+1)
## var.nm = variable name on the hist title, must be the string in the double quotes
## coefficient matrix of the final model
## Bootstrap sampling distribution of the estimated coefficients
x1.1 <- seq(min(bt.coef.mtrx[,var.id]), max(bt.coef.mtrx[,var.id]), length=301 )
y1.1 <- dnorm(x1.1, mean(bt.coef.mtrx[,var.id]), sd(bt.coef.mtrx[,var.id]))
# height of the histogram - use it to make a nice-looking histogram.
highestbar = max(hist(bt.coef.mtrx[,var.id], plot = FALSE)$density)
ylimit <- max(c(y1.1,highestbar))
hist(bt.coef.mtrx[,var.id], probability = TRUE, main = var.nm, xlab="",
col = "azure1",ylim=c(0,ylimit), border="lightseagreen")
lines(x = x1.1, y = y1.1, col = "red3")
lines(density(bt.coef.mtrx[,var.id], adjust=2), col="blue")
#legend("topright", c(""))
}
par(mfrow=c(2,3)) # histograms of bootstrap coefs
boot.hist(bt.coef.mtrx=coef.mtrx, var.id=1, var.nm ="Intercept" )
boot.hist(bt.coef.mtrx=coef.mtrx, var.id=2, var.nm ="Kms_Driven" )
boot.hist(bt.coef.mtrx=coef.mtrx, var.id=3, var.nm ="Present_Price" )
boot.hist(bt.coef.mtrx=coef.mtrx, var.id=4, var.nm ="Year" )
Next a Regression Coefficient Matrix is performed.
num.p = dim(coef.mtrx)[2] # number of parameters
btc.ci = NULL
btc.wd = NULL
for (i in 1:num.p){
lci.025 = round(quantile(coef.mtrx[, i], 0.025, type = 2),8)
uci.975 = round(quantile(coef.mtrx[, i],0.975, type = 2 ),8)
btc.wd[i] = uci.975 - lci.025
btc.ci[i] = paste("[", round(lci.025,4),", ", round(uci.975,4),"]")
}
#as.data.frame(btc.ci)
cmtrx <- summary(sqrt.price.log.dist)$coef
kable(as.data.frame(cbind(formatC(cmtrx,4,format="f"), btc.ci.95=btc.ci)),
caption = "Regression Coefficient Matrix")
| Estimate | Std. Error | t value | Pr(>|t|) | btc.ci.95 | |
|---|---|---|---|---|---|
| (Intercept) | -231.3386 | 23.6393 | -9.7862 | 0.0000 | [ -199.4917 , -185.9534 ] |
| Kms_Driven | 0.0000 | 0.0000 | 0.6916 | 0.4897 | [ 0 , 0 ] |
| Present_Price | 0.1020 | 0.0034 | 29.8922 | 0.0000 | [ -0.0135 , 0.0153 ] |
| year | 0.1154 | 0.0117 | 9.8385 | 0.0000 | [ 0.0932 , 0.1 ] |
3.4 Combining All Inferential Statistics Finally, we put all inferential statistics in a single table so we can compare these results.
kable(as.data.frame(cbind(formatC(cmtrx[,-3],4,format="f"), btr.ci.95=btr.ci, btc.ci.95=btc.ci)),
caption="Final Combined Inferential Statistics: p-values and Bootstrap CIs")
| Estimate | Std. Error | Pr(>|t|) | btr.ci.95 | btc.ci.95 | |
|---|---|---|---|---|---|
| (Intercept) | -231.3386 | 23.6393 | 0.0000 | [ -279.4027 , -185.7892 ] | [ -199.4917 , -185.9534 ] |
| Kms_Driven | 0.0000 | 0.0000 | 0.4897 | [ 0 , 0 ] | [ 0 , 0 ] |
| Present_Price | 0.1020 | 0.0034 | 0.0000 | [ 0.0953 , 0.1079 ] | [ -0.0135 , 0.0153 ] |
| year | 0.1154 | 0.0117 | 0.0000 | [ 0.0928 , 0.1393 ] | [ 0.0932 , 0.1 ] |
Interpreting the results. The predictor variable Kms_Driven is not statistically significant in predicting the response variable (p-value = 0.4897), and its coefficient estimate is close to zero (0.0000).The other two predictor variables, Present_Price and year, are both statistically significant in predicting the response variable, with p-values of 0.0000. The coefficient estimate for Present_Price is 0.1020, while the coefficient estimate for year is 0.1154, indicating that the predictor variable is associated with an increase in the response variable. Additionally, the intercept is -231.3386, but since it was transformed using a square root transformation, it should be interpreted with caution. The bootstrap confidence intervals provided in the output above indicate the range of values that the coefficients of the regression model are likely to fall within. For example, the bootstrap 95% confidence interval for the intercept is [-278.5945, -183.517], which suggests that with 95% confidence, the true population value for the intercept falls within this range.
In this case, seeing as how a normal distribution is assumed, it would be better to stick with the parametric test. while the bootstrap is good for models in which normality is not assumed, that is not the case here, so a parametric test would be better.
Going forwards, picking a better data set would be advised. Choosing one with less categorical variables and with variables more associated with the response variable would help. The application of this model could help in predictions for car prices, but seeing the lack of normality, it might be best to approach cautiously.