This data set involves cars and various information about them. The data set car_data includes the variables: name, year, selling price, present price, kilometers driven, fuel type, seller type, transmission, and owner. The total sample size is 301. This analysis seeks to run a multi-linear regression, analyze conditions, transform the response variable with a box-cox, perform a goodness of fit test, build candidate models and select a final model. The final model will then be Bootstrapped and compared. In this particular instance the response variable will be selling price and the predictor variables are year, Present Price, and kilometers.

Below the full model is provided, along with a residual analysis along with discretizing Fuel Type.

library(kableExtra)
car_data <- read.csv("https://raw.githubusercontent.com/Kyle-Weber/STA321/main/www/car%20data.csv", header = TRUE)
car <- car_data[, -1]
price <- car$Selling_Price
year <- car$Year

#Discretizing continuous variable Fuel_Type
data <- data.frame(
  Fuel_Type = c("Petrol", "Diesel", "Petrol", "Petrol", "Diesel")
)

# create a new variable based on the Fuel_Type variable
data$Fuel_Type_Discrete <- ifelse(data$Fuel_Type == "Diesel", 1, 0)

#Printing out full model and residual analysis
full.model = lm(price ~ ., data = car)
kable(summary(full.model)$coef, caption ="Statistics of Regression Coefficients")
## Warning in summary.lm(full.model): essentially perfect fit: summary may be
## unreliable
Statistics of Regression Coefficients
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0 0 -3.363786e+00 0.0008719
Year 0 0 3.384092e+00 0.0008122
Selling_Price 1 0 3.969546e+16 0.0000000
Present_Price 0 0 -1.571441e+00 0.1171669
Kms_Driven 0 0 7.070469e-01 0.4801031
Fuel_TypeDiesel 0 0 1.722070e-02 0.9862723
Fuel_TypePetrol 0 0 -1.594567e-01 0.8734197
Seller_TypeIndividual 0 0 5.253902e-01 0.5997120
TransmissionManual 0 0 -6.726510e-01 0.5017031
Owner 0 0 -7.921000e-02 0.9369200
par(mfrow=c(2,2))
plot(full.model)

In this chunk, a new model is created without the categorical or irrelevant variables.

my_model <- lm(price ~ year + Kms_Driven + Present_Price, data = car)
kable(summary(my_model)$coef, caption ="Statistics of Regression Coefficients")
Statistics of Regression Coefficients
Estimate Std. Error t value Pr(>|t|)
(Intercept) -968.4712735 93.6812401 -10.3379425 0.0000000
year 0.4813043 0.0464931 10.3521639 0.0000000
Kms_Driven -0.0000012 0.0000035 -0.3440474 0.7310541
Present_Price 0.5256219 0.0135265 38.8586658 0.0000000
par(mfrow=c(2,2))
plot(my_model)

In this case, the data is not considered normal and does not pass the conditions, so a boxcox transformation is needed in order to try and correct the data.

library(MASS)
par(pty = "s", mfrow = c(2, 2), oma=c(.1,.1,.1,.1), mar=c(4, 0, 2, 0))

boxcox(price ~ Kms_Driven + Year + Present_Price, data = car, lambda = seq(0, 1, length = 10),
xlab=expression(paste(lambda, ": Kms_Driven")))

boxcox(price ~ Kms_Driven + Year + Present_Price, data = car, lambda = seq(-0.5, 1, length = 10),
xlab=expression(paste(lambda, ": Kms_Driven")))

boxcox(price ~ log(1+Year) + Kms_Driven + Present_Price, data = car, lambda = seq(-0.5, 1, length = 10),
xlab=expression(paste(lambda, ": log-year")))

boxcox(price ~ log(1+Year) + Kms_Driven + log(Present_Price), data = car, lambda = seq(-0.5, 1, length = 10),
xlab=expression(paste(lambda, ": log-year, log-Present_Price")))

A square root transformation is performed, in this case the transformation has helped the Q-Q plot quite a bit. The residuals are now fitting better on the line then before.

library(kableExtra)
sqrt.price.log.dist <- lm((price)^0.5 ~ Year + Kms_Driven + Present_Price, 
                          data = car_data)

kable(summary(sqrt.price.log.dist)$coef, caption = "sqrt-transformed model")
sqrt-transformed model
Estimate Std. Error t value Pr(>|t|)
(Intercept) -231.3385898 23.6393323 -9.7861728 0.0000000
Year 0.1154252 0.0117320 9.8385114 0.0000000
Kms_Driven 0.0000006 0.0000009 0.6916139 0.4897201
Present_Price 0.1020296 0.0034133 29.8922196 0.0000000
par(mfrow = c(2,2))
plot(sqrt.price.log.dist)

A log-transformation is also used to try and normalize the residuals, but it is less effective than the square root transformation.

log.price = lm(log(price) ~ year + Kms_Driven + Present_Price, data = car)
kable(summary(log.price)$coef, caption = "log-transformed model")
log-transformed model
Estimate Std. Error t value Pr(>|t|)
(Intercept) -280.9530932 38.6550334 -7.2682150 0.0000000
year 0.1395664 0.0191841 7.2751007 0.0000000
Kms_Driven 0.0000007 0.0000015 0.4701361 0.6386028
Present_Price 0.1054324 0.0055813 18.8901433 0.0000000
par(mfrow = c(2,2))
plot(log.price)

This displays the goodness of fit measurements for the three models, the full model, the square root transformation, and the log transformation. The full model has the highest R-squared value with a value of 1 a perfect fit, pottentianly indicated overfitting. The next best model presented is the square root transformation. The square root transformation outperforms the log transformation in the fact that it has lower sum of squares(SSE), Akaike information criterion (AIC), and simulation-based calibration (SBC) values while also having a higher R^2 value.

select=function(m){ 
 e = m$resid                           
 n0 = length(e)                        
 SSE=(m$df)*(summary(m)$sigma)^2       
 R.sq=summary(m)$r.squared             
 R.adj=summary(m)$adj.r                
 MSE=(summary(m)$sigma)^2              
 Cp=(SSE/MSE)-(n0-2*(n0-m$df))         
 AIC=n0*log(SSE)-n0*log(n0)+2*(n0-m$df)          
 SBC=n0*log(SSE)-n0*log(n0)+(log(n0))*(n0-m$df)  
 X=model.matrix(m)                     
 H=X%*%solve(t(X)%*%X)%*%t(X)         
 d=e/(1-diag(H))                       
 PRESS=t(d)%*%d   
 tbl = as.data.frame(cbind(SSE=SSE, R.sq=R.sq, R.adj = R.adj, Cp = Cp, AIC = AIC, SBC = SBC, PRD = PRESS))
 names(tbl)=c("SSE", "R.sq", "R.adj", "Cp", "AIC", "SBC", "PRESS")
 tbl
 }
output.sum = rbind(select(full.model), select(sqrt.price.log.dist), select(log.price))
## Warning in summary.lm(m): essentially perfect fit: summary may be unreliable

## Warning in summary.lm(m): essentially perfect fit: summary may be unreliable

## Warning in summary.lm(m): essentially perfect fit: summary may be unreliable

## Warning in summary.lm(m): essentially perfect fit: summary may be unreliable
row.names(output.sum) = c("full.model", "sqrt.price.log.dist", "log.price")
kable(output.sum, caption = "Goodness-of-fit Measures of Candidate Models")
Goodness-of-fit Measures of Candidate Models
SSE R.sq R.adj Cp AIC SBC PRESS
full.model 0.00000 1.0000000 1.0000000 10 -20947.7353 -20910.6642 0.00000
sqrt.price.log.dist 73.97152 0.7775812 0.7753345 4 -414.4324 -399.6040 98.94122
log.price 197.79088 0.5922889 0.5881706 4 -118.3899 -103.5614 256.49424

The final model chosen is the square root transformation.

kable(summary(sqrt.price.log.dist)$coef, caption = "Inferential Statistics of Final Model")
Inferential Statistics of Final Model
Estimate Std. Error t value Pr(>|t|)
(Intercept) -231.3385898 23.6393323 -9.7861728 0.0000000
Year 0.1154252 0.0117320 9.8385114 0.0000000
Kms_Driven 0.0000006 0.0000009 0.6916139 0.4897201
Present_Price 0.1020296 0.0034133 29.8922196 0.0000000

In this section two different methods for bootstrapping the final model are performed. The first bootstrap method chosen is the bootstrap residuals (btr) method. In the chunk below, two normal-density curves were placed on each of the histograms.

sqrt.price.log.dist <- lm(sqrt(price) ~ Kms_Driven + Present_Price + year, data = car)  
cmtrx <- coef(sqrt.price.log.dist)

model.resid = sqrt.price.log.dist$residuals
##
B=1000
num.p = dim(model.matrix(sqrt.price.log.dist))[2]   # number of parameters
samp.n = dim(model.matrix(sqrt.price.log.dist))[1]  # sample size
btr.mtrx = matrix(rep(0, num.p*B), ncol=num.p) # zero matrix to store boot coefs
for (i in 1:B){
  ## Bootstrap response values
  bt.sqrt.price = sqrt.price.log.dist$fitted.values + 
        sample(sqrt.price.log.dist$residuals, samp.n, replace = TRUE)  # bootstrap residuals
  # replace sqrt(PriceUnitArea) with bootstrap sqrt.price
  car$bt.sqrt.price = bt.sqrt.price   # send the boot response to the data
  btr.model = lm(bt.sqrt.price ~ Kms_Driven + Present_Price + year, data = car)   # b
  btr.mtrx[i,] = btr.model$coefficients
} 
coef.mtrx <- btr.mtrx


boot.hist = function(bt.coef.mtrx, var.id, var.nm){
  ## bt.coef.mtrx = matrix for storing bootstrap estimates of coefficients
  ## var.id = variable ID (1, 2, ..., k+1)
  ## var.nm = variable name on the hist title, must be the string in the double quotes
  ## Bootstrap sampling distribution of the estimated coefficients
  x1.1 <- seq(min(bt.coef.mtrx[,var.id]), max(bt.coef.mtrx[,var.id]), length=301 )
  y1.1 <- dnorm(x1.1, mean(bt.coef.mtrx[,var.id]), sd(bt.coef.mtrx[,var.id]))
  # height of the histogram - use it to make a nice-looking histogram.
  highestbar = max(hist(bt.coef.mtrx[,var.id], plot = FALSE)$density) 
  ylimit <- max(c(y1.1,highestbar))
  hist(bt.coef.mtrx[,var.id], probability = TRUE, main = var.nm, xlab="", 
       col = "azure1",ylim=c(0,ylimit), border="lightseagreen")
  lines(x = x1.1, y = y1.1, col = "red3")       # normal density curve         
  lines(density(bt.coef.mtrx[,var.id], adjust=2), col="blue")    # loess curve
} 

par(mfrow=c(2,3))  # histograms of bootstrap coefs
boot.hist(bt.coef.mtrx=coef.mtrx, var.id=1, var.nm ="Intercept" )
boot.hist(bt.coef.mtrx=coef.mtrx, var.id=2, var.nm ="Kms_Driven" )
boot.hist(bt.coef.mtrx=coef.mtrx, var.id=3, var.nm ="Present_Price" )
boot.hist(bt.coef.mtrx=coef.mtrx, var.id=4, var.nm ="Year" )

A Bootstrap Regression Coefficient Matrix is also provided below.

num.p = dim(btr.mtrx)[2]  # number of parameters
btr.ci = NULL
btr.wd = NULL
for (i in 1:num.p){
  lci.025 = round(quantile(btr.mtrx[, i], 0.025, type = 2), 8)
  uci.975 = round(quantile(btr.mtrx[, i], 0.975, type = 2), 8)
  btr.wd[i] = uci.975 - lci.025
  btr.ci[i] = paste("[", round(lci.025, 4), ", ", round(uci.975, 4), "]")
}
# print the regression coefficient matrix with 95% residual bootstrap CI
cmtrx <- summary(sqrt.price.log.dist)$coef

kable(as.data.frame(cbind(sqrt.price.log.dist = formatC(cmtrx,4,format="f"), 
                        btr.ci.95=btr.ci)), 
      caption = "Regression Coefficient Matrix with 95% Residual Bootstrap CI", 
      align = "c", digits = 4)
Regression Coefficient Matrix with 95% Residual Bootstrap CI
Estimate Std. Error t value Pr(>|t|) btr.ci.95
(Intercept) -231.3386 23.6393 -9.7862 0.0000 [ -279.4027 , -185.7892 ]
Kms_Driven 0.0000 0.0000 0.6916 0.4897 [ 0 , 0 ]
Present_Price 0.1020 0.0034 29.8922 0.0000 [ 0.0953 , 0.1079 ]
year 0.1154 0.0117 9.8385 0.0000 [ 0.0928 , 0.1393 ]

The second method is Bootstrap cases (btc). In the chunk below, two normal-density curves are again placed on each of the histograms.

car_data <- read.csv("https://raw.githubusercontent.com/Kyle-Weber/STA321/main/www/car%20data.csv", header = TRUE)
cmtrx <- coef(sqrt.price.log.dist)

# fit the model
sqrt.price.log.dist = lm(sqrt(price) ~ Kms_Driven + Present_Price + year, data = car)

# set the number of bootstrap replicates
B = 1000

# get the number of parameters in the model and sample size
num.p = dim(model.frame(sqrt.price.log.dist))[2]
smpl.n = dim(model.frame(sqrt.price.log.dist))[1]

# zero matrix to store bootstrap coefficients 
coef.mtrx = matrix(rep(0, B*num.p), ncol = num.p)

# loop through bootstrap replicates
for (i in 1:B) {
  # generate bootstrap sample
  bootc.id = sample(1:smpl.n, smpl.n, replace = TRUE)
  # fit the model to the bootstrap sample
  sqrt.price.log.dist.btc = lm(sqrt(price) ~  Kms_Driven + Present_Price + year, data = car[bootc.id, ])
  # extract coefficients from the bootstrap regression model
  coef.mtrx[i, ] = coef(sqrt.price.log.dist.btc)
}

boot.hist = function(cmtrx, bt.coef.mtrx, var.id, var.nm){
  ## bt.coef.mtrx = matrix for storing bootstrap estimates of coefficients
  ## var.id = variable ID (1, 2, ..., k+1)
  ## var.nm = variable name on the hist title, must be the string in the double quotes
  ## coefficient matrix of the final model
  ## Bootstrap sampling distribution of the estimated coefficients
  x1.1 <- seq(min(bt.coef.mtrx[,var.id]), max(bt.coef.mtrx[,var.id]), length=301 )
  y1.1 <- dnorm(x1.1, mean(bt.coef.mtrx[,var.id]), sd(bt.coef.mtrx[,var.id]))
  # height of the histogram - use it to make a nice-looking histogram.
  highestbar = max(hist(bt.coef.mtrx[,var.id], plot = FALSE)$density) 
  ylimit <- max(c(y1.1,highestbar))
  hist(bt.coef.mtrx[,var.id], probability = TRUE, main = var.nm, xlab="", 
       col = "azure1",ylim=c(0,ylimit), border="lightseagreen")
  lines(x = x1.1, y = y1.1, col = "red3")
  lines(density(bt.coef.mtrx[,var.id], adjust=2), col="blue") 
  #legend("topright", c(""))
}


par(mfrow=c(2,3))  # histograms of bootstrap coefs
boot.hist(bt.coef.mtrx=coef.mtrx, var.id=1, var.nm ="Intercept" )
boot.hist(bt.coef.mtrx=coef.mtrx, var.id=2, var.nm ="Kms_Driven" )
boot.hist(bt.coef.mtrx=coef.mtrx, var.id=3, var.nm ="Present_Price" )
boot.hist(bt.coef.mtrx=coef.mtrx, var.id=4, var.nm ="Year" )

Next a Regression Coefficient Matrix is performed.

num.p = dim(coef.mtrx)[2]  # number of parameters
btc.ci = NULL
btc.wd = NULL
for (i in 1:num.p){
  lci.025 = round(quantile(coef.mtrx[, i], 0.025, type = 2),8)
  uci.975 = round(quantile(coef.mtrx[, i],0.975, type = 2 ),8)
  btc.wd[i] =  uci.975 - lci.025
  btc.ci[i] = paste("[", round(lci.025,4),", ", round(uci.975,4),"]")
 }
#as.data.frame(btc.ci)
cmtrx <- summary(sqrt.price.log.dist)$coef

kable(as.data.frame(cbind(formatC(cmtrx,4,format="f"), btc.ci.95=btc.ci)), 
      
      caption = "Regression Coefficient Matrix")
Regression Coefficient Matrix
Estimate Std. Error t value Pr(>|t|) btc.ci.95
(Intercept) -231.3386 23.6393 -9.7862 0.0000 [ -199.4917 , -185.9534 ]
Kms_Driven 0.0000 0.0000 0.6916 0.4897 [ 0 , 0 ]
Present_Price 0.1020 0.0034 29.8922 0.0000 [ -0.0135 , 0.0153 ]
year 0.1154 0.0117 9.8385 0.0000 [ 0.0932 , 0.1 ]

3.4 Combining All Inferential Statistics Finally, we put all inferential statistics in a single table so we can compare these results.

kable(as.data.frame(cbind(formatC(cmtrx[,-3],4,format="f"), btr.ci.95=btr.ci, btc.ci.95=btc.ci)), 
      caption="Final Combined Inferential Statistics: p-values and Bootstrap CIs")
Final Combined Inferential Statistics: p-values and Bootstrap CIs
Estimate Std. Error Pr(>|t|) btr.ci.95 btc.ci.95
(Intercept) -231.3386 23.6393 0.0000 [ -279.4027 , -185.7892 ] [ -199.4917 , -185.9534 ]
Kms_Driven 0.0000 0.0000 0.4897 [ 0 , 0 ] [ 0 , 0 ]
Present_Price 0.1020 0.0034 0.0000 [ 0.0953 , 0.1079 ] [ -0.0135 , 0.0153 ]
year 0.1154 0.0117 0.0000 [ 0.0928 , 0.1393 ] [ 0.0932 , 0.1 ]

Interpreting the results. The predictor variable Kms_Driven is not statistically significant in predicting the response variable (p-value = 0.4897), and its coefficient estimate is close to zero (0.0000).The other two predictor variables, Present_Price and year, are both statistically significant in predicting the response variable, with p-values of 0.0000. The coefficient estimate for Present_Price is 0.1020, while the coefficient estimate for year is 0.1154, indicating that the predictor variable is associated with an increase in the response variable. Additionally, the intercept is -231.3386, but since it was transformed using a square root transformation, it should be interpreted with caution. The bootstrap confidence intervals provided in the output above indicate the range of values that the coefficients of the regression model are likely to fall within. For example, the bootstrap 95% confidence interval for the intercept is [-278.5945, -183.517], which suggests that with 95% confidence, the true population value for the intercept falls within this range.

In this case, seeing as how a normal distribution is assumed, it would be better to stick with the parametric test. while the bootstrap is good for models in which normality is not assumed, that is not the case here, so a parametric test would be better.

Going forwards, picking a better data set would be advised. Choosing one with less categorical variables and with variables more associated with the response variable would help. The application of this model could help in predictions for car prices, but seeing the lack of normality, it might be best to approach cautiously.