Due February 18, 2023

Problem Overview

The goal of this homework is hands-on practice with data exploration and cleaning, classification, and model selection. You will:

  1. Conduct basic exploratory analysis of a data set
  2. Develop linear and logistic regression models
  3. Interpret your models
  4. Partition your dataset and evaluate your models in terms of classification performance

The Assignment

The data in the accompanying file “dc_airbnb_file.csv” (posted on Canvas) contains data about 4936 Airbnb.com listings from the Washington, DC area. This is a small subset of the data that you will eventually use for the class project. The variables in this dataset should be self-explanatory but we have also provided a data dictionary.

Your task is to develop models to predict the target variable “high_booking_rate”, which labels whether a listing is popular (i.e. spends most of the time booked) or not.

Please answer the questions below clearly and concisely, providing tables or plots where applicable. Turn in a well-formatted compiled HTML document using R Markdown, containing clear answers to the questions and R code in the appropriate places.

RUBRIC: To receive a passing score on this assignment, you must do the following:

  1. Turn in a well-formatted compiled HTML document using R markdown.
  2. Provide clear answers to the questions and the correct R commands as necessary, in the appropriate places. You may answer up to three sub-questions incorrectly (i.e. incorrect R command; missing answer to question) and still receive a P on this assignment (for example, 1(a) counts as one sub-question).
  3. The entire document must be clear, concise, and readable.

Note that this assignment is somewhat open-ended and there are many ways to answer these questions. I don’t require that we have exactly the same answers in order for you to receive full credit.

airbnb <- read_csv("dc_airbnb_hw1.csv")  #read the dataset in R
## Rows: 4936 Columns: 12
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr (7): name, bed_type, cancellation_policy, cleaning_fee, price, property_...
## dbl (5): accommodates, bedrooms, beds, host_total_listings_count, high_booki...
## 
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
names(airbnb)                       #variables used in dataset
##  [1] "name"                      "accommodates"             
##  [3] "bed_type"                  "bedrooms"                 
##  [5] "beds"                      "cancellation_policy"      
##  [7] "cleaning_fee"              "host_total_listings_count"
##  [9] "price"                     "property_type"            
## [11] "room_type"                 "high_booking_rate"

0: Example answer

What is the mean of the accommodates variable?

ANSWER: The mean number of people that can be accommodated in a listing in this dataset is 3.522893.

age_mean <- airbnb %>%
  summarise(mean_accommodates = mean(accommodates))

1: EDA and Data Cleaning

  1. Clean up and process some of the variables by doing the following (I recommend you do these in the order in which they are listed):

Hint: there are lots of ways to deal with cleaning_fee and price. I like the parse_number function from the readr library.

Code below:

airbnb <- airbnb %>%
  mutate(cancellation_policy = if_else(cancellation_policy %in% c('strict', 'super_strict_30'), 'strict', as.character(cancellation_policy))) %>%
  mutate(cleaning_fee = parse_number(cleaning_fee), price = parse_number(price)) %>%
  mutate(cleaning_fee = if_else(is.na(cleaning_fee), 0, cleaning_fee),
         price = if_else(is.na(price), 0, price)) %>%
  mutate(beds = if_else(is.na(beds), mean(beds, na.rm = TRUE), beds),
         bedrooms = if_else(is.na(bedrooms), mean(bedrooms, na.rm=TRUE), bedrooms),
         host_total_listings_count = if_else(is.na(host_total_listings_count), mean(host_total_listings_count, na.rm=TRUE), host_total_listings_count))
  1. Create five new variables:

Hint: there are lots of ways to do the property_type variable, but case_when might be a useful function that’s part of the tidyverse.

Variables created below:

airbnb <- airbnb %>%
  mutate(
    price_per_person = price / accommodates,
    has_cleaning_fee = if_else(cleaning_fee > 0, "YES", "NO"),
    bed_category = if_else(bed_type == "Real Bed", "bed", "other"),
    property_category = case_when(
      property_type %in% c("Apartment", "Serviced apartment", "Loft") ~ "apartment",
      property_type %in% c("Bed & Breakfast", "Boutique hotel", "Hostel") ~ "hotel",
      property_type %in% c("Townhouse", "Condominium") ~ "condo",
      property_type %in% c("Bungalow", "House") ~ "house",
      TRUE ~ "other"
    ),
    ppp_ind = as.integer(price_per_person > median(price_per_person))
  )

# Convert property_category to a factor
airbnb$property_category <- as.factor(airbnb$property_category)
airbnb$property_type <- as.factor(airbnb$property_type)
  1. Convert the remaining character variables to factors
airbnb <- airbnb %>%
  mutate(
    bed_type = as.factor(bed_type),
    cancellation_policy = as.factor(cancellation_policy),
    room_type = as.factor(room_type)
  )
  1. Construct and report boxplots of price_per_person and log(price_per_person), broken up by values of high_booking_rate. Does it appear that there is a relationship between price_per_person and high_booking_rate? If so, what is the relationship?

There seems to be a relationship between price per person and high booing rate. For units with a low booking rate, the average price per person was higher and had more variability as well. When the unit had a high booking rate the average price per person was lower, and there was less variability in the price per person for those units.

# Boxtplot regular
ggplot(airbnb, aes(x = as.factor(high_booking_rate), y = price_per_person)) +
  geom_boxplot() +
  labs(title = "Boxplot of price_per_person by high_booking_rate",
       x = "high_booking_rate",
       y = "price_per_person")

# Boxplot for Log() version
ggplot(airbnb, aes(x = as.factor(high_booking_rate), y = log(price_per_person))) +
  geom_boxplot() +
  labs(title = "Boxplot of log(price_per_person) by high_booking_rate",
       x = "high_booking_rate",
       y = "log(price_per_person)")

  1. Construct a two-way table of high_booking_rate by property_category. Does it appear that any property categories are particularly bad or good in terms of high booking rates?

(hint: prop.table might be especially helpful here)

Yes, it seems that condos tend to get high booking rates more often than the average property type, and it also seems that hotels tend to get high booking rates less often than the average property type.

two_way_table <- table(airbnb$property_category, airbnb$high_booking_rate)
proportions <- prop.table(two_way_table, margin = 1) * 100
proportions
##            
##                    0        1
##   apartment 74.97207 25.02793
##   condo     69.35484 30.64516
##   hotel     80.64516 19.35484
##   house     76.74568 23.25432
##   other     71.42857 28.57143

2: Linear Regression

  1. Train a linear regression to predict high_booking_rate using the variables listed below. Report the R^2.

R^2 was 0.07788. The linear model was created using high booking rate as target variable, and re-classified this variable as numeric for the model to work (was getting r error because I had it as a factor)

model_a <- lm(high_booking_rate ~ cancellation_policy + cleaning_fee + price_per_person + ppp_ind + has_cleaning_fee + accommodates + bed_category + bedrooms + beds + host_total_listings_count + property_category, data = airbnb)

summary(model_a)
## 
## Call:
## lm(formula = high_booking_rate ~ cancellation_policy + cleaning_fee + 
##     price_per_person + ppp_ind + has_cleaning_fee + accommodates + 
##     bed_category + bedrooms + beds + host_total_listings_count + 
##     property_category, data = airbnb)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -1.0122 -0.2869 -0.1763  0.4341  0.9940 
## 
## Coefficients:
##                               Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                  1.859e-01  2.104e-02   8.834  < 2e-16 ***
## cancellation_policymoderate  8.539e-02  1.582e-02   5.399 7.03e-08 ***
## cancellation_policystrict    6.924e-02  1.595e-02   4.342 1.44e-05 ***
## cleaning_fee                -1.257e-03  1.897e-04  -6.630 3.71e-11 ***
## price_per_person            -4.784e-04  1.408e-04  -3.399 0.000683 ***
## ppp_ind                     -8.899e-02  1.391e-02  -6.399 1.71e-10 ***
## has_cleaning_feeYES          1.472e-01  1.732e-02   8.498  < 2e-16 ***
## accommodates                 1.339e-02  5.981e-03   2.239 0.025187 *  
## bed_categoryother           -3.722e-02  3.490e-02  -1.066 0.286302    
## bedrooms                    -8.126e-02  1.190e-02  -6.827 9.73e-12 ***
## beds                         4.362e-02  8.970e-03   4.863 1.19e-06 ***
## host_total_listings_count   -2.231e-04  7.148e-05  -3.122 0.001809 ** 
## property_categorycondo       6.596e-02  1.982e-02   3.328 0.000882 ***
## property_categoryhotel       1.446e-02  5.410e-02   0.267 0.789317    
## property_categoryhouse      -4.866e-05  1.438e-02  -0.003 0.997300    
## property_categoryother      -1.082e-02  5.111e-02  -0.212 0.832356    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.4169 on 4920 degrees of freedom
## Multiple R-squared:  0.07788,    Adjusted R-squared:  0.07507 
## F-statistic:  27.7 on 15 and 4920 DF,  p-value: < 2.2e-16
  1. What is the predicted value of high_booking_rate for a listing with the following characteristics, and how would you interpret this prediction?

New Listing:

Hint: you will have to do the same processing on this new listing as you applied to the main dataset, but feel free to do it by hand rather than using the same code as before.

According to my calculation the result would be a high booking rate of 1, given that the predicted HBR was 1.1273369 using the logistic regression model

median_price_townhouse <- airbnb %>%
  filter(property_type == "Townhouse") %>%
  summarize(median_price_per_person = median(price_per_person))
predicted_HBR = (1.186e+00)+(6.924e-02)-(30*1.257e-03)-(50*4.784e-04)-(8.899e-02)+(1.472e-01)-(3*8.126e-02)+(4*1.339e-02)-(2.231e-04)+(6.596e-02)
predicted_HBR
## [1] 1.127337
  1. Do you have any reservations about this predicted high_booking_rate? That is, would you feel sufficiently comfortable with this prediction in order to take some business action based on it? Why or why not?

No, this is a classification problem and I wouldn’t want to use Linear Regression for this type of problem. Logistic Regression is much better

3: Logistic Regression

  1. Train a Logistic Regression model using the same variables as in 2a. Report the AIC of your model.

Hint: don’t worry if you get a warning here…

Logistic regression model AIC was 5114.8:

model_b <- glm(high_booking_rate ~ cancellation_policy + cleaning_fee + price_per_person + ppp_ind + has_cleaning_fee + accommodates + bed_category + bedrooms + beds + host_total_listings_count + property_category, data = airbnb, family = 'binomial')
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
summary(model_b)
## 
## Call:
## glm(formula = high_booking_rate ~ cancellation_policy + cleaning_fee + 
##     price_per_person + ppp_ind + has_cleaning_fee + accommodates + 
##     bed_category + bedrooms + beds + host_total_listings_count + 
##     property_category, family = "binomial", data = airbnb)
## 
## Coefficients:
##                              Estimate Std. Error z value Pr(>|z|)    
## (Intercept)                 -0.957757   0.162486  -5.894 3.76e-09 ***
## cancellation_policymoderate  0.510129   0.093365   5.464 4.66e-08 ***
## cancellation_policystrict    0.445123   0.095815   4.646 3.39e-06 ***
## cleaning_fee                -0.006944   0.001299  -5.345 9.04e-08 ***
## price_per_person            -0.020410   0.002961  -6.892 5.50e-12 ***
## ppp_ind                      0.015119   0.110320   0.137  0.89099    
## has_cleaning_feeYES          0.844378   0.107386   7.863 3.75e-15 ***
## accommodates                 0.042407   0.035135   1.207  0.22744    
## bed_categoryother           -0.262019   0.208180  -1.259  0.20817    
## bedrooms                    -0.400393   0.069181  -5.788 7.14e-09 ***
## beds                         0.222971   0.050902   4.380 1.18e-05 ***
## host_total_listings_count   -0.007873   0.003214  -2.449  0.01431 *  
## property_categorycondo       0.351059   0.109369   3.210  0.00133 ** 
## property_categoryhotel       0.117113   0.351570   0.333  0.73905    
## property_categoryhouse      -0.054731   0.083288  -0.657  0.51109    
## property_categoryother      -0.149575   0.291518  -0.513  0.60789    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 5560.1  on 4935  degrees of freedom
## Residual deviance: 5082.8  on 4920  degrees of freedom
## AIC: 5114.8
## 
## Number of Fisher Scoring iterations: 8
  1. What is the coefficient for price_per_person? Provide a precise (numerical) interpretation of the coefficient.

The coefficient is -0.020410. This means that for each one-unit increase in price_per_person, the odds of being classified as high_booking_rate (Y=1) decrease by a factor of exp(-0.020410), holding all other variables constant.

  1. What is the coefficient for property_category = condo? Provide a precise (numerical) interpretation of this coefficient.

The coefficient is 0.351059. This means that for each one-unit increase in price_per_person, the odds of being classified as high_booking_rate (Y=1) increase by a factor of exp(0.351059), holding all other variables constant.

  1. Compute the predicted probability that the same listing as in #2b will have high_booking_rate = 1. Hint: you should use the predict function, but you need to specify type = “response” when predicting probabilities from logistic regression (otherwise, it will predict the value of logit). For example: predict(mymodel, newdata = mydata, type = “response”).

Predicted Y=0, because response output predicted probability was 0.33 which is below 0.50 cutoff probability

new_listing <- data.frame(
  cancellation_policy = "strict",
  cleaning_fee = 30,
  price_per_person = 50,
  ppp_ind = 1,
  has_cleaning_fee = "YES",
  accommodates = 4,
  bed_category = "bed",
  bedrooms = 3,
  beds = 4,
  host_total_listings_count = 1,
  property_category = "condo"
)

# Predict the probability of high_booking_rate = 1
predicted_probability <- predict(model_b, newdata = new_listing, type = "response")

# Print the result
print(predicted_probability)
##         1 
## 0.3366128
  1. If you were to pick one model to use for the purposes of explaining the relationship between the features and the target variable, which would it be, and why?

I would definitely pick the logistic regression simply because I feel it is the most appropriate model for this type of problem. While is coefficients are not as easy to interpret as a linear regression, I still think they are “interpretable” enough to draw valuable conclusions from them, and they are more accurate for this type of modeling problem (classification).

4: Classification and Evaluation

  1. Set the random seed to 12345. Split the data into 70% training and 30% validation sets, retrain the linear and logistic regression models using the training data only, and report the resulting R^2 and AIC, respectively.

R-squared for model_a was 0.0806 and AIC for model_b was 3585.3369

set.seed(12345)
index <- sample(1:nrow(airbnb), 0.7 * nrow(airbnb))
training_data <- airbnb[index, ]
validation_data <- airbnb[-index, ]

model_a_training <- lm(high_booking_rate ~ cancellation_policy + cleaning_fee + price_per_person + ppp_ind + has_cleaning_fee + accommodates + bed_category + bedrooms + beds + host_total_listings_count + property_category, data = training_data)


model_b_training <- glm(high_booking_rate ~ cancellation_policy + cleaning_fee + price_per_person + ppp_ind + has_cleaning_fee + accommodates + bed_category + bedrooms + beds + host_total_listings_count + property_category, data = training_data, family = 'binomial')

r_squared_training_a <- summary(model_a_training)$r.squared
aic_training_b <- AIC(model_b_training)
  1. Compute the RMSE in the training and validation sets for the linear model (do not do the classifications, just use the predicted score). Which RMSE is higher (training or validation), and does this make sense? Why or why not?

RMSE on Train = 0.41489 and RMSE on Validation = 0.4257. Since the RMSE on the validation set is higher than the RMSE on the training set, it suggests that the model performs better on the training data than on new, unseen data (validation set). This situation is expected since Models are often trained to fit the training data well, but the true test of their performance is how well they generalize to new, unseen data.

predictions_train <- predict(model_a_training, newdata = training_data)
predictions_valid <- predict(model_a_training, newdata = validation_data)
rmse_train <- sqrt(mean((training_data$high_booking_rate - predictions_train)^2))
rmse_valid <- sqrt(mean((validation_data$high_booking_rate - predictions_valid)^2))
  1. For each model (both the linear and logistic regression models learned in the training data), display the confusion matrix resulting from using a cutoff of 0.5 to do the classifications in the validation data set.

ANSWER TO QUESTION 4c HERE:

linear_predictions_valid <- predict(model_a_training, newdata = validation_data)
linear_predictions_binary <- ifelse(linear_predictions_valid >= 0.5, 1, 0)
linear_confusion_matrix <- table(Actual = validation_data$high_booking_rate, Predicted = linear_predictions_binary)
print(linear_confusion_matrix)
##       Predicted
## Actual    0    1
##      0 1101    4
##      1  372    4
logistic_predictions_valid <- predict(model_b_training, newdata = validation_data, type = "response")
logistic_predictions_binary <- ifelse(logistic_predictions_valid >= 0.5, 1, 0)
logistic_confusion_matrix <- table(Actual = validation_data$high_booking_rate, Predicted = logistic_predictions_binary)
print(logistic_confusion_matrix)
##       Predicted
## Actual    0    1
##      0 1081   24
##      1  349   27
  1. Report the accuracy, TPR, and FPR using the two confusion matrices in 4c. Which model is the most accurate?

The logistic regression model is more accurate but barely, by only 0.2% accuracy. The Logistic model has a much better TPR but a higher FPR, which is a tradeoff that would need to be evaluated in a business context.

calculate_metrics <- function(confusion_matrix) {
  TP <- confusion_matrix[2, 2]
  TN <- confusion_matrix[1, 1]
  FP <- confusion_matrix[1, 2]
  FN <- confusion_matrix[2, 1]
  accuracy <- (TP + TN) / sum(confusion_matrix)
  TPR <- TP / (TP + FN)
  FPR <- FP / (FP + TN)
  return(c(Accuracy = accuracy, TPR = TPR, FPR = FPR))
}

linear_metrics <- calculate_metrics(linear_confusion_matrix)
linear_metrics
##   Accuracy        TPR        FPR 
## 0.74611749 0.01063830 0.00361991
logistic_metrics <- calculate_metrics(logistic_confusion_matrix)
logistic_metrics
##   Accuracy        TPR        FPR 
## 0.74814315 0.07180851 0.02171946
  1. For the more accurate model, compute the accuracy, TPR, and FPR using cutoffs of .4 and .6 in the validation data. Which cutoff has the highest accuracy, highest TPR, and highest FPR?

Highest Accuracy was cutoff of 0.5 (second place was 0.6) Highest TPR was cutoff of 0.4 Highest FPR was cutoff of 0.4

## cutoff of 0.4
linear_predictions_valid <- predict(model_a_training, newdata = validation_data)
linear_predictions_binary <- ifelse(linear_predictions_valid >= 0.4, 1, 0)
linear_confusion_matrix <- table(Actual = validation_data$high_booking_rate, Predicted = linear_predictions_binary)
print(linear_confusion_matrix)
##       Predicted
## Actual    0    1
##      0 1025   80
##      1  297   79
linear_metrics <- calculate_metrics(linear_confusion_matrix)
linear_metrics
##   Accuracy        TPR        FPR 
## 0.74544227 0.21010638 0.07239819
##cutoff 0.4
logistic_predictions_valid <- predict(model_b_training, newdata = validation_data, type = "response")
logistic_predictions_binary <- ifelse(logistic_predictions_valid >= 0.4, 1, 0)
logistic_confusion_matrix <- table(Actual = validation_data$high_booking_rate, Predicted = logistic_predictions_binary)
print(logistic_confusion_matrix)
##       Predicted
## Actual   0   1
##      0 992 113
##      1 277  99
logistic_metrics <- calculate_metrics(logistic_confusion_matrix)
logistic_metrics
##  Accuracy       TPR       FPR 
## 0.7366644 0.2632979 0.1022624
## cutoff of 0.6
linear_predictions_valid <- predict(model_a_training, newdata = validation_data)
linear_predictions_binary <- ifelse(linear_predictions_valid >= 0.6, 1, 0)
linear_confusion_matrix <- table(Actual = validation_data$high_booking_rate, Predicted = linear_predictions_binary)
print(linear_confusion_matrix)
##       Predicted
## Actual    0    1
##      0 1104    1
##      1  375    1
linear_metrics <- calculate_metrics(linear_confusion_matrix)
linear_metrics
##     Accuracy          TPR          FPR 
## 0.7461174882 0.0026595745 0.0009049774
##cutoff 0.6
logistic_predictions_valid <- predict(model_b_training, newdata = validation_data, type = "response")
logistic_predictions_binary <- ifelse(logistic_predictions_valid >= 0.6, 1, 0)
logistic_confusion_matrix <- table(Actual = validation_data$high_booking_rate, Predicted = logistic_predictions_binary)
print(logistic_confusion_matrix)
##       Predicted
## Actual    0    1
##      0 1104    1
##      1  375    1
logistic_metrics <- calculate_metrics(logistic_confusion_matrix)
logistic_metrics
##     Accuracy          TPR          FPR 
## 0.7461174882 0.0026595745 0.0009049774
  1. In your opinion, which cutoff of the three (.4, .5, .6) yields the best results for this application? Explain your reasoning.

Given the results above, I believe the best cutoff is 0.4 because it achieved the best results for both models. The accuracy was quite high but this was the same for almost all model types and cutoff choices. The real reason to pick the cutoff of 0.4 is because it gave the best TPR. If our goal is to accurately predict the true High_Booking_Rate units, this was the best model specification to pick them out. Especially when using the logistic regression, this was the best model type and cutoff choice for this application.