The goal of this homework is hands-on practice with data exploration and cleaning, classification, and model selection. You will:
The Assignment
The data in the accompanying file “dc_airbnb_file.csv” (posted on Canvas) contains data about 4936 Airbnb.com listings from the Washington, DC area. This is a small subset of the data that you will eventually use for the class project. The variables in this dataset should be self-explanatory but we have also provided a data dictionary.
Your task is to develop models to predict the target variable “high_booking_rate”, which labels whether a listing is popular (i.e. spends most of the time booked) or not.
Please answer the questions below clearly and concisely, providing tables or plots where applicable. Turn in a well-formatted compiled HTML document using R Markdown, containing clear answers to the questions and R code in the appropriate places.
RUBRIC: To receive a passing score on this assignment, you must do the following:
Note that this assignment is somewhat open-ended and there are many ways to answer these questions. I don’t require that we have exactly the same answers in order for you to receive full credit.
airbnb <- read_csv("dc_airbnb_hw1.csv") #read the dataset in R
## Rows: 4936 Columns: 12
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr (7): name, bed_type, cancellation_policy, cleaning_fee, price, property_...
## dbl (5): accommodates, bedrooms, beds, host_total_listings_count, high_booki...
##
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
names(airbnb) #variables used in dataset
## [1] "name" "accommodates"
## [3] "bed_type" "bedrooms"
## [5] "beds" "cancellation_policy"
## [7] "cleaning_fee" "host_total_listings_count"
## [9] "price" "property_type"
## [11] "room_type" "high_booking_rate"
What is the mean of the accommodates variable?
ANSWER: The mean number of people that can be accommodated in a listing in this dataset is 3.522893.
age_mean <- airbnb %>%
summarise(mean_accommodates = mean(accommodates))
Hint: there are lots of ways to deal with cleaning_fee and price. I like the parse_number function from the readr library.
Code below:
airbnb <- airbnb %>%
mutate(cancellation_policy = if_else(cancellation_policy %in% c('strict', 'super_strict_30'), 'strict', as.character(cancellation_policy))) %>%
mutate(cleaning_fee = parse_number(cleaning_fee), price = parse_number(price)) %>%
mutate(cleaning_fee = if_else(is.na(cleaning_fee), 0, cleaning_fee),
price = if_else(is.na(price), 0, price)) %>%
mutate(beds = if_else(is.na(beds), mean(beds, na.rm = TRUE), beds),
bedrooms = if_else(is.na(bedrooms), mean(bedrooms, na.rm=TRUE), bedrooms),
host_total_listings_count = if_else(is.na(host_total_listings_count), mean(host_total_listings_count, na.rm=TRUE), host_total_listings_count))
price_per_person is the nightly price per accommodates
has_cleaning_fee is YES if there is a cleaning fee, and NO otherwise
bed_category is “bed” if the bed_type is Real Bed and “other” otherwise
property_category has the following values:
make sure to convert property_category to a factor!
ppp_ind is 1 if the price_per_person is greater than the median for the property_category, and 0 otherwise
Hint: there are lots of ways to do the property_type variable, but case_when might be a useful function that’s part of the tidyverse.
Variables created below:
airbnb <- airbnb %>%
mutate(
price_per_person = price / accommodates,
has_cleaning_fee = if_else(cleaning_fee > 0, "YES", "NO"),
bed_category = if_else(bed_type == "Real Bed", "bed", "other"),
property_category = case_when(
property_type %in% c("Apartment", "Serviced apartment", "Loft") ~ "apartment",
property_type %in% c("Bed & Breakfast", "Boutique hotel", "Hostel") ~ "hotel",
property_type %in% c("Townhouse", "Condominium") ~ "condo",
property_type %in% c("Bungalow", "House") ~ "house",
TRUE ~ "other"
),
ppp_ind = as.integer(price_per_person > median(price_per_person))
)
# Convert property_category to a factor
airbnb$property_category <- as.factor(airbnb$property_category)
airbnb$property_type <- as.factor(airbnb$property_type)
airbnb <- airbnb %>%
mutate(
bed_type = as.factor(bed_type),
cancellation_policy = as.factor(cancellation_policy),
room_type = as.factor(room_type)
)
There seems to be a relationship between price per person and high booing rate. For units with a low booking rate, the average price per person was higher and had more variability as well. When the unit had a high booking rate the average price per person was lower, and there was less variability in the price per person for those units.
# Boxtplot regular
ggplot(airbnb, aes(x = as.factor(high_booking_rate), y = price_per_person)) +
geom_boxplot() +
labs(title = "Boxplot of price_per_person by high_booking_rate",
x = "high_booking_rate",
y = "price_per_person")
# Boxplot for Log() version
ggplot(airbnb, aes(x = as.factor(high_booking_rate), y = log(price_per_person))) +
geom_boxplot() +
labs(title = "Boxplot of log(price_per_person) by high_booking_rate",
x = "high_booking_rate",
y = "log(price_per_person)")
(hint: prop.table might be especially helpful here)
Yes, it seems that condos tend to get high booking rates more often than the average property type, and it also seems that hotels tend to get high booking rates less often than the average property type.
two_way_table <- table(airbnb$property_category, airbnb$high_booking_rate)
proportions <- prop.table(two_way_table, margin = 1) * 100
proportions
##
## 0 1
## apartment 74.97207 25.02793
## condo 69.35484 30.64516
## hotel 80.64516 19.35484
## house 76.74568 23.25432
## other 71.42857 28.57143
R^2 was 0.07788. The linear model was created using high booking rate as target variable, and re-classified this variable as numeric for the model to work (was getting r error because I had it as a factor)
model_a <- lm(high_booking_rate ~ cancellation_policy + cleaning_fee + price_per_person + ppp_ind + has_cleaning_fee + accommodates + bed_category + bedrooms + beds + host_total_listings_count + property_category, data = airbnb)
summary(model_a)
##
## Call:
## lm(formula = high_booking_rate ~ cancellation_policy + cleaning_fee +
## price_per_person + ppp_ind + has_cleaning_fee + accommodates +
## bed_category + bedrooms + beds + host_total_listings_count +
## property_category, data = airbnb)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.0122 -0.2869 -0.1763 0.4341 0.9940
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 1.859e-01 2.104e-02 8.834 < 2e-16 ***
## cancellation_policymoderate 8.539e-02 1.582e-02 5.399 7.03e-08 ***
## cancellation_policystrict 6.924e-02 1.595e-02 4.342 1.44e-05 ***
## cleaning_fee -1.257e-03 1.897e-04 -6.630 3.71e-11 ***
## price_per_person -4.784e-04 1.408e-04 -3.399 0.000683 ***
## ppp_ind -8.899e-02 1.391e-02 -6.399 1.71e-10 ***
## has_cleaning_feeYES 1.472e-01 1.732e-02 8.498 < 2e-16 ***
## accommodates 1.339e-02 5.981e-03 2.239 0.025187 *
## bed_categoryother -3.722e-02 3.490e-02 -1.066 0.286302
## bedrooms -8.126e-02 1.190e-02 -6.827 9.73e-12 ***
## beds 4.362e-02 8.970e-03 4.863 1.19e-06 ***
## host_total_listings_count -2.231e-04 7.148e-05 -3.122 0.001809 **
## property_categorycondo 6.596e-02 1.982e-02 3.328 0.000882 ***
## property_categoryhotel 1.446e-02 5.410e-02 0.267 0.789317
## property_categoryhouse -4.866e-05 1.438e-02 -0.003 0.997300
## property_categoryother -1.082e-02 5.111e-02 -0.212 0.832356
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.4169 on 4920 degrees of freedom
## Multiple R-squared: 0.07788, Adjusted R-squared: 0.07507
## F-statistic: 27.7 on 15 and 4920 DF, p-value: < 2.2e-16
New Listing:
Hint: you will have to do the same processing on this new listing as you applied to the main dataset, but feel free to do it by hand rather than using the same code as before.
According to my calculation the result would be a high booking rate of 1, given that the predicted HBR was 1.1273369 using the logistic regression model
median_price_townhouse <- airbnb %>%
filter(property_type == "Townhouse") %>%
summarize(median_price_per_person = median(price_per_person))
predicted_HBR = (1.186e+00)+(6.924e-02)-(30*1.257e-03)-(50*4.784e-04)-(8.899e-02)+(1.472e-01)-(3*8.126e-02)+(4*1.339e-02)-(2.231e-04)+(6.596e-02)
predicted_HBR
## [1] 1.127337
No, this is a classification problem and I wouldn’t want to use Linear Regression for this type of problem. Logistic Regression is much better
Hint: don’t worry if you get a warning here…
Logistic regression model AIC was 5114.8:
model_b <- glm(high_booking_rate ~ cancellation_policy + cleaning_fee + price_per_person + ppp_ind + has_cleaning_fee + accommodates + bed_category + bedrooms + beds + host_total_listings_count + property_category, data = airbnb, family = 'binomial')
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
summary(model_b)
##
## Call:
## glm(formula = high_booking_rate ~ cancellation_policy + cleaning_fee +
## price_per_person + ppp_ind + has_cleaning_fee + accommodates +
## bed_category + bedrooms + beds + host_total_listings_count +
## property_category, family = "binomial", data = airbnb)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -0.957757 0.162486 -5.894 3.76e-09 ***
## cancellation_policymoderate 0.510129 0.093365 5.464 4.66e-08 ***
## cancellation_policystrict 0.445123 0.095815 4.646 3.39e-06 ***
## cleaning_fee -0.006944 0.001299 -5.345 9.04e-08 ***
## price_per_person -0.020410 0.002961 -6.892 5.50e-12 ***
## ppp_ind 0.015119 0.110320 0.137 0.89099
## has_cleaning_feeYES 0.844378 0.107386 7.863 3.75e-15 ***
## accommodates 0.042407 0.035135 1.207 0.22744
## bed_categoryother -0.262019 0.208180 -1.259 0.20817
## bedrooms -0.400393 0.069181 -5.788 7.14e-09 ***
## beds 0.222971 0.050902 4.380 1.18e-05 ***
## host_total_listings_count -0.007873 0.003214 -2.449 0.01431 *
## property_categorycondo 0.351059 0.109369 3.210 0.00133 **
## property_categoryhotel 0.117113 0.351570 0.333 0.73905
## property_categoryhouse -0.054731 0.083288 -0.657 0.51109
## property_categoryother -0.149575 0.291518 -0.513 0.60789
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 5560.1 on 4935 degrees of freedom
## Residual deviance: 5082.8 on 4920 degrees of freedom
## AIC: 5114.8
##
## Number of Fisher Scoring iterations: 8
The coefficient is -0.020410. This means that for each one-unit increase in price_per_person, the odds of being classified as high_booking_rate (Y=1) decrease by a factor of exp(-0.020410), holding all other variables constant.
The coefficient is 0.351059. This means that for each one-unit increase in price_per_person, the odds of being classified as high_booking_rate (Y=1) increase by a factor of exp(0.351059), holding all other variables constant.
Predicted Y=0, because response output predicted probability was 0.33 which is below 0.50 cutoff probability
new_listing <- data.frame(
cancellation_policy = "strict",
cleaning_fee = 30,
price_per_person = 50,
ppp_ind = 1,
has_cleaning_fee = "YES",
accommodates = 4,
bed_category = "bed",
bedrooms = 3,
beds = 4,
host_total_listings_count = 1,
property_category = "condo"
)
# Predict the probability of high_booking_rate = 1
predicted_probability <- predict(model_b, newdata = new_listing, type = "response")
# Print the result
print(predicted_probability)
## 1
## 0.3366128
I would definitely pick the logistic regression simply because I feel it is the most appropriate model for this type of problem. While is coefficients are not as easy to interpret as a linear regression, I still think they are “interpretable” enough to draw valuable conclusions from them, and they are more accurate for this type of modeling problem (classification).
R-squared for model_a was 0.0806 and AIC for model_b was 3585.3369
set.seed(12345)
index <- sample(1:nrow(airbnb), 0.7 * nrow(airbnb))
training_data <- airbnb[index, ]
validation_data <- airbnb[-index, ]
model_a_training <- lm(high_booking_rate ~ cancellation_policy + cleaning_fee + price_per_person + ppp_ind + has_cleaning_fee + accommodates + bed_category + bedrooms + beds + host_total_listings_count + property_category, data = training_data)
model_b_training <- glm(high_booking_rate ~ cancellation_policy + cleaning_fee + price_per_person + ppp_ind + has_cleaning_fee + accommodates + bed_category + bedrooms + beds + host_total_listings_count + property_category, data = training_data, family = 'binomial')
r_squared_training_a <- summary(model_a_training)$r.squared
aic_training_b <- AIC(model_b_training)
RMSE on Train = 0.41489 and RMSE on Validation = 0.4257. Since the RMSE on the validation set is higher than the RMSE on the training set, it suggests that the model performs better on the training data than on new, unseen data (validation set). This situation is expected since Models are often trained to fit the training data well, but the true test of their performance is how well they generalize to new, unseen data.
predictions_train <- predict(model_a_training, newdata = training_data)
predictions_valid <- predict(model_a_training, newdata = validation_data)
rmse_train <- sqrt(mean((training_data$high_booking_rate - predictions_train)^2))
rmse_valid <- sqrt(mean((validation_data$high_booking_rate - predictions_valid)^2))
ANSWER TO QUESTION 4c HERE:
linear_predictions_valid <- predict(model_a_training, newdata = validation_data)
linear_predictions_binary <- ifelse(linear_predictions_valid >= 0.5, 1, 0)
linear_confusion_matrix <- table(Actual = validation_data$high_booking_rate, Predicted = linear_predictions_binary)
print(linear_confusion_matrix)
## Predicted
## Actual 0 1
## 0 1101 4
## 1 372 4
logistic_predictions_valid <- predict(model_b_training, newdata = validation_data, type = "response")
logistic_predictions_binary <- ifelse(logistic_predictions_valid >= 0.5, 1, 0)
logistic_confusion_matrix <- table(Actual = validation_data$high_booking_rate, Predicted = logistic_predictions_binary)
print(logistic_confusion_matrix)
## Predicted
## Actual 0 1
## 0 1081 24
## 1 349 27
The logistic regression model is more accurate but barely, by only 0.2% accuracy. The Logistic model has a much better TPR but a higher FPR, which is a tradeoff that would need to be evaluated in a business context.
calculate_metrics <- function(confusion_matrix) {
TP <- confusion_matrix[2, 2]
TN <- confusion_matrix[1, 1]
FP <- confusion_matrix[1, 2]
FN <- confusion_matrix[2, 1]
accuracy <- (TP + TN) / sum(confusion_matrix)
TPR <- TP / (TP + FN)
FPR <- FP / (FP + TN)
return(c(Accuracy = accuracy, TPR = TPR, FPR = FPR))
}
linear_metrics <- calculate_metrics(linear_confusion_matrix)
linear_metrics
## Accuracy TPR FPR
## 0.74611749 0.01063830 0.00361991
logistic_metrics <- calculate_metrics(logistic_confusion_matrix)
logistic_metrics
## Accuracy TPR FPR
## 0.74814315 0.07180851 0.02171946
Highest Accuracy was cutoff of 0.5 (second place was 0.6) Highest TPR was cutoff of 0.4 Highest FPR was cutoff of 0.4
## cutoff of 0.4
linear_predictions_valid <- predict(model_a_training, newdata = validation_data)
linear_predictions_binary <- ifelse(linear_predictions_valid >= 0.4, 1, 0)
linear_confusion_matrix <- table(Actual = validation_data$high_booking_rate, Predicted = linear_predictions_binary)
print(linear_confusion_matrix)
## Predicted
## Actual 0 1
## 0 1025 80
## 1 297 79
linear_metrics <- calculate_metrics(linear_confusion_matrix)
linear_metrics
## Accuracy TPR FPR
## 0.74544227 0.21010638 0.07239819
##cutoff 0.4
logistic_predictions_valid <- predict(model_b_training, newdata = validation_data, type = "response")
logistic_predictions_binary <- ifelse(logistic_predictions_valid >= 0.4, 1, 0)
logistic_confusion_matrix <- table(Actual = validation_data$high_booking_rate, Predicted = logistic_predictions_binary)
print(logistic_confusion_matrix)
## Predicted
## Actual 0 1
## 0 992 113
## 1 277 99
logistic_metrics <- calculate_metrics(logistic_confusion_matrix)
logistic_metrics
## Accuracy TPR FPR
## 0.7366644 0.2632979 0.1022624
## cutoff of 0.6
linear_predictions_valid <- predict(model_a_training, newdata = validation_data)
linear_predictions_binary <- ifelse(linear_predictions_valid >= 0.6, 1, 0)
linear_confusion_matrix <- table(Actual = validation_data$high_booking_rate, Predicted = linear_predictions_binary)
print(linear_confusion_matrix)
## Predicted
## Actual 0 1
## 0 1104 1
## 1 375 1
linear_metrics <- calculate_metrics(linear_confusion_matrix)
linear_metrics
## Accuracy TPR FPR
## 0.7461174882 0.0026595745 0.0009049774
##cutoff 0.6
logistic_predictions_valid <- predict(model_b_training, newdata = validation_data, type = "response")
logistic_predictions_binary <- ifelse(logistic_predictions_valid >= 0.6, 1, 0)
logistic_confusion_matrix <- table(Actual = validation_data$high_booking_rate, Predicted = logistic_predictions_binary)
print(logistic_confusion_matrix)
## Predicted
## Actual 0 1
## 0 1104 1
## 1 375 1
logistic_metrics <- calculate_metrics(logistic_confusion_matrix)
logistic_metrics
## Accuracy TPR FPR
## 0.7461174882 0.0026595745 0.0009049774
Given the results above, I believe the best cutoff is 0.4 because it achieved the best results for both models. The accuracy was quite high but this was the same for almost all model types and cutoff choices. The real reason to pick the cutoff of 0.4 is because it gave the best TPR. If our goal is to accurately predict the true High_Booking_Rate units, this was the best model specification to pick them out. Especially when using the logistic regression, this was the best model type and cutoff choice for this application.