Problem Statement Travel is a crucial industry in the economy, since tourism revenue is crucial for many countries and cities around the world. This revenue goes directly to supporting local businesses, generates jobs, and provides tax income that promotes cultural and economic development in communities. Tourism reliant countries rely on travel to boost their GDP and keep the economy running. It is important that hotel chains understand if time of the month influences, and if that relationship is more significant than the other variables in the dataset
Approach We plan to address this problem by understanding what the relationship is between hotel stays during specific times of the year. We want to determine if there is a significant relationship between month of the year and the number of hotel stays during this time. We will be using the hotels dataset to analyze this issue and determine whether month of the year determines the total amount of hotel stays. We will do this in a few ways:
This analysis will allow hotel chains and tourism reliant companies determine whether it is appropriate to anticipate specific months of the year that might bring a higher number of tourists. Do hotels that plan to renovate their properties need to do this in the winter months when there are fewer tourists? Do restaurants, shops, and bars need to hire more employees for the summer months? Should governments increase or decrease the amount of tax income expected due to lower-than-expected tourists in the summer months? This analysis will seek to answer these questions and similar problems posed by touris cycles across the year.
For this analysis, we plan to use these different packages:
ggplot to visualize the data knitr to generate the report kknn to perform K Nearest Neighbor models tidyverse to perform data cleaning
The data we will use for the analysis is the hotels data set. This dataset is derived from an open hotel booking demand source, courtesy of Antonio, Almeida, and Nunes, 2019. This datatset contains information about various hotel stays from around the world with information such as Month, Year, and Week when the stay occurred, how many weekend and week nights the hotel lasted, the country where the stay occured, and other pertinent information. The data has a total of 119,391 and rows of data with 32 different variables. Missing values occur as instances of NA or also appear as “NULL” values. It is unclear at this point whether NULL values occur due to incomplete data at the time of research, since these “NULL” values differ from NA values in the data
## [1] "The number of NA values in this dataset is: 4"
## [1] "The number of NULL values in this dataset is: 129421"
Now that we have some facts about the data it’s time to perform the data cleaning process. First, let’s take a look at the variables in the data set:
summary(hotels)
## hotel is_canceled lead_time arrival_date_year
## Length:119390 Min. :0.0000 Min. : 0 Min. :2015
## Class :character 1st Qu.:0.0000 1st Qu.: 18 1st Qu.:2016
## Mode :character Median :0.0000 Median : 69 Median :2016
## Mean :0.3704 Mean :104 Mean :2016
## 3rd Qu.:1.0000 3rd Qu.:160 3rd Qu.:2017
## Max. :1.0000 Max. :737 Max. :2017
##
## arrival_date_month arrival_date_week_number arrival_date_day_of_month
## Length:119390 Min. : 1.00 Min. : 1.0
## Class :character 1st Qu.:16.00 1st Qu.: 8.0
## Mode :character Median :28.00 Median :16.0
## Mean :27.17 Mean :15.8
## 3rd Qu.:38.00 3rd Qu.:23.0
## Max. :53.00 Max. :31.0
##
## stays_in_weekend_nights stays_in_week_nights adults
## Min. : 0.0000 Min. : 0.0 Min. : 0.000
## 1st Qu.: 0.0000 1st Qu.: 1.0 1st Qu.: 2.000
## Median : 1.0000 Median : 2.0 Median : 2.000
## Mean : 0.9276 Mean : 2.5 Mean : 1.856
## 3rd Qu.: 2.0000 3rd Qu.: 3.0 3rd Qu.: 2.000
## Max. :19.0000 Max. :50.0 Max. :55.000
##
## children babies meal country
## Min. : 0.0000 Min. : 0.000000 Length:119390 Length:119390
## 1st Qu.: 0.0000 1st Qu.: 0.000000 Class :character Class :character
## Median : 0.0000 Median : 0.000000 Mode :character Mode :character
## Mean : 0.1039 Mean : 0.007949
## 3rd Qu.: 0.0000 3rd Qu.: 0.000000
## Max. :10.0000 Max. :10.000000
## NA's :4
## market_segment distribution_channel is_repeated_guest
## Length:119390 Length:119390 Min. :0.00000
## Class :character Class :character 1st Qu.:0.00000
## Mode :character Mode :character Median :0.00000
## Mean :0.03191
## 3rd Qu.:0.00000
## Max. :1.00000
##
## previous_cancellations previous_bookings_not_canceled reserved_room_type
## Min. : 0.00000 Min. : 0.0000 Length:119390
## 1st Qu.: 0.00000 1st Qu.: 0.0000 Class :character
## Median : 0.00000 Median : 0.0000 Mode :character
## Mean : 0.08712 Mean : 0.1371
## 3rd Qu.: 0.00000 3rd Qu.: 0.0000
## Max. :26.00000 Max. :72.0000
##
## assigned_room_type booking_changes deposit_type agent
## Length:119390 Min. : 0.0000 Length:119390 Length:119390
## Class :character 1st Qu.: 0.0000 Class :character Class :character
## Mode :character Median : 0.0000 Mode :character Mode :character
## Mean : 0.2211
## 3rd Qu.: 0.0000
## Max. :21.0000
##
## company days_in_waiting_list customer_type adr
## Length:119390 Min. : 0.000 Length:119390 Min. : -6.38
## Class :character 1st Qu.: 0.000 Class :character 1st Qu.: 69.29
## Mode :character Median : 0.000 Mode :character Median : 94.58
## Mean : 2.321 Mean : 101.83
## 3rd Qu.: 0.000 3rd Qu.: 126.00
## Max. :391.000 Max. :5400.00
##
## required_car_parking_spaces total_of_special_requests reservation_status
## Min. :0.00000 Min. :0.0000 Length:119390
## 1st Qu.:0.00000 1st Qu.:0.0000 Class :character
## Median :0.00000 Median :0.0000 Mode :character
## Mean :0.06252 Mean :0.5714
## 3rd Qu.:0.00000 3rd Qu.:1.0000
## Max. :8.00000 Max. :5.0000
##
## reservation_status_date
## Length:119390
## Class :character
## Mode :character
##
##
##
##
First, let’s see if we have any NA values, and correct them if so:
## hotel is_canceled
## 0 0
## lead_time arrival_date_year
## 0 0
## arrival_date_month arrival_date_week_number
## 0 0
## arrival_date_day_of_month stays_in_weekend_nights
## 0 0
## stays_in_week_nights adults
## 0 0
## children babies
## 4 0
## meal country
## 0 0
## market_segment distribution_channel
## 0 0
## is_repeated_guest previous_cancellations
## 0 0
## previous_bookings_not_canceled reserved_room_type
## 0 0
## assigned_room_type booking_changes
## 0 0
## deposit_type agent
## 0 0
## company days_in_waiting_list
## 0 0
## customer_type adr
## 0 0
## required_car_parking_spaces total_of_special_requests
## 0 0
## reservation_status reservation_status_date
## 0 0
Looks like the children variable has 4 NA values. Let’s correct them by converting them to the median children value:
## [1] "The median number of children is: 0"
Let’s check if there are any missing variables in the children variable:
## hotel is_canceled
## 0 0
## lead_time arrival_date_year
## 0 0
## arrival_date_month arrival_date_week_number
## 0 0
## arrival_date_day_of_month stays_in_weekend_nights
## 0 0
## stays_in_week_nights adults
## 0 0
## children babies
## 0 0
## meal country
## 0 0
## market_segment distribution_channel
## 0 0
## is_repeated_guest previous_cancellations
## 0 0
## previous_bookings_not_canceled reserved_room_type
## 0 0
## assigned_room_type booking_changes
## 0 0
## deposit_type agent
## 0 0
## company days_in_waiting_list
## 0 0
## customer_type adr
## 0 0
## required_car_parking_spaces total_of_special_requests
## 0 0
## reservation_status reservation_status_date
## 0 0
Now, let’s talk about the NULL values we talked about earlier. The article posted by Antonio, Almeida, and Nunes, 2019 states that “the PMS assured no missing data exists in its database tables. However, in some categorical variables like Agent or Company, “NULL” is presented as one of the categories. This should not be considered a missing value, but rather as “not applicable”. For example, if a booking “Agent” is defined as “NULL” it means that the booking did not came from a travel agent.” Due to this, there is no cleaning necessary, since this is clearly part of the original data. Due to this, we will leave “NULL” values as they are. However, I will convert them to “Unknown” to make the analysis simpler.
Now, let’s look at the outliers in the data. This will allow us to understand how distorted the data is from the means, and that could help understand how effective the prediction models will be later in the report. First, I will split the hotels data into another dataset called hotels_num with only the numeric variables to simplify the visualization.
Looks like the lead_time, previous_bookings_not_cancelled, adults variable has significant outliers, and adr has a big outlier equal to ~5000. Looking at the summary table, O also saw that children and babies had a max value of 10. Although there may be a chance that 10 children and 10 babies stayed in a hotel room, there is a possibility these are outliers in the data. I will also analyze the stays_in_week_nights and stays_in_weekend_night variables since they are of interest to this report. Let’s take a look at the boxplots for these variables:
First, let’s fix the adr value of 5400. Since we want to maintain the integrity of the original data, we will create another data file to address the outliers. This will allow us to perform models on the data with outliers and without outliers.
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## -6.38 69.29 94.58 101.79 126.00 510.00
Now that we fixed that, let’s take a look at our boxplot again:
Looks like lead_time and adr have multiple outliers. Let’s truncate the outliers to 400 for lead_time and 230 for adr:
That looks better. Now, let’s look at the clean data in some visualizations. First, we will look at the normality of both stays_in_week_nights and stays_in_weekend_nights:
#ggplot(hotels_no_outliers, aes(x=stays_in_week_nights))+ # geom_histogram(binwidth = 3, fill = “green”, alpha=0.7) + # ggtitle(“Distribution of stays_in_week_nights”) + # xlab(“stays_in_week_nights”) + # ylab(“Frequency”)
It appears like the data is not as normal as we would like. This could point to innefective linear models, since an assumption of linear regression is that the data should be as normal as possible. Now we will look at stays_in_weekend_nights:
Once again, this data is not normal. The same issues could arise when performing a linear regression model on this variable.
Now, one big focus on this report is to analyze whether there is a correlation between hotel stays and time of the year. We will look at a bar graph to determine whether there is a relationship between both stays_in_week_nights & stays_in_weekend_nights and arrival_date_year & arrival_date_month.
First let’s do week nights:
Looks like week night stays peaked in 2016 at around 175,000 stays, with 2017 lower at 110,000 but still higher than ~50,000 in 2015. Let’s look at month:
It looks like the summer months, specifically July & August have the highest amount of booked stays. From January to August stays increase exponentially, with a large decline in September that continues declining until December. This is incredibly important information. We can see here that tourism companies experience most of their stays during the summer months, although Spring months also bring a considerable amount of tourists compared to Winter months.
Let’s see if weekend nights display the same data:
This data aligns with stays in week nights. 2016 is the peak year of hotel stays, with 2017 having the second highest amount of stays, followed by 2016
Now months:
Unsurprisingly, the data looks very similar to week nights, with August bringing the highest quantity of weekend stays. Surprisingly, however, is that April happens to have higher total stays than May & June. We also see that compared to week night stays, December has higher stays than November. These trend could result from travelers making more weekend trips during these months that they may not be able to make during the week.
Now that we cleaned the data and looked at some results, let’s take a look at a summary of our new dataset without outliers and missing values. The condensed table below displays all the variables with the first 6 results. We see that the country & agent variables show NULL values, but earlier in the report we stated that those NULL values are part of the data, and therefore we will not make any changes to them.
| hotel | is_canceled | lead_time | arrival_date_year | arrival_date_month | arrival_date_week_number | arrival_date_day_of_month | stays_in_weekend_nights | stays_in_week_nights | adults | children | babies | meal | country | market_segment | distribution_channel | is_repeated_guest | previous_cancellations | previous_bookings_not_canceled | reserved_room_type | assigned_room_type | booking_changes | deposit_type | agent | company | days_in_waiting_list | customer_type | adr | required_car_parking_spaces | total_of_special_requests | reservation_status | reservation_status_date |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Resort Hotel | 0 | 342 | 2015 | July | 27 | 1 | 0 | 0 | 2 | 0 | 0 | BB | PRT | Direct | Direct | 0 | 0 | 0 | C | C | 3 | No Deposit | NULL | NULL | 0 | Transient | 0 | 0 | 0 | Check-Out | 2015-07-01 |
| Resort Hotel | 0 | 370 | 2015 | July | 27 | 1 | 0 | 0 | 2 | 0 | 0 | BB | PRT | Direct | Direct | 0 | 0 | 0 | C | C | 4 | No Deposit | NULL | NULL | 0 | Transient | 0 | 0 | 0 | Check-Out | 2015-07-01 |
| Resort Hotel | 0 | 7 | 2015 | July | 27 | 1 | 0 | 1 | 1 | 0 | 0 | BB | GBR | Direct | Direct | 0 | 0 | 0 | A | C | 0 | No Deposit | NULL | NULL | 0 | Transient | 75 | 0 | 0 | Check-Out | 2015-07-02 |
| Resort Hotel | 0 | 13 | 2015 | July | 27 | 1 | 0 | 1 | 1 | 0 | 0 | BB | GBR | Corporate | Corporate | 0 | 0 | 0 | A | A | 0 | No Deposit | 304 | NULL | 0 | Transient | 75 | 0 | 0 | Check-Out | 2015-07-02 |
| Resort Hotel | 0 | 14 | 2015 | July | 27 | 1 | 0 | 2 | 2 | 0 | 0 | BB | GBR | Online TA | TA/TO | 0 | 0 | 0 | A | A | 0 | No Deposit | 240 | NULL | 0 | Transient | 98 | 0 | 1 | Check-Out | 2015-07-03 |
| Resort Hotel | 0 | 14 | 2015 | July | 27 | 1 | 0 | 2 | 2 | 0 | 0 | BB | GBR | Online TA | TA/TO | 0 | 0 | 0 | A | A | 0 | No Deposit | 240 | NULL | 0 | Transient | 98 | 0 | 1 | Check-Out | 2015-07-03 |
Now, let’s talk about our variables of interest. Our mission in this report is to analyze what the relationship is between hotel stays during the year. To solve this problem, we will be looking closely at 4 variables of interest: stays_in_week_nights, stays_in_weekend_nights, arrival_date_year, and arrival_date_month. First, let’s take a look at week night stays:
It looks like the majority of week night stays are from 0-5 nights. After this, the majority of stays greater than 5 declines to below 5,000 total. Let’s look at weekend nights now:
It appears that with weekend nights, the largest number of weekend nights is 0. This is important to note, since it could imply that a majority of travel occurs during the weekday. This will become an important part of the analysis later on.
Now let’s look at month and year:
Here, it looks like the highest amount of stays occured during May 2017. Let’s take a look at week night stays:
It appears that January - June had no bookings during 2015. This inevitably proves why 2015 has lower stays than either 2016 or 2017.
To uncover new information not evident from this data set, I plan to use correlation plots and correlation tables s to look at the relationships between variables in the data, especially stays_in_week_nights and stays_in_weekend_nights. This will allow us to understand whether year and month are viable indicators for predicting how many hotel stays will happen. Similarly, using visualizations has so far allowed us to look at the data on a deeper level, understanding not just how many hotel stays happened in a certain month and/or year, but how that compares over the 3 years this data was recorded. We must scrutinize and analyze every aspect of the data to understand critical relationships that are not evident from a brief glance at the data. We will also employ statistical models and methods to derive relationships that simply could not be obtained from tables and visualizations employed so far in the report.
Correlation between variables: Now, we’d like to understand what the relationships are between variables of significance and the rest of the dataset:
It appears as though there is not a very strong correlation between week night stays and arrival year (0.03) or between weekend night stays and arrival year (0.02). However, the correlation does exist, and we plan to explore that correlation and determine how effective it is at predicting week night and weekend night stays.
Currently, we do not know the statistical relationships between week night/ weekend night stays and the other variables in the data, specifically arrival year and arrival month. We must understand how our variables of interest are affected by other variables to make accurate predictions in the data. We are going to have to understand how to apply linear and KNN models to understand how effectively we can make predictions in the data. We will have to know how the predicted values compare to actual values in the data, and if there are interactions in the variables that will aid us in building accurate and reliable models.
We plan to slice the data into training and testing sets. This will allow us to test the MSE of the training data and understand how effective that model is in predicting out-of-scope data. This is meant to ensure the models are unbiased and statistically significant. Failure to do this would question the integrity of our models, and it is standard statistical procedure when building effective models.
Now we will conduct linear models to understand if they can accurately predict this data. First, the data must be split into testing and training data sets. There is a potential problem that the same levels will not appear in both the training and testing sets, so I will convert the variables in question into factors to eliminate this problem.
Let’s convert the data into training and testing data sets
# We will use the random seed 2023
set.seed(2023)
train_indices <- sample(1:NROW(hotels_no_outliers),NROW(hotels_no_outliers)*.7)
# Training Data
train_data <- hotels_no_outliers[train_indices, ]
# Testing Data
test_data <- hotels_no_outliers[-train_indices, ]
In sample LM Now we will begin with the first LM. I will begin by modelling stays_in_week_nights with arrival year and month as the predictors with no interaction term. “Actual” represents data from the actual training data, and predicted is from the linear model. This format extends to all the linear models we will look at. As columns are added to the right, they will represent the predicted values from each model we analyze. This is for the in-scope model:
## Actual predicted
## 21968 0 2.513892
## 25961 1 2.766916
## 73459 1 2.928321
## 106028 4 2.239232
## 96753 2 2.589036
## 9757 2 2.407038
##
## Call:
## lm(formula = stays_in_week_nights ~ arrival_date_year + arrival_date_month,
## data = train_data)
##
## Residuals:
## Min 1Q Median 3Q Max
## -2.928 -1.296 -0.407 0.704 47.233
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -221.45015 23.11937 -9.579 < 2e-16 ***
## arrival_date_year 0.11089 0.01146 9.672 < 2e-16 ***
## arrival_date_monthFebruary 0.02570 0.03862 0.666 0.506
## arrival_date_monthMarch 0.41125 0.03724 11.043 < 2e-16 ***
## arrival_date_monthApril 0.24751 0.03642 6.796 1.09e-11 ***
## arrival_date_monthMay 0.22872 0.03599 6.356 2.08e-10 ***
## arrival_date_monthJune 0.51201 0.03658 13.998 < 2e-16 ***
## arrival_date_monthJuly 0.66427 0.03597 18.470 < 2e-16 ***
## arrival_date_monthAugust 0.71479 0.03571 20.018 < 2e-16 ***
## arrival_date_monthSeptember 0.48639 0.03895 12.489 < 2e-16 ***
## arrival_date_monthOctober 0.17195 0.03837 4.481 7.43e-06 ***
## arrival_date_monthNovember 0.34234 0.04178 8.194 2.56e-16 ***
## arrival_date_monthDecember 0.30440 0.04202 7.244 4.38e-13 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 1.895 on 83560 degrees of freedom
## Multiple R-squared: 0.01406, Adjusted R-squared: 0.01391
## F-statistic: 99.27 on 12 and 83560 DF, p-value: < 2.2e-16
Looking at this summary table, we see that the majority of the months are statistically significant, evident by their low p-values. However, it seems like May’s p-value of 0.53 means it is not statistically significant in explaining our response variable. The low R^2 of 0.01 also tells us that arrival year and month only explain 1% of the variance in week night stays. We will compare this when we analyze our interaction variable. We also see that winter months like January and February also decrease the number of week night stays, evident by their negative Estimate values. This could be due to the fact that 2015 has 0 stays during these months, and stays in January and February are much lower than the other months.
## [1] "The MSE for the Linear Model is: 3.59"
With an MSE of 3.59, this model will serve as our baseline MSE. Since we have not tested any other linear models, we cannot say for certain whether this model is accurate or not. The value of 3.59 is high, but we have to judge this against other models to be sure.
Out-of-sample MSE Let’s see how the model stacks up with our test data
## [1] "The MSE for the LM is: 3.59"
It appears like the MSE is exactly the same as our training model. This tells us that the linear model for both the in-scope and out of scope are testing consistently, and the data is not being underfitted or overfitted. These are good signs, which suggest this LM is consisent and reliable. Now, we must compare it to a model with an interaction term. Instead of just doing
## Actual predicted lm2
## 21968 0 2.513892 2.435013
## 25961 1 2.766916 2.789823
## 73459 1 2.928321 2.963724
## 106028 4 2.239232 2.311989
## 96753 2 2.589036 2.552028
## 9757 2 2.407038 2.491267
## [1] "The MSE for the Linear Model is: 3.59"
With an MSE equal to the LM without the interaction term, it is clear that the interaction term is not significant, and therefore produces no impact to the accuracy of our model. Even so, we must test the out-of-scope data to ensure the model is reliable.
##
## Call:
## lm(formula = stays_in_week_nights ~ arrival_date_year + arrival_date_month +
## arrival_date_year * arrival_date_month, data = train_data)
##
## Residuals:
## Min 1Q Median 3Q Max
## -2.964 -1.312 -0.363 0.688 47.210
##
## Coefficients:
## Estimate Std. Error t value
## (Intercept) -6.042e+02 1.221e+02 -4.948
## arrival_date_year 3.007e-01 6.055e-02 4.966
## arrival_date_monthFebruary 8.230e+01 1.584e+02 0.520
## arrival_date_monthMarch 6.957e+01 1.529e+02 0.455
## arrival_date_monthApril 2.659e+02 1.497e+02 1.777
## arrival_date_monthMay 5.005e+02 1.481e+02 3.379
## arrival_date_monthJune 3.207e+02 1.503e+02 2.133
## arrival_date_monthJuly 6.079e+02 1.330e+02 4.570
## arrival_date_monthAugust 3.061e+02 1.316e+02 2.326
## arrival_date_monthSeptember 5.364e+02 1.513e+02 3.545
## arrival_date_monthOctober 4.939e+02 1.499e+02 3.296
## arrival_date_monthNovember 1.043e+03 1.691e+02 6.167
## arrival_date_monthDecember -8.292e+00 1.655e+02 -0.050
## arrival_date_year:arrival_date_monthFebruary -4.079e-02 7.854e-02 -0.519
## arrival_date_year:arrival_date_monthMarch -3.429e-02 7.583e-02 -0.452
## arrival_date_year:arrival_date_monthApril -1.317e-01 7.423e-02 -1.775
## arrival_date_year:arrival_date_monthMay -2.481e-01 7.344e-02 -3.377
## arrival_date_year:arrival_date_monthJune -1.588e-01 7.455e-02 -2.130
## arrival_date_year:arrival_date_monthJuly -3.011e-01 6.597e-02 -4.565
## arrival_date_year:arrival_date_monthAugust -1.514e-01 6.527e-02 -2.320
## arrival_date_year:arrival_date_monthSeptember -2.658e-01 7.505e-02 -3.542
## arrival_date_year:arrival_date_monthOctober -2.449e-01 7.433e-02 -3.294
## arrival_date_year:arrival_date_monthNovember -5.172e-01 8.389e-02 -6.166
## arrival_date_year:arrival_date_monthDecember 4.364e-03 8.209e-02 0.053
## Pr(>|t|)
## (Intercept) 7.52e-07 ***
## arrival_date_year 6.86e-07 ***
## arrival_date_monthFebruary 0.603293
## arrival_date_monthMarch 0.649146
## arrival_date_monthApril 0.075651 .
## arrival_date_monthMay 0.000728 ***
## arrival_date_monthJune 0.032927 *
## arrival_date_monthJuly 4.89e-06 ***
## arrival_date_monthAugust 0.020033 *
## arrival_date_monthSeptember 0.000393 ***
## arrival_date_monthOctober 0.000983 ***
## arrival_date_monthNovember 6.98e-10 ***
## arrival_date_monthDecember 0.960041
## arrival_date_year:arrival_date_monthFebruary 0.603486
## arrival_date_year:arrival_date_monthMarch 0.651172
## arrival_date_year:arrival_date_monthApril 0.075936 .
## arrival_date_year:arrival_date_monthMay 0.000732 ***
## arrival_date_year:arrival_date_monthJune 0.033212 *
## arrival_date_year:arrival_date_monthJuly 5.01e-06 ***
## arrival_date_year:arrival_date_monthAugust 0.020338 *
## arrival_date_year:arrival_date_monthSeptember 0.000398 ***
## arrival_date_year:arrival_date_monthOctober 0.000987 ***
## arrival_date_year:arrival_date_monthNovember 7.05e-10 ***
## arrival_date_year:arrival_date_monthDecember 0.957599
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 1.894 on 83549 degrees of freedom
## Multiple R-squared: 0.01528, Adjusted R-squared: 0.01501
## F-statistic: 56.37 on 23 and 83549 DF, p-value: < 2.2e-16
This is very interesting. It appears that our interaction terms don’t pose a significant effect on the model’s effectiveness. The R^2 value increased minimally, which indicates that the variance cannot be further explained by the interaction term.
## [1] "The MSE for the LM is: 3.59"
Once again, the model is reliable and consistent with the in-scope linear model. This MSE of 3.59 is not better or worse, which means we can rely on the model parameters. Now, I’d like to compare stays_in_weekend_nights with the same parameters:
## Actual predicted
## 21968 1 0.8372720
## 25961 0 1.1092168
## 73459 0 1.1078397
## 106028 0 0.8865543
## 96753 0 0.9617968
## 9757 1 0.8945704
## [1] "The MSE for the Linear Model is: 0.99"
Well this is a surprise! It appears that this LM for weekend nights has a much lower MSE than week nights did, with an MSE of 0.99. Compared to 3.59, this LM is more affective at predicting weekend night stays than week night stays by 2.6. Let’s see if this accuracy extends to the testing data:
## [1] "The MSE for the LM is: 0.97"
It appears that this MSE is just 0.02 lower than the training MSE. We can assume that this testing model is still very consistent with the training model, and it is not significantly underfitting or overfitting data.
Now, I am curious to see whether the interaction term has any effect on the weekend night model:
## Actual predicted lm2
## 21968 1 0.8372720 0.8249337
## 25961 0 1.1092168 1.1137985
## 73459 0 1.1078397 1.0643287
## 106028 0 0.8865543 0.9039509
## 96753 0 0.9617968 0.9340982
## 9757 1 0.8945704 0.8933482
## [1] "The MSE for the Linear Model is: 0.99"
The MSE of 0.99 means that the interaction term had no significant impact on the initial model. This means that the interaction is not statistically significant. Now the out-of-scope data:
## [1] "The MSE for the LM is: 0.97"
Once again, the MSE is equal to the out-of-scope MSE of 0.97. This means that the testing data is fitted correctly, and shows how effective and reliable the model is at predicting out of scope data.
After executing our linear models, it is time to examine the effectiveness of KNN models compared to the LM models. We will begin by analyzing stays_in_week_nights with k=3
## Actual predicted lm2 knn1
## 21968 0 2.513892 2.435013 1.555556
## 25961 1 2.766916 2.789823 2.666667
## 73459 1 2.928321 2.963724 1.444444
## 106028 4 2.239232 2.311989 3.000000
## 96753 2 2.589036 2.552028 1.555556
## 9757 2 2.407038 2.491267 1.555556
## [1] "The KNN MSE with k=3 is: 4.25"
It appears like this MSE of 4.23 is much higher than the 3.59 that we calculated for the Linear Model with the same parameters. This tells us that the LM is more affective at predicting stays_in_week_nights than the KNN model is, since the KNN MSE is 0.64 higher than the LM Now, let’s check the out of scope data:
## [1] "The Test KNN MSE with k=3 is: 4.28"
It looks like the KNN for the out-of-scope data is only .003 lower than the training KNN. This shows us that the testing model is fitted correctly, and this is a reliable and consistent model when paired with testing data.
Now we will analyze this with k = 5:
## Actual predicted lm2 knn1 knn2
## 21968 0 2.513892 2.435013 1.555556 1.60
## 25961 1 2.766916 2.789823 2.666667 1.40
## 73459 1 2.928321 2.963724 1.444444 1.40
## 106028 4 2.239232 2.311989 3.000000 2.96
## 96753 2 2.589036 2.552028 1.555556 2.44
## 9757 2 2.407038 2.491267 1.555556 2.60
## [1] "The KNN MSE with k=5 is: 4.32"
It looks like the MSE of this KNN model with k=5 is only .06 higher than the k=3 MSE. This shows that a higher k value is not a significant contributor to the prediction model of KNN. Out-of-scope:
## [1] "The Test KNN MSE with k=5 is: 4.34"
The out of scope MSE shows that the testing model is 0.09 higher than the training MSE. This could be attributable to overfitting of the data, which makes the testing model less accurate and less reliable.
Now, I will do the same procedure for stays_in_weekend_nights:
## Actual predicted lm2 knn11
## 21968 1 0.8372720 0.8249337 1.1111111
## 25961 0 1.1092168 1.1137985 1.2222222
## 73459 0 1.1078397 1.0643287 0.0000000
## 106028 0 0.8865543 0.9039509 0.0000000
## 96753 0 0.9617968 0.9340982 0.0000000
## 9757 1 0.8945704 0.8933482 0.5555556
## [1] "The KNN MSE with k=3 is: 1.38"
It appears like this MSE of 1.36 is much higher than the 0.99 that we calculated for the Linear Model with the same parameters. This tells us that the LM is more effective at predicting stays_in_weekend_nights than the KNN model is, since the KNN MSE is 0.35 higher than the LM Now, let’s check the out of scope data:
## [1] "The Test KNN MSE with k=3 is: 1.36"
It looks like the KNN for the out-of-scope data is only .02 lower than the training KNN. This shows us that the testing model is fitted correctly, and this is a reliable and consistent model when paired with testing data.
Now we will analyze this with k = 5:
## Actual predicted lm2 knn11 knn22
## 21968 1 0.8372720 0.8249337 1.1111111 0.56
## 25961 0 1.1092168 1.1137985 1.2222222 0.64
## 73459 0 1.1078397 1.0643287 0.0000000 0.00
## 106028 0 0.8865543 0.9039509 0.0000000 0.56
## 96753 0 0.9617968 0.9340982 0.0000000 0.96
## 9757 1 0.8945704 0.8933482 0.5555556 0.92
## [1] "The KNN MSE with k=5 is: 1.27"
It looks like the MSE of this KNN model with k=5 is 0.09 lower than the k=3 training MSE. This shows that a higher k value is not a significant contributor to the prediction model of KNN. Out-of-scope:
## [1] "The Test KNN MSE with k=5 is: 1.25"
With an MSE only 0.02 lower, we can state that the testing MSE is fitted correctly, and is a reliable model relative to the in-scope data. Compared to the testing MSE of the linear model, it not as accurate as the 0.97 MSE obtained from that LM.
library(ROCR)
hotels_no_outliers$arrival_month_factor <- as.character(hotels_no_outliers$arrival_date_month)
hotels_no_outliers$arrival_month_factor <- recode(as.character(hotels_no_outliers$arrival_month),
"June" = 1,
"July" = 1,
"August" = 1,
"December" = 1,
"November" = 0,
"January" = 0,
"February" = 0,
"March" = 0,
"April" = 0,
"May" = 0,
"September" = 0,
"October" = 0)
hotels_no_outliers$arrival_month_factor <- as.logical(hotels_no_outliers$arrival_month_factor)
Since a logistic regression analyzes a binomial independent variable with one or multiple dependent variables, we decided to create the arrival_month_factor variable that describes if the month a reservation was made in a hotel during the holiday months (Thanksgiving and Christmas) as well as the summer vacation months (June, July and August).
table(hotels_no_outliers$arrival_date_month)
##
## January February March April May June July August
## 5929 8068 9794 11089 11791 10939 12661 13877
## September October November December
## 10508 11160 6794 6780
table(hotels_no_outliers$arrival_month_factor)
##
## FALSE TRUE
## 75133 44257
These tables show the recurrences of each month in the dataset and in the holiday/vacation months.
hotels_glm <- hotels_no_outliers %>%
transmute(
arrival_month_factor,
stays_in_weekend_nights = as.integer(stays_in_weekend_nights),
stays_in_week_nights = as.integer(stays_in_week_nights),
arrival_date_year= as.factor(arrival_date_year)
)
set.seed(2023)
train_indices <- sample(1:NROW(hotels_glm),NROW(hotels_glm)*.7)
# Training Data
train_data <- hotels_glm[train_indices, ]
# Testing Data
test_data <- hotels_glm[-train_indices, ]
library(ROCR)
hotels.train.glm0 <- glm(train_data$arrival_month_factor ~ ., family = binomial, data = train_data)
summary(hotels.train.glm0)
##
## Call:
## glm(formula = train_data$arrival_month_factor ~ ., family = binomial,
## data = train_data)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -0.505039 0.019073 -26.479 < 2e-16 ***
## stays_in_weekend_nights 0.065606 0.008342 7.865 3.7e-15 ***
## stays_in_week_nights 0.073239 0.004418 16.578 < 2e-16 ***
## arrival_date_year2016 -0.446479 0.019570 -22.814 < 2e-16 ***
## arrival_date_year2017 -0.206772 0.020440 -10.116 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 110025 on 83572 degrees of freedom
## Residual deviance: 108807 on 83568 degrees of freedom
## AIC: 108817
##
## Number of Fisher Scoring iterations: 4
The summary of the logistic regression model indicates that individual independent variables have statistically significant relationships with the dependent binomial variable, due to the low p-values. However, despite the statistical significance of these relationships, the overall model fit, as indicated by the high null deviance, residual deviance, and AIC values, suggests that the model may not adequately capture the underlying patterns in the data.
pred.train.glm0 <- predict(hotels.train.glm0, type = "response")
hist(pred.train.glm0)
table(predict(hotels.train.glm0, type = "response") >0.50)
##
## FALSE TRUE
## 81777 1796
table(predict(hotels.train.glm0, type = "response") >0.40)
##
## FALSE TRUE
## 62675 20898
table(predict(hotels.train.glm0, type = "response") >0.30)
##
## FALSE TRUE
## 7829 75744
costfunc <- function(obs, pred.p, pcut) {
weight1 <- 1
weight0 <- 1
c1 <- (obs == 1) & (pred.p < pcut)
c0 <- (obs == 0) & (pred.p >= pcut)
cost <- mean(weight1 * c1 + weight0 * c0)
return(cost)
}
p.seq <- seq(0.01, 1, 0.01)
cost <- rep(0, length(p.seq))
for (i in 1:length(p.seq)) {
cost[i] <- costfunc(obs = train_data$arrival_month_factor, pred.p = pred.train.glm0, pcut = p.seq[i])
}
plot(p.seq, cost)
# find the optimal pcut
pcut = p.seq[which(cost==min(cost))]
print(pcut)
## [1] 0.48
In order for us to properly asses the logistic regression model, we need to get the optimal pcut value for the model. In this case, we did some basic calculations to get the pcut value that had the minimal cost. We first get the instances where the predicted probability belonging to the positive class is less than the pcut, to identify false negatives, and the other way arounf for alse positives. Based on the distrubution of our data we got an optimal pcut value of 0.48.
# get binary prediction
month.glm.train<- (pred.train.glm0 >pcut)*1
# get confusion matrix
table(train_data$arrival_month_factor, month.glm.train, dnn = c("True", "Predicted"))
## Predicted
## True 0 1
## FALSE 51127 1633
## TRUE 29073 1740
MR<- mean(train_data$arrival_month_factor!=month.glm.train)
# False positive rate
FPR <- sum((train_data$arrival_month_factor == 0 & month.glm.train == 1) / sum(train_data$arrival_month_factor == 0))
# False negative rate
FNR <- sum((train_data$arrival_month_factor == 1 & month.glm.train == 0) / sum(train_data$arrival_month_factor == 1))
cat("FPR:", FPR, "FNR:", FNR, "MR", MR, "\n")
## FPR: 0.03095148 FNR: 0.9435303 MR 0.3674153
These values suggest that the model is performing well in terms of minimizing false positives, but it has a high rate of false negatives, which means it’s not effectively identifying positive cases.
pred <- prediction(pred.train.glm0, train_data$arrival_month_factor)
perf <- performance(pred, "tpr", "fpr")
plot(perf, colorize=TRUE)
unlist(slot(performance(pred, "auc"), "y.values"))
## [1] 0.5779262
The area under the curve suggest that the model is not predicting the data properly. The auc indicates that the model has a similar performance as randomly guessing the value of the variable. Coupled with the fact that there is a very high likelihood of misinterpreting a negative case, we don’t think it is an appropiate estimation model.
hotels.test.glm0 <- glm(test_data$arrival_month_factor ~ ., family = binomial, data = test_data)
summary(hotels.test.glm0)
##
## Call:
## glm(formula = test_data$arrival_month_factor ~ ., family = binomial,
## data = test_data)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -0.482937 0.028883 -16.720 < 2e-16 ***
## stays_in_weekend_nights 0.052960 0.012734 4.159 3.20e-05 ***
## stays_in_week_nights 0.075259 0.006654 11.310 < 2e-16 ***
## arrival_date_year2016 -0.436577 0.029615 -14.742 < 2e-16 ***
## arrival_date_year2017 -0.189515 0.030922 -6.129 8.85e-10 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 47403 on 35816 degrees of freedom
## Residual deviance: 46888 on 35812 degrees of freedom
## AIC: 46898
##
## Number of Fisher Scoring iterations: 4
pred.test.glm0 <- predict(hotels.test.glm0, type = "response")
costfunc <- function(obs, pred.p, pcut) {
weight1 <- 1
weight0 <- 1
c1 <- (obs == 1) & (pred.p < pcut)
c0 <- (obs == 0) & (pred.p >= pcut)
cost <- mean(weight1 * c1 + weight0 * c0)
return(cost)
}
p.seq <- seq(0.01, 1, 0.01)
cost <- rep(0, length(p.seq))
for (i in 1:length(p.seq)) {
cost[i] <- costfunc(obs = test_data$arrival_month_factor, pred.p = pred.test.glm0, pcut = p.seq[i])
}
# find the optimal pcut
pcut = p.seq[which(cost==min(cost))]
pcut
## [1] 0.45
# get binary prediction
month.glm.test<- (pred.test.glm0 >pcut)*1
# get confusion matrix
table(test_data$arrival_month_factor, month.glm.test, dnn = c("True", "Predicted"))
## Predicted
## True 0 1
## FALSE 20648 1725
## TRUE 11589 1855
MR<- mean(test_data$arrival_month_factor!=month.glm.test)
# False positive rate
FPR <- sum((test_data$arrival_month_factor == 0 & month.glm.test == 1) / sum(train_data$arrival_month_factor == 0))
# False negative rate
FNR <- sum((test_data$arrival_month_factor == 1 & month.glm.test == 0) / sum(train_data$arrival_month_factor == 1))
cat("FPR:", FPR, "FNR:", FNR, "MR", MR, "\n")
## FPR: 0.03269522 FNR: 0.3761075 MR 0.3717229
pred <- prediction(pred.test.glm0, test_data$arrival_month_factor)
perf1 <- performance(pred, "tpr", "fpr")
plot(perf1, colorize=TRUE)
unlist(slot(performance(pred, "auc"), "y.values"))
## [1] 0.5757393
Both the training and testing data suggests that it has a relatively similar distribution than randomly guessing. Yet the logistic regression for testing data, has a significant lower values for the false positive rate, thus indicating a better prediction model for the binomial descriptive variable.
We did not use all the variables, mainly because we are trying to understand whether arrival year and arrival month are significant predictors of stays in week nights and stays in weekend nights, and what kind of model could predict that relationship more effectively. We saw in the visualizations that there are fluctuations in hotel stays over time, so it is clear that tourists have a preference for what months to travel in. Similarly, our correlation table showed that there is a .02 correlation between arrival year and weekend night stays, and a .03 correlation between arrival year and week night stays. We sought to explore this relationship on a deeper level, and it is why we excluded the other variables.
Theoretically, the linear models are more effective at predicting week night and weekend night stays based on arrival date and arrival year. Let’s recap what the MSE values told us:
Week night stays: MSE of 3.59 for the test linear models MSE of both arrival_date_year + arrival_date_month MSE of 4.26 for the test KNN model of arrival_date_year + arrival_date_month with k=3 MSE of 4.34 for the test KNN model of arrival_date_year + arrival_date_month with k=5
The lower MSE of the linear model shows that it is a more effective way of predicting week night stays based on arrival_date_year + arrival_date_month as the predictor variables. Even though a lower MSE would have been preferred, the difference of 0.67 from the k=3 model shows a vast decline in the accuracy of KNN. Even feature scaling did not improve the model, making the MSE 0.75 higher.
Weekend night stays: The weekend night models were more effective than week night models: The test MSE of 0.97 for the LM models of arrival_date_year + arrival_date_month were much more accurate than week night stays MSE increasd to 1.34 for the test MSE of the KNN with k=3, an increase of .37 Even though feature scaling improved the k=5 KNN test MSE to 1.25, it was still less accurate by 0.28 than the LM
Assumptions of each model:
Some assumptions of the linear model is that the data is linear and normally distributed. We saw at the beginning of the data that both week night stays and weekend night stays were not normally distributed. Due to this fact, it poses a dilemma to the linear regression model. It is hard to determine linearity between hotel stays and time of the year, since they are not continuous variables. Due to this, we must question the effectiveness of the linear model at effectively predicting week night stays and weekend night stays.
Some assumptions of the KNN model is that similar data points are close to one another. This assumption is essential because an accurate prediction of one data point is dependent on other plots close to it. Feature scaling allows for this assumption to be adjusted, so that each data point can pull either less or more data points around it to make more accurate predictions. We saw in this report that increasing the k value from 3 to 5 had notable changes in the MSE of the model. A higher k makes the prediction smoother but less detailed. On the other hand, a lower k value can make it more susceptible to noise.
Based on the results and the assumptions of the model, I would select the KNN model in practice. Even though initial results deemed it less accurate, the data can be standardized to make the distribution more normal. Additionally, the fact that k can be adjusted means that there is a possibility to increase the effectiveness of the model, but must be done so cautiously. As seen in each model, the linear models had better in-sample and out-of-sample MSE. However, I believe that KNN leverages the fact that humans are creatures of similarity. When it’s summer time, more people want to travel. If a particular destination becomes viral on social media or there are great deals to travel there, more people will take advantage and travel to that destination during the time window the offer is available. Even though it will never be possible to predict human behavior, I believe that the best way to predict human nature is by looking at what other humans are doing. For that reason, KNN is better in practice, while LM is better theoretically.