Load data

The Zillow Home Price Data are time series datasets available online via this link. The datasets we will be focusing is updated monthly:

Facebook Prophet model

We are still focusing on home value monthly data in Cincinnati. To perform Prophet Model, we need to change name to our dataset: name our date variable –ds; name our time series variable (home_value) –y.

cin_home_value<- home_value%>%
  filter(RegionID==394466)%>%
  pivot_longer(6:269,names_to="date",values_to="home_value")%>%
  mutate(new_month= as.Date(date, "%Y-%m-%d"))

#change to prophet format data
prophet_data = cin_home_value %>% 
    rename(ds = new_month, # Have to name our date variable "ds"
    y = home_value)  # Have to name our time series "y"

prophet_data%>%
  head(5)%>%
  kbl(caption = "Table 1: Prophet Format Data: Cincinnati Home Value")%>%
  kable_classic(full_width = F, html_font = "Cambria")
Table 1: Prophet Format Data: Cincinnati Home Value
RegionID SizeRank RegionName RegionType StateName date y ds
394466 28 Cincinnati, OH Msa OH 2000-01-31 129379 2000-01-31
394466 28 Cincinnati, OH Msa OH 2000-02-29 129364 2000-02-29
394466 28 Cincinnati, OH Msa OH 2000-03-31 129290 2000-03-31
394466 28 Cincinnati, OH Msa OH 2000-04-30 129780 2000-04-30
394466 28 Cincinnati, OH Msa OH 2000-05-31 130454 2000-05-31

Prophet model is usually used for daily data, fitting monthly data in this model is also applicable but we are asking for daily forecasts in this case. We will start with a basic forecasting using Prophet with frequency as daily.

We took months before “2017-08-31” as train dataset (approximately 80%) and time at “2017-08-31” and after as test dataset (approximately 20%). We can plot an interactive plot for forecasting after modeling.

Additionally, we will plot lines of forecast home value with actual Home value for comparison.

train1 = prophet_data %>% 
  filter(ds<ymd("2017-08-31"))
test1 = prophet_data %>%
  filter(ds>=ymd("2017-08-31"))
model1 = prophet(train1)
future1 = make_future_dataframe(model1,periods = 1583)#freq=daily; 53 month ahead
forecast1 = predict(model1,future1)

#plot interactive forecast plot
dyplot.prophet(model1,forecast1)
#plot actual vs forecast line for comparision
ggplot()+
  geom_line(data=forecast1,aes(x=as.Date(ds),y=yhat,col="blue"))+
  geom_line(data=cin_home_value,aes(x=as.Date(new_month),y=home_value,col="red"))+
  scale_color_identity(name = "Legend",      #for manual legend
                       breaks = c("red","blue"),
                       labels = c("Actual","Forecast"),
                       guide = "legend")+
  labs(title = "Cincy Home Value: Actual value vs Prophet basic forecast",
       x = "Date",
       y = "Cincinnati Home Value")

As we noticed from above that the output is strange as the forecast is not smooth and it generates daily instead of monthly forecast. Now we will change the frequency to month.

train = prophet_data %>% 
  filter(ds<ymd("2017-08-31"))
test = prophet_data %>%
  filter(ds>=ymd("2017-08-31"))
model = prophet(train)
future = make_future_dataframe(model,periods = 53,freq = 'month')
forecast = predict(model,future)

#plot interactive forecast plot
dyplot.prophet(model,forecast)
#plot actual vs forecast line for comparision
ggplot()+
  geom_line(data=forecast,aes(x=as.Date(ds),y=yhat,col="blue"))+
  geom_line(data=cin_home_value,aes(x=as.Date(new_month),y=home_value,col="red"))+
  scale_color_identity(name = "Legend",      #for manual legend
                       breaks = c("red","blue"),
                       labels = c("Actual","Forecast"),
                       guide = "legend")+
  labs(title = "Cincy Home Value: Actual value vs Prophet basic forecast",
       x = "Date",
       y = "Cincinnati Home Value")

The plots looks more smooth. However, we can notice from the plots that predicted home value is clearly lower than actual home value after the time of “2017-08-31”. Thus, we will look into trend and seasonality impact on home_value:

Decomposition of the elements

prophet_plot_components(model,forecast)

We can see from the decompose plot above that there’s a clear trend but seasonality is less clear. It might because this home value dataset is seasonally adjusted and only includes the middle price tier of homes. However, there might be cyclical component that follows the economic patterns (eg: 2008 financial crisis; 2019 COVID, etc). Now let’s check the change points:

Examination of the changepoints

The input of changepoints allowed us to increase the fit and its flexibility. We can examine the changepoints identified for the “trend” part of the time series from home value plot above. There is at least 5 change point (1 around 2005, 2 around 2010, 1 around 2015, 1 around 2019):

model_changepoint = prophet(train,n.changepoints=5)
future_changepoint = make_future_dataframe(model,periods = 53,freq = 'month')
forecast_changepoint = predict(model_changepoint,future_changepoint)
plot(model_changepoint,forecast_changepoint)+
  add_changepoints_to_plot(model_changepoint)+
  theme_bw()+xlab("Date")+ylab("Home value")

prophet_plot_components(model_changepoint,forecast_changepoint)

We noticed from the trend plot that setting changepoint 5 roghtly captured trend changes. If we set higher changepoint number, there’s chance of overfitting.

Saturation Points

From the forecast output above, the predicted values are all above 0. In our model, we do not need to take into account a saturating maximum point as the home value. However, we must also specify a maximum capacity if using logistic growth with a minimum. Thus, we could set a reasonable minimun and maximum value: usually home value ( Cincinnati median home value) is unlikely to go below 0 dollar and go above $0.3 million.

train_saturation = prophet_data %>% 
  filter(ds<ymd("2017-08-31"))
test_saturation = prophet_data %>%
  filter(ds>=ymd("2017-08-31"))
model_saturation = prophet(train_saturation)
future_saturation = make_future_dataframe(model_saturation,periods = 53,freq = 'month')#freq=monthly; 53 month ahead

# Set "floor" in training data
train_saturation$floor = 0
train_saturation$cap=300000
future_saturation$floor = 0
future_saturation$cap=300000

# Set floor in forecast data
future_saturation$floor = 0
future_saturation$cap=300000
sat_model = prophet(train_saturation,growth='logistic')
sat_forecast = predict(sat_model,future_saturation)
plot(sat_model,sat_forecast)+ylim(0,300000)+
theme_bw()+xlab("Date")+ylab("Home Value")

From the plot, we noticed that all the home values are within the range of (0.1,0.3) million. Therefore, we do not need to specify the saturating points.

Identification and assessment of seasonality

From above decomposition, we noticed that we do not have yearly seasonality. It might because this value is a smoothed, seasonally adjusted measure of the typical home value and market changes across a given region and housing type. Now we will try to identity daily, weekly, as well as additibe/ multiplicative seasonality:

  • Daily/weekly seasonality We start with model_season with daily.seasonality and weekly.seasonality option set to TURE to check daily and weekly seasonality:
model_season = prophet(train, daily.seasonality = TRUE, weekly.seasonality = TRUE)
future_season = make_future_dataframe(model_season,periods = 1583)#freq=daily; 53 month ahead
forecast_season = predict(model_season,future_season)

#plot interactive forecast plot
dyplot.prophet(model_season,forecast_season)
#decompose
prophet_plot_components(model_season,forecast_season)

Daily or weekly seasonality also not very obvious. Now we will dive into additive/ multiplicative seasonality:

  • additive seasonality
additive = prophet(train)
add_fcst = predict(additive,future)
plot(additive,add_fcst)

prophet_plot_components(additive,add_fcst)

  • multiplicative seasonality
multi = prophet(train,seasonality.mode = 'multiplicative')
multi_fcst = predict(multi,future)
plot(multi,multi_fcst)

prophet_plot_components(multi,multi_fcst)

Decompose plots of additive or multiplicative seasonality appear to be similar: seasonality is not obvious. Thus, we might only have trend and no seasonality.

Assessment of holidays

We want to try with inclusion of holidays. The result might not be very accurate with important holidays as we are examining monthly rather than daily data:

#Can use built-in holidays, or specify our own
model_holiday = prophet(train,fit=FALSE,daily.seasonality = TRUE, weekly.seasonality = TRUE)
model_holiday = add_country_holidays(model_holiday,country_name = 'US')
model_holiday = fit.prophet(model_holiday,train)
forecast_holiday = predict(model_holiday,future1)

dyplot.prophet(model_holiday,forecast_holiday)
prophet_plot_components(model_holiday,forecast_holiday)

The fitted line looks very well. Additionally, the plots indicated a few import days over years. These days maybe corresponding to our changing points in trend. Now we can look into those days:

forecast_holiday %>%
  filter(holidays != 0) %>%
  select_if(~ !is.numeric(.) || sum(.) != 0)%>%
  select(ds,holidays,`Memorial Day`,`New Year's Day (Observed)`)%>%
  kbl(caption = "Table 2: Important Holidays")%>%
  kable_classic(full_width = F, html_font = "Cambria")
Table 2: Important Holidays
ds holidays Memorial Day New Year’s Day (Observed)
2004-05-31 1267.9140 1267.914 0.0000
2004-12-31 -399.0302 0.000 -399.0302
2010-05-31 1267.9140 1267.914 0.0000
2010-12-31 -399.0302 0.000 -399.0302
2018-05-28 1267.9140 1267.914 0.0000
2019-05-27 1267.9140 1267.914 0.0000
2020-05-25 1267.9140 1267.914 0.0000
2021-05-31 1267.9140 1267.914 0.0000

We may notice that the returning result is reasonable:

Important positive influential holiday is Memorial day which is in May when the market is in peak season and buyers are active. On the contrary, negative correlated holiday is New Year’s day, which is when the housing market is in slow season. Moreover, we can see decrease in 2004 and 2010. As for 2010-2011, we had the housing market crash after the 2008 financial crisis.

forecast_holiday %>%
  filter(holidays != 0) %>%
  dplyr::select(-ds:-additive_terms_upper, -holidays:-holidays_upper, -weekly:-yhat, -contains("upper"), -contains("lower")) %>%
  mutate_all(~ if_else(. == 0, as.numeric(NA), .)) %>%
  summarize_if(is.numeric, ~ max(., na.rm = T)) %>%
  pivot_longer(
    cols = `Christmas Day`:`Washington's Birthday`,
    names_to = 'holiday', 
    values_to = 'effect'
  ) %>%
ggplot() +
  geom_col(aes(effect,holiday))+
  theme_bw()

Outlier detection

anom_train = prophet_data %>% 
  filter(ds<ymd("2017-08-31"))
anom_test = prophet_data %>% 
  filter(ds>=ymd("2017-08-31"))
anom_model = prophet(anom_train)
anom_future = make_future_dataframe(anom_model,periods = 53,freq = 'month')
forecast = predict(anom_model,anom_future)
forecast_plot_data = forecast %>% 
  as_tibble() %>% 
  mutate(ds = as.Date(ds)) %>% 
  filter(ds>=ymd("2017-08-31")) %>% 
  mutate(y = anom_test$y) %>% 
  mutate(anomaly = if_else(y>yhat_upper | y<yhat_lower,TRUE,as.logical(NA)))
forecast_plot_data %>% 
  ggplot()+
    geom_line(aes(ds,y))+
    geom_point(aes(ds,y,color=anomaly))+
    geom_line(aes(ds,yhat),color='red')+
    geom_ribbon(aes(ds,ymin=yhat_lower,ymax=yhat_upper),alpha=0.4,fill='blue') + 
  scale_colour_discrete(na.translate = F)

We can see that our actual values in test dataset are much higher than the predicted values. Therefore, all points are detected anomaly. However, we can not delete these data, as they are generated under COVID situation which is an anomaly period of time and business trend.

Rolling window Cross-Validation

We want to use rolling window cross-validation to assess performance of the model at meaningful thresholds depending on the data (eg: 1 year out for monthly data) and then use various metrics (eg: RMSE,MAE,MAPE).

  • Cross Validation on holiday model
df.cv_holiday <- cross_validation(model_holiday, initial = 4*365,horizon = 365, units = 'days')

unique(df.cv_holiday$cutoff)%>%
  kbl(caption = "Table 3: Holidays model CV cutoff")%>%
  kable_classic(full_width = F, html_font = "Cambria")
Table 3: Holidays model CV cutoff
x
2004-02-02 12:00:00
2004-08-03 00:00:00
2005-02-01 12:00:00
2005-08-03 00:00:00
2006-02-01 12:00:00
2006-08-03 00:00:00
2007-02-01 12:00:00
2007-08-03 00:00:00
2008-02-01 12:00:00
2008-08-02 00:00:00
2009-01-31 12:00:00
2009-08-02 00:00:00
2010-01-31 12:00:00
2010-08-02 00:00:00
2011-01-31 12:00:00
2011-08-02 00:00:00
2012-01-31 12:00:00
2012-08-01 00:00:00
2013-01-30 12:00:00
2013-08-01 00:00:00
2014-01-30 12:00:00
2014-08-01 00:00:00
2015-01-30 12:00:00
2015-08-01 00:00:00
2016-01-30 12:00:00
2016-07-31 00:00:00 
df.cv_holiday %>% 
  ggplot()+
  geom_point(aes(ds,y)) +
  geom_point(aes(ds,yhat,color=factor(cutoff)))+
  theme_bw()+
  xlab("Date")+
  ylab("Home value")+
  scale_color_discrete(name = 'Cutoff')

performance_metrics(df.cv_holiday)%>%
  kbl(caption = "Table 4: Holidays model CV performance metrics")%>%
  kable_classic(full_width = F, html_font = "Cambria")
Table 4: Holidays model CV performance metrics
horizon mse rmse mae mape mdape smape coverage
1380 hours 7085812 2661.919 1847.714 0.0127661 0.0088902 0.0128621 0.1693548
1392 hours 6830350 2613.494 1828.086 0.0125604 0.0088902 0.0126436 0.1935484
1404 hours 7929313 2815.904 2004.096 0.0137685 0.0089556 0.0138474 0.1935484
1416 hours 8326729 2885.607 2134.955 0.0147409 0.0099573 0.0148185 0.2258065
1428 hours 8925242 2987.514 2227.183 0.0154213 0.0099573 0.0155174 0.2338710
1440 hours 9782313 3127.669 2321.821 0.0161352 0.0109473 0.0162752 0.2338710
1452 hours 9628944 3103.054 2311.548 0.0160471 0.0109473 0.0161895 0.2258065
1464 hours 9701966 3114.798 2331.786 0.0161688 0.0121212 0.0163127 0.2258065
2100 hours 8151873 2855.149 2072.723 0.0142187 0.0109473 0.0143113 0.2903226
2124 hours 10653239 3263.930 2488.605 0.0170862 0.0125826 0.0171767 0.2419355
2136 hours 10717861 3273.815 2522.003 0.0172941 0.0137316 0.0173834 0.2338710
2148 hours 11607770 3407.018 2606.955 0.0180729 0.0128247 0.0182391 0.2580645
2160 hours 11352351 3369.325 2607.621 0.0180847 0.0137316 0.0182502 0.2338710
2172 hours 11216270 3349.070 2586.079 0.0179203 0.0128247 0.0180888 0.2258065
2184 hours 11792074 3433.959 2619.637 0.0181315 0.0128247 0.0183149 0.2258065
2208 hours 11505619 3391.993 2605.975 0.0179717 0.0128247 0.0181463 0.2177419
2844 hours 10081178 3175.087 2346.764 0.0160457 0.0128247 0.0161706 0.2903226
2856 hours 10113834 3180.225 2369.322 0.0161690 0.0128247 0.0162876 0.2822581
2868 hours 10650577 3263.522 2476.660 0.0169122 0.0128247 0.0170360 0.2580645
2880 hours 10977307 3313.202 2621.293 0.0180028 0.0151552 0.0181192 0.2338710
2892 hours 12358409 3515.453 2777.737 0.0191477 0.0151552 0.0193032 0.2338710
2904 hours 14357527 3789.133 2931.750 0.0202899 0.0172276 0.0205209 0.2258065
2916 hours 14253307 3775.355 2913.183 0.0201432 0.0150671 0.0203787 0.2258065
2928 hours 14524051 3811.043 2968.882 0.0204882 0.0172276 0.0207292 0.2258065
3564 hours 12279925 3504.272 2647.313 0.0180714 0.0150671 0.0182375 0.2903226
3588 hours 15215440 3900.697 3084.814 0.0211192 0.0181413 0.0212932 0.2177419
3600 hours 15354601 3918.495 3130.387 0.0214060 0.0182110 0.0215772 0.2096774
3612 hours 17190227 4146.110 3253.759 0.0224584 0.0182110 0.0227317 0.2258065
3624 hours 16918182 4113.172 3218.015 0.0222109 0.0187418 0.0224809 0.2580645
3636 hours 16781197 4096.486 3194.597 0.0220248 0.0152658 0.0223010 0.2580645
3648 hours 17269927 4155.710 3217.739 0.0221652 0.0156254 0.0224531 0.2580645
3672 hours 17037972 4127.708 3227.050 0.0221259 0.0156254 0.0224043 0.2580645
4308 hours 14889876 3858.740 2929.773 0.0199010 0.0156254 0.0201010 0.3548387
4332 hours 17594367 4194.564 3322.505 0.0226795 0.0203274 0.0228883 0.2822581
4344 hours 15710286 3963.620 3077.467 0.0209027 0.0172238 0.0210921 0.3225806
4356 hours 19101619 4370.540 3391.115 0.0233096 0.0187418 0.0236241 0.3064516
4368 hours 17969227 4239.013 3358.235 0.0230029 0.0222755 0.0232267 0.2983871
4380 hours 17227279 4150.576 3303.528 0.0225634 0.0222755 0.0227665 0.2903226
4392 hours 19851147 4455.463 3518.704 0.0241879 0.0175517 0.0245044 0.2580645
4416 hours 20588467 4537.452 3617.151 0.0247918 0.0234997 0.0251231 0.2580645
5016 hours 18019475 4244.935 3322.038 0.0225760 0.0175517 0.0228247 0.2983871
5040 hours 19584840 4425.476 3542.102 0.0240430 0.0234997 0.0242296 0.2741935
5052 hours 18301392 4278.013 3338.617 0.0225972 0.0197556 0.0227711 0.3225806
5064 hours 21050377 4588.069 3610.164 0.0247866 0.0234997 0.0250675 0.2903226
5076 hours 21696987 4658.003 3789.650 0.0259998 0.0259075 0.0262356 0.2419355
5088 hours 20711486 4550.987 3754.801 0.0255614 0.0259075 0.0257638 0.2258065
5100 hours 24205695 4919.928 3995.778 0.0273792 0.0259075 0.0277396 0.2258065
5124 hours 24254011 4924.836 4004.360 0.0274321 0.0259075 0.0277935 0.2258065
5760 hours 21769420 4665.771 3698.630 0.0251265 0.0197556 0.0254030 0.2903226
5772 hours 20421540 4519.020 3499.653 0.0237404 0.0197556 0.0239631 0.3440860
5784 hours 21911436 4680.965 3639.825 0.0247012 0.0191914 0.0249144 0.3306452
5796 hours 23601327 4858.120 3958.128 0.0269945 0.0259075 0.0272247 0.2500000
5808 hours 25180261 5017.994 4119.196 0.0281903 0.0223888 0.0284723 0.2258065
5820 hours 27535914 5247.467 4224.078 0.0290345 0.0223888 0.0294426 0.2258065
5832 hours 27644336 5257.788 4235.336 0.0290310 0.0223888 0.0294559 0.2258065
5844 hours 27326546 5227.480 4203.571 0.0288354 0.0209749 0.0292542 0.2258065
6480 hours 23858964 4884.564 3851.119 0.0261895 0.0205783 0.0264908 0.2661290
6504 hours 26782552 5175.186 4169.212 0.0282877 0.0248724 0.0285277 0.2473118
6516 hours 26994411 5195.615 4210.219 0.0285457 0.0260384 0.0287818 0.2177419
6528 hours 29529574 5434.112 4456.319 0.0305953 0.0224780 0.0310035 0.1693548
6540 hours 29242776 5407.659 4454.195 0.0306165 0.0260384 0.0310294 0.1612903
6552 hours 29074404 5392.069 4442.977 0.0303074 0.0260384 0.0307263 0.1612903
6564 hours 30811671 5550.826 4527.305 0.0308395 0.0260384 0.0313125 0.1612903
6588 hours 29796426 5458.610 4469.971 0.0303514 0.0252500 0.0307951 0.1612903
7224 hours 26723551 5169.483 4124.258 0.0278534 0.0252500 0.0281878 0.2258065
7236 hours 26469106 5144.814 4052.574 0.0273652 0.0252500 0.0276596 0.2473118
7248 hours 27369799 5231.615 4161.627 0.0281096 0.0224780 0.0284085 0.2338710
7260 hours 28774453 5364.182 4437.877 0.0301684 0.0295566 0.0304749 0.1774194
7272 hours 31675690 5628.116 4639.708 0.0316485 0.0273066 0.0320463 0.1612903
7284 hours 36247338 6020.576 4872.729 0.0333665 0.0295566 0.0339473 0.1612903
7296 hours 36729861 6060.517 4903.854 0.0334686 0.0295566 0.0340743 0.1612903
7308 hours 36430203 6035.744 4890.075 0.0333955 0.0289497 0.0339958 0.1612903
7944 hours 31867621 5645.141 4469.368 0.0302531 0.0252500 0.0306949 0.2258065
7968 hours 34731632 5893.355 4790.511 0.0323881 0.0301323 0.0327692 0.2150538
7980 hours 35127034 5926.806 4856.017 0.0328047 0.0319223 0.0331780 0.2096774
7992 hours 39436339 6279.836 5174.038 0.0353709 0.0289497 0.0359697 0.1693548
8004 hours 39089330 6252.146 5143.341 0.0352034 0.0319223 0.0357977 0.1612903
8016 hours 38828669 6231.265 5147.085 0.0349462 0.0319223 0.0355429 0.1612903
8028 hours 40680158 6378.100 5251.867 0.0356294 0.0319223 0.0362751 0.1612903
8052 hours 39581249 6291.363 5210.728 0.0352145 0.0319223 0.0358262 0.1612903
8688 hours 35425308 5951.916 4810.196 0.0323424 0.0319223 0.0328037 0.2258065
8712 hours 38865763 6234.241 5151.744 0.0347855 0.0349362 0.0352124 0.2150538
8724 hours 36237637 6019.770 4868.439 0.0327822 0.0264664 0.0331871 0.2338710
8736 hours 42574661 6524.926 5351.314 0.0365124 0.0349362 0.0371695 0.1774194
8748 hours 39604089 6293.178 5186.128 0.0352838 0.0332404 0.0357510 0.1612903
8760 hours 39307861 6269.598 5196.775 0.0350580 0.0332404 0.0355287 0.1612903 
plot_cross_validation_metric(df.cv_holiday, metric = 'rmse')

It does not look very well. Now Let’s compare other models:

  • Model Comparision

  • Model 1: changepoint in trend model with freq=‘monthly’, changepoint=5;

  • Model 2: seasonality model with freq=‘daily’ and week/daily seasonality=TRUE;

  • Model 3: Additive seasonality model;

  • Model 4: Multiplicative seasonality model.

#mod1
mod1 = prophet(train,n.changepoints=5)
mod1_future = make_future_dataframe(mod1,periods = 53,freq = 'month')
mod1_forecast = predict(mod1,mod1_future)

df_cv_mod1 <- cross_validation(mod1, initial = 4*365,horizon = 365, units = 'days')
metrics1 = performance_metrics(df_cv_mod1) %>% 
  mutate(model = 'mod1')

#mod2  
mod2 = prophet(train, daily.seasonality = TRUE, weekly.seasonality = TRUE)
mod2_future = make_future_dataframe(mod2,periods = 1583)#freq=daily; 53 month ahead
mod2_forecast = predict(mod2,mod2_future)

df_cv_mod2 <- cross_validation(mod2, initial = 4*365,horizon = 365, units = 'days')
metrics2 = performance_metrics(df_cv_mod2) %>% 
  mutate(model = "mod2")
metrics1 %>% 
bind_rows(metrics2) %>% 
ggplot()+
geom_line(aes(horizon,rmse,color=model))

#mod3
mod3 = prophet(train,seasonality.mode='additive')
mod3_forecast = predict(mod3,future)
df_cv_mod3 <- cross_validation(mod3, initial = 4*365,horizon = 365, units = 'days')
metrics3 = performance_metrics(df_cv_mod3) %>% 
  mutate(model = 'mod3')

#mod4
mod4 = prophet(train,seasonality.mode='multiplicative')
forecast4 = predict(mod4,future)
df_cv_mod4 <- cross_validation(mod4, initial = 4*365,horizon = 365, units = 'days')
metrics4 = performance_metrics(df_cv_mod4) %>% 
  mutate(model = "mod4")
metrics3 %>% 
bind_rows(metrics4) %>% 
ggplot()+
geom_line(aes(horizon,rmse,color=model))

performance_metrics(df.cv_holiday)%>%
  kbl(caption = "Table 5: Model 1 CV performance metrics")%>%
  kable_classic(full_width = F, html_font = "Cambria")
Table 5: Model 1 CV performance metrics
horizon mse rmse mae mape mdape smape coverage
1380 hours 7085812 2661.919 1847.714 0.0127661 0.0088902 0.0128621 0.1693548
1392 hours 6830350 2613.494 1828.086 0.0125604 0.0088902 0.0126436 0.1935484
1404 hours 7929313 2815.904 2004.096 0.0137685 0.0089556 0.0138474 0.1935484
1416 hours 8326729 2885.607 2134.955 0.0147409 0.0099573 0.0148185 0.2258065
1428 hours 8925242 2987.514 2227.183 0.0154213 0.0099573 0.0155174 0.2338710
1440 hours 9782313 3127.669 2321.821 0.0161352 0.0109473 0.0162752 0.2338710
1452 hours 9628944 3103.054 2311.548 0.0160471 0.0109473 0.0161895 0.2258065
1464 hours 9701966 3114.798 2331.786 0.0161688 0.0121212 0.0163127 0.2258065
2100 hours 8151873 2855.149 2072.723 0.0142187 0.0109473 0.0143113 0.2903226
2124 hours 10653239 3263.930 2488.605 0.0170862 0.0125826 0.0171767 0.2419355
2136 hours 10717861 3273.815 2522.003 0.0172941 0.0137316 0.0173834 0.2338710
2148 hours 11607770 3407.018 2606.955 0.0180729 0.0128247 0.0182391 0.2580645
2160 hours 11352351 3369.325 2607.621 0.0180847 0.0137316 0.0182502 0.2338710
2172 hours 11216270 3349.070 2586.079 0.0179203 0.0128247 0.0180888 0.2258065
2184 hours 11792074 3433.959 2619.637 0.0181315 0.0128247 0.0183149 0.2258065
2208 hours 11505619 3391.993 2605.975 0.0179717 0.0128247 0.0181463 0.2177419
2844 hours 10081178 3175.087 2346.764 0.0160457 0.0128247 0.0161706 0.2903226
2856 hours 10113834 3180.225 2369.322 0.0161690 0.0128247 0.0162876 0.2822581
2868 hours 10650577 3263.522 2476.660 0.0169122 0.0128247 0.0170360 0.2580645
2880 hours 10977307 3313.202 2621.293 0.0180028 0.0151552 0.0181192 0.2338710
2892 hours 12358409 3515.453 2777.737 0.0191477 0.0151552 0.0193032 0.2338710
2904 hours 14357527 3789.133 2931.750 0.0202899 0.0172276 0.0205209 0.2258065
2916 hours 14253307 3775.355 2913.183 0.0201432 0.0150671 0.0203787 0.2258065
2928 hours 14524051 3811.043 2968.882 0.0204882 0.0172276 0.0207292 0.2258065
3564 hours 12279925 3504.272 2647.313 0.0180714 0.0150671 0.0182375 0.2903226
3588 hours 15215440 3900.697 3084.814 0.0211192 0.0181413 0.0212932 0.2177419
3600 hours 15354601 3918.495 3130.387 0.0214060 0.0182110 0.0215772 0.2096774
3612 hours 17190227 4146.110 3253.759 0.0224584 0.0182110 0.0227317 0.2258065
3624 hours 16918182 4113.172 3218.015 0.0222109 0.0187418 0.0224809 0.2580645
3636 hours 16781197 4096.486 3194.597 0.0220248 0.0152658 0.0223010 0.2580645
3648 hours 17269927 4155.710 3217.739 0.0221652 0.0156254 0.0224531 0.2580645
3672 hours 17037972 4127.708 3227.050 0.0221259 0.0156254 0.0224043 0.2580645
4308 hours 14889876 3858.740 2929.773 0.0199010 0.0156254 0.0201010 0.3548387
4332 hours 17594367 4194.564 3322.505 0.0226795 0.0203274 0.0228883 0.2822581
4344 hours 15710286 3963.620 3077.467 0.0209027 0.0172238 0.0210921 0.3225806
4356 hours 19101619 4370.540 3391.115 0.0233096 0.0187418 0.0236241 0.3064516
4368 hours 17969227 4239.013 3358.235 0.0230029 0.0222755 0.0232267 0.2983871
4380 hours 17227279 4150.576 3303.528 0.0225634 0.0222755 0.0227665 0.2903226
4392 hours 19851147 4455.463 3518.704 0.0241879 0.0175517 0.0245044 0.2580645
4416 hours 20588467 4537.452 3617.151 0.0247918 0.0234997 0.0251231 0.2580645
5016 hours 18019475 4244.935 3322.038 0.0225760 0.0175517 0.0228247 0.2983871
5040 hours 19584840 4425.476 3542.102 0.0240430 0.0234997 0.0242296 0.2741935
5052 hours 18301392 4278.013 3338.617 0.0225972 0.0197556 0.0227711 0.3225806
5064 hours 21050377 4588.069 3610.164 0.0247866 0.0234997 0.0250675 0.2903226
5076 hours 21696987 4658.003 3789.650 0.0259998 0.0259075 0.0262356 0.2419355
5088 hours 20711486 4550.987 3754.801 0.0255614 0.0259075 0.0257638 0.2258065
5100 hours 24205695 4919.928 3995.778 0.0273792 0.0259075 0.0277396 0.2258065
5124 hours 24254011 4924.836 4004.360 0.0274321 0.0259075 0.0277935 0.2258065
5760 hours 21769420 4665.771 3698.630 0.0251265 0.0197556 0.0254030 0.2903226
5772 hours 20421540 4519.020 3499.653 0.0237404 0.0197556 0.0239631 0.3440860
5784 hours 21911436 4680.965 3639.825 0.0247012 0.0191914 0.0249144 0.3306452
5796 hours 23601327 4858.120 3958.128 0.0269945 0.0259075 0.0272247 0.2500000
5808 hours 25180261 5017.994 4119.196 0.0281903 0.0223888 0.0284723 0.2258065
5820 hours 27535914 5247.467 4224.078 0.0290345 0.0223888 0.0294426 0.2258065
5832 hours 27644336 5257.788 4235.336 0.0290310 0.0223888 0.0294559 0.2258065
5844 hours 27326546 5227.480 4203.571 0.0288354 0.0209749 0.0292542 0.2258065
6480 hours 23858964 4884.564 3851.119 0.0261895 0.0205783 0.0264908 0.2661290
6504 hours 26782552 5175.186 4169.212 0.0282877 0.0248724 0.0285277 0.2473118
6516 hours 26994411 5195.615 4210.219 0.0285457 0.0260384 0.0287818 0.2177419
6528 hours 29529574 5434.112 4456.319 0.0305953 0.0224780 0.0310035 0.1693548
6540 hours 29242776 5407.659 4454.195 0.0306165 0.0260384 0.0310294 0.1612903
6552 hours 29074404 5392.069 4442.977 0.0303074 0.0260384 0.0307263 0.1612903
6564 hours 30811671 5550.826 4527.305 0.0308395 0.0260384 0.0313125 0.1612903
6588 hours 29796426 5458.610 4469.971 0.0303514 0.0252500 0.0307951 0.1612903
7224 hours 26723551 5169.483 4124.258 0.0278534 0.0252500 0.0281878 0.2258065
7236 hours 26469106 5144.814 4052.574 0.0273652 0.0252500 0.0276596 0.2473118
7248 hours 27369799 5231.615 4161.627 0.0281096 0.0224780 0.0284085 0.2338710
7260 hours 28774453 5364.182 4437.877 0.0301684 0.0295566 0.0304749 0.1774194
7272 hours 31675690 5628.116 4639.708 0.0316485 0.0273066 0.0320463 0.1612903
7284 hours 36247338 6020.576 4872.729 0.0333665 0.0295566 0.0339473 0.1612903
7296 hours 36729861 6060.517 4903.854 0.0334686 0.0295566 0.0340743 0.1612903
7308 hours 36430203 6035.744 4890.075 0.0333955 0.0289497 0.0339958 0.1612903
7944 hours 31867621 5645.141 4469.368 0.0302531 0.0252500 0.0306949 0.2258065
7968 hours 34731632 5893.355 4790.511 0.0323881 0.0301323 0.0327692 0.2150538
7980 hours 35127034 5926.806 4856.017 0.0328047 0.0319223 0.0331780 0.2096774
7992 hours 39436339 6279.836 5174.038 0.0353709 0.0289497 0.0359697 0.1693548
8004 hours 39089330 6252.146 5143.341 0.0352034 0.0319223 0.0357977 0.1612903
8016 hours 38828669 6231.265 5147.085 0.0349462 0.0319223 0.0355429 0.1612903
8028 hours 40680158 6378.100 5251.867 0.0356294 0.0319223 0.0362751 0.1612903
8052 hours 39581249 6291.363 5210.728 0.0352145 0.0319223 0.0358262 0.1612903
8688 hours 35425308 5951.916 4810.196 0.0323424 0.0319223 0.0328037 0.2258065
8712 hours 38865763 6234.241 5151.744 0.0347855 0.0349362 0.0352124 0.2150538
8724 hours 36237637 6019.770 4868.439 0.0327822 0.0264664 0.0331871 0.2338710
8736 hours 42574661 6524.926 5351.314 0.0365124 0.0349362 0.0371695 0.1774194
8748 hours 39604089 6293.178 5186.128 0.0352838 0.0332404 0.0357510 0.1612903
8760 hours 39307861 6269.598 5196.775 0.0350580 0.0332404 0.0355287 0.1612903 
plot_cross_validation_metric(df_cv_mod1, metric = 'rmse')

performance_metrics(df.cv_holiday)%>%
  kbl(caption = "Table 6: Model 2 CV performance metrics")%>%
  kable_classic(full_width = F, html_font = "Cambria")
Table 6: Model 2 CV performance metrics
horizon mse rmse mae mape mdape smape coverage
1380 hours 7085812 2661.919 1847.714 0.0127661 0.0088902 0.0128621 0.1693548
1392 hours 6830350 2613.494 1828.086 0.0125604 0.0088902 0.0126436 0.1935484
1404 hours 7929313 2815.904 2004.096 0.0137685 0.0089556 0.0138474 0.1935484
1416 hours 8326729 2885.607 2134.955 0.0147409 0.0099573 0.0148185 0.2258065
1428 hours 8925242 2987.514 2227.183 0.0154213 0.0099573 0.0155174 0.2338710
1440 hours 9782313 3127.669 2321.821 0.0161352 0.0109473 0.0162752 0.2338710
1452 hours 9628944 3103.054 2311.548 0.0160471 0.0109473 0.0161895 0.2258065
1464 hours 9701966 3114.798 2331.786 0.0161688 0.0121212 0.0163127 0.2258065
2100 hours 8151873 2855.149 2072.723 0.0142187 0.0109473 0.0143113 0.2903226
2124 hours 10653239 3263.930 2488.605 0.0170862 0.0125826 0.0171767 0.2419355
2136 hours 10717861 3273.815 2522.003 0.0172941 0.0137316 0.0173834 0.2338710
2148 hours 11607770 3407.018 2606.955 0.0180729 0.0128247 0.0182391 0.2580645
2160 hours 11352351 3369.325 2607.621 0.0180847 0.0137316 0.0182502 0.2338710
2172 hours 11216270 3349.070 2586.079 0.0179203 0.0128247 0.0180888 0.2258065
2184 hours 11792074 3433.959 2619.637 0.0181315 0.0128247 0.0183149 0.2258065
2208 hours 11505619 3391.993 2605.975 0.0179717 0.0128247 0.0181463 0.2177419
2844 hours 10081178 3175.087 2346.764 0.0160457 0.0128247 0.0161706 0.2903226
2856 hours 10113834 3180.225 2369.322 0.0161690 0.0128247 0.0162876 0.2822581
2868 hours 10650577 3263.522 2476.660 0.0169122 0.0128247 0.0170360 0.2580645
2880 hours 10977307 3313.202 2621.293 0.0180028 0.0151552 0.0181192 0.2338710
2892 hours 12358409 3515.453 2777.737 0.0191477 0.0151552 0.0193032 0.2338710
2904 hours 14357527 3789.133 2931.750 0.0202899 0.0172276 0.0205209 0.2258065
2916 hours 14253307 3775.355 2913.183 0.0201432 0.0150671 0.0203787 0.2258065
2928 hours 14524051 3811.043 2968.882 0.0204882 0.0172276 0.0207292 0.2258065
3564 hours 12279925 3504.272 2647.313 0.0180714 0.0150671 0.0182375 0.2903226
3588 hours 15215440 3900.697 3084.814 0.0211192 0.0181413 0.0212932 0.2177419
3600 hours 15354601 3918.495 3130.387 0.0214060 0.0182110 0.0215772 0.2096774
3612 hours 17190227 4146.110 3253.759 0.0224584 0.0182110 0.0227317 0.2258065
3624 hours 16918182 4113.172 3218.015 0.0222109 0.0187418 0.0224809 0.2580645
3636 hours 16781197 4096.486 3194.597 0.0220248 0.0152658 0.0223010 0.2580645
3648 hours 17269927 4155.710 3217.739 0.0221652 0.0156254 0.0224531 0.2580645
3672 hours 17037972 4127.708 3227.050 0.0221259 0.0156254 0.0224043 0.2580645
4308 hours 14889876 3858.740 2929.773 0.0199010 0.0156254 0.0201010 0.3548387
4332 hours 17594367 4194.564 3322.505 0.0226795 0.0203274 0.0228883 0.2822581
4344 hours 15710286 3963.620 3077.467 0.0209027 0.0172238 0.0210921 0.3225806
4356 hours 19101619 4370.540 3391.115 0.0233096 0.0187418 0.0236241 0.3064516
4368 hours 17969227 4239.013 3358.235 0.0230029 0.0222755 0.0232267 0.2983871
4380 hours 17227279 4150.576 3303.528 0.0225634 0.0222755 0.0227665 0.2903226
4392 hours 19851147 4455.463 3518.704 0.0241879 0.0175517 0.0245044 0.2580645
4416 hours 20588467 4537.452 3617.151 0.0247918 0.0234997 0.0251231 0.2580645
5016 hours 18019475 4244.935 3322.038 0.0225760 0.0175517 0.0228247 0.2983871
5040 hours 19584840 4425.476 3542.102 0.0240430 0.0234997 0.0242296 0.2741935
5052 hours 18301392 4278.013 3338.617 0.0225972 0.0197556 0.0227711 0.3225806
5064 hours 21050377 4588.069 3610.164 0.0247866 0.0234997 0.0250675 0.2903226
5076 hours 21696987 4658.003 3789.650 0.0259998 0.0259075 0.0262356 0.2419355
5088 hours 20711486 4550.987 3754.801 0.0255614 0.0259075 0.0257638 0.2258065
5100 hours 24205695 4919.928 3995.778 0.0273792 0.0259075 0.0277396 0.2258065
5124 hours 24254011 4924.836 4004.360 0.0274321 0.0259075 0.0277935 0.2258065
5760 hours 21769420 4665.771 3698.630 0.0251265 0.0197556 0.0254030 0.2903226
5772 hours 20421540 4519.020 3499.653 0.0237404 0.0197556 0.0239631 0.3440860
5784 hours 21911436 4680.965 3639.825 0.0247012 0.0191914 0.0249144 0.3306452
5796 hours 23601327 4858.120 3958.128 0.0269945 0.0259075 0.0272247 0.2500000
5808 hours 25180261 5017.994 4119.196 0.0281903 0.0223888 0.0284723 0.2258065
5820 hours 27535914 5247.467 4224.078 0.0290345 0.0223888 0.0294426 0.2258065
5832 hours 27644336 5257.788 4235.336 0.0290310 0.0223888 0.0294559 0.2258065
5844 hours 27326546 5227.480 4203.571 0.0288354 0.0209749 0.0292542 0.2258065
6480 hours 23858964 4884.564 3851.119 0.0261895 0.0205783 0.0264908 0.2661290
6504 hours 26782552 5175.186 4169.212 0.0282877 0.0248724 0.0285277 0.2473118
6516 hours 26994411 5195.615 4210.219 0.0285457 0.0260384 0.0287818 0.2177419
6528 hours 29529574 5434.112 4456.319 0.0305953 0.0224780 0.0310035 0.1693548
6540 hours 29242776 5407.659 4454.195 0.0306165 0.0260384 0.0310294 0.1612903
6552 hours 29074404 5392.069 4442.977 0.0303074 0.0260384 0.0307263 0.1612903
6564 hours 30811671 5550.826 4527.305 0.0308395 0.0260384 0.0313125 0.1612903
6588 hours 29796426 5458.610 4469.971 0.0303514 0.0252500 0.0307951 0.1612903
7224 hours 26723551 5169.483 4124.258 0.0278534 0.0252500 0.0281878 0.2258065
7236 hours 26469106 5144.814 4052.574 0.0273652 0.0252500 0.0276596 0.2473118
7248 hours 27369799 5231.615 4161.627 0.0281096 0.0224780 0.0284085 0.2338710
7260 hours 28774453 5364.182 4437.877 0.0301684 0.0295566 0.0304749 0.1774194
7272 hours 31675690 5628.116 4639.708 0.0316485 0.0273066 0.0320463 0.1612903
7284 hours 36247338 6020.576 4872.729 0.0333665 0.0295566 0.0339473 0.1612903
7296 hours 36729861 6060.517 4903.854 0.0334686 0.0295566 0.0340743 0.1612903
7308 hours 36430203 6035.744 4890.075 0.0333955 0.0289497 0.0339958 0.1612903
7944 hours 31867621 5645.141 4469.368 0.0302531 0.0252500 0.0306949 0.2258065
7968 hours 34731632 5893.355 4790.511 0.0323881 0.0301323 0.0327692 0.2150538
7980 hours 35127034 5926.806 4856.017 0.0328047 0.0319223 0.0331780 0.2096774
7992 hours 39436339 6279.836 5174.038 0.0353709 0.0289497 0.0359697 0.1693548
8004 hours 39089330 6252.146 5143.341 0.0352034 0.0319223 0.0357977 0.1612903
8016 hours 38828669 6231.265 5147.085 0.0349462 0.0319223 0.0355429 0.1612903
8028 hours 40680158 6378.100 5251.867 0.0356294 0.0319223 0.0362751 0.1612903
8052 hours 39581249 6291.363 5210.728 0.0352145 0.0319223 0.0358262 0.1612903
8688 hours 35425308 5951.916 4810.196 0.0323424 0.0319223 0.0328037 0.2258065
8712 hours 38865763 6234.241 5151.744 0.0347855 0.0349362 0.0352124 0.2150538
8724 hours 36237637 6019.770 4868.439 0.0327822 0.0264664 0.0331871 0.2338710
8736 hours 42574661 6524.926 5351.314 0.0365124 0.0349362 0.0371695 0.1774194
8748 hours 39604089 6293.178 5186.128 0.0352838 0.0332404 0.0357510 0.1612903
8760 hours 39307861 6269.598 5196.775 0.0350580 0.0332404 0.0355287 0.1612903 
plot_cross_validation_metric(df_cv_mod2, metric = 'rmse')

performance_metrics(df.cv_holiday)%>%
  kbl(caption = "Table 7: Model 3 CV performance metrics")%>%
  kable_classic(full_width = F, html_font = "Cambria")
Table 7: Model 3 CV performance metrics
horizon mse rmse mae mape mdape smape coverage
1380 hours 7085812 2661.919 1847.714 0.0127661 0.0088902 0.0128621 0.1693548
1392 hours 6830350 2613.494 1828.086 0.0125604 0.0088902 0.0126436 0.1935484
1404 hours 7929313 2815.904 2004.096 0.0137685 0.0089556 0.0138474 0.1935484
1416 hours 8326729 2885.607 2134.955 0.0147409 0.0099573 0.0148185 0.2258065
1428 hours 8925242 2987.514 2227.183 0.0154213 0.0099573 0.0155174 0.2338710
1440 hours 9782313 3127.669 2321.821 0.0161352 0.0109473 0.0162752 0.2338710
1452 hours 9628944 3103.054 2311.548 0.0160471 0.0109473 0.0161895 0.2258065
1464 hours 9701966 3114.798 2331.786 0.0161688 0.0121212 0.0163127 0.2258065
2100 hours 8151873 2855.149 2072.723 0.0142187 0.0109473 0.0143113 0.2903226
2124 hours 10653239 3263.930 2488.605 0.0170862 0.0125826 0.0171767 0.2419355
2136 hours 10717861 3273.815 2522.003 0.0172941 0.0137316 0.0173834 0.2338710
2148 hours 11607770 3407.018 2606.955 0.0180729 0.0128247 0.0182391 0.2580645
2160 hours 11352351 3369.325 2607.621 0.0180847 0.0137316 0.0182502 0.2338710
2172 hours 11216270 3349.070 2586.079 0.0179203 0.0128247 0.0180888 0.2258065
2184 hours 11792074 3433.959 2619.637 0.0181315 0.0128247 0.0183149 0.2258065
2208 hours 11505619 3391.993 2605.975 0.0179717 0.0128247 0.0181463 0.2177419
2844 hours 10081178 3175.087 2346.764 0.0160457 0.0128247 0.0161706 0.2903226
2856 hours 10113834 3180.225 2369.322 0.0161690 0.0128247 0.0162876 0.2822581
2868 hours 10650577 3263.522 2476.660 0.0169122 0.0128247 0.0170360 0.2580645
2880 hours 10977307 3313.202 2621.293 0.0180028 0.0151552 0.0181192 0.2338710
2892 hours 12358409 3515.453 2777.737 0.0191477 0.0151552 0.0193032 0.2338710
2904 hours 14357527 3789.133 2931.750 0.0202899 0.0172276 0.0205209 0.2258065
2916 hours 14253307 3775.355 2913.183 0.0201432 0.0150671 0.0203787 0.2258065
2928 hours 14524051 3811.043 2968.882 0.0204882 0.0172276 0.0207292 0.2258065
3564 hours 12279925 3504.272 2647.313 0.0180714 0.0150671 0.0182375 0.2903226
3588 hours 15215440 3900.697 3084.814 0.0211192 0.0181413 0.0212932 0.2177419
3600 hours 15354601 3918.495 3130.387 0.0214060 0.0182110 0.0215772 0.2096774
3612 hours 17190227 4146.110 3253.759 0.0224584 0.0182110 0.0227317 0.2258065
3624 hours 16918182 4113.172 3218.015 0.0222109 0.0187418 0.0224809 0.2580645
3636 hours 16781197 4096.486 3194.597 0.0220248 0.0152658 0.0223010 0.2580645
3648 hours 17269927 4155.710 3217.739 0.0221652 0.0156254 0.0224531 0.2580645
3672 hours 17037972 4127.708 3227.050 0.0221259 0.0156254 0.0224043 0.2580645
4308 hours 14889876 3858.740 2929.773 0.0199010 0.0156254 0.0201010 0.3548387
4332 hours 17594367 4194.564 3322.505 0.0226795 0.0203274 0.0228883 0.2822581
4344 hours 15710286 3963.620 3077.467 0.0209027 0.0172238 0.0210921 0.3225806
4356 hours 19101619 4370.540 3391.115 0.0233096 0.0187418 0.0236241 0.3064516
4368 hours 17969227 4239.013 3358.235 0.0230029 0.0222755 0.0232267 0.2983871
4380 hours 17227279 4150.576 3303.528 0.0225634 0.0222755 0.0227665 0.2903226
4392 hours 19851147 4455.463 3518.704 0.0241879 0.0175517 0.0245044 0.2580645
4416 hours 20588467 4537.452 3617.151 0.0247918 0.0234997 0.0251231 0.2580645
5016 hours 18019475 4244.935 3322.038 0.0225760 0.0175517 0.0228247 0.2983871
5040 hours 19584840 4425.476 3542.102 0.0240430 0.0234997 0.0242296 0.2741935
5052 hours 18301392 4278.013 3338.617 0.0225972 0.0197556 0.0227711 0.3225806
5064 hours 21050377 4588.069 3610.164 0.0247866 0.0234997 0.0250675 0.2903226
5076 hours 21696987 4658.003 3789.650 0.0259998 0.0259075 0.0262356 0.2419355
5088 hours 20711486 4550.987 3754.801 0.0255614 0.0259075 0.0257638 0.2258065
5100 hours 24205695 4919.928 3995.778 0.0273792 0.0259075 0.0277396 0.2258065
5124 hours 24254011 4924.836 4004.360 0.0274321 0.0259075 0.0277935 0.2258065
5760 hours 21769420 4665.771 3698.630 0.0251265 0.0197556 0.0254030 0.2903226
5772 hours 20421540 4519.020 3499.653 0.0237404 0.0197556 0.0239631 0.3440860
5784 hours 21911436 4680.965 3639.825 0.0247012 0.0191914 0.0249144 0.3306452
5796 hours 23601327 4858.120 3958.128 0.0269945 0.0259075 0.0272247 0.2500000
5808 hours 25180261 5017.994 4119.196 0.0281903 0.0223888 0.0284723 0.2258065
5820 hours 27535914 5247.467 4224.078 0.0290345 0.0223888 0.0294426 0.2258065
5832 hours 27644336 5257.788 4235.336 0.0290310 0.0223888 0.0294559 0.2258065
5844 hours 27326546 5227.480 4203.571 0.0288354 0.0209749 0.0292542 0.2258065
6480 hours 23858964 4884.564 3851.119 0.0261895 0.0205783 0.0264908 0.2661290
6504 hours 26782552 5175.186 4169.212 0.0282877 0.0248724 0.0285277 0.2473118
6516 hours 26994411 5195.615 4210.219 0.0285457 0.0260384 0.0287818 0.2177419
6528 hours 29529574 5434.112 4456.319 0.0305953 0.0224780 0.0310035 0.1693548
6540 hours 29242776 5407.659 4454.195 0.0306165 0.0260384 0.0310294 0.1612903
6552 hours 29074404 5392.069 4442.977 0.0303074 0.0260384 0.0307263 0.1612903
6564 hours 30811671 5550.826 4527.305 0.0308395 0.0260384 0.0313125 0.1612903
6588 hours 29796426 5458.610 4469.971 0.0303514 0.0252500 0.0307951 0.1612903
7224 hours 26723551 5169.483 4124.258 0.0278534 0.0252500 0.0281878 0.2258065
7236 hours 26469106 5144.814 4052.574 0.0273652 0.0252500 0.0276596 0.2473118
7248 hours 27369799 5231.615 4161.627 0.0281096 0.0224780 0.0284085 0.2338710
7260 hours 28774453 5364.182 4437.877 0.0301684 0.0295566 0.0304749 0.1774194
7272 hours 31675690 5628.116 4639.708 0.0316485 0.0273066 0.0320463 0.1612903
7284 hours 36247338 6020.576 4872.729 0.0333665 0.0295566 0.0339473 0.1612903
7296 hours 36729861 6060.517 4903.854 0.0334686 0.0295566 0.0340743 0.1612903
7308 hours 36430203 6035.744 4890.075 0.0333955 0.0289497 0.0339958 0.1612903
7944 hours 31867621 5645.141 4469.368 0.0302531 0.0252500 0.0306949 0.2258065
7968 hours 34731632 5893.355 4790.511 0.0323881 0.0301323 0.0327692 0.2150538
7980 hours 35127034 5926.806 4856.017 0.0328047 0.0319223 0.0331780 0.2096774
7992 hours 39436339 6279.836 5174.038 0.0353709 0.0289497 0.0359697 0.1693548
8004 hours 39089330 6252.146 5143.341 0.0352034 0.0319223 0.0357977 0.1612903
8016 hours 38828669 6231.265 5147.085 0.0349462 0.0319223 0.0355429 0.1612903
8028 hours 40680158 6378.100 5251.867 0.0356294 0.0319223 0.0362751 0.1612903
8052 hours 39581249 6291.363 5210.728 0.0352145 0.0319223 0.0358262 0.1612903
8688 hours 35425308 5951.916 4810.196 0.0323424 0.0319223 0.0328037 0.2258065
8712 hours 38865763 6234.241 5151.744 0.0347855 0.0349362 0.0352124 0.2150538
8724 hours 36237637 6019.770 4868.439 0.0327822 0.0264664 0.0331871 0.2338710
8736 hours 42574661 6524.926 5351.314 0.0365124 0.0349362 0.0371695 0.1774194
8748 hours 39604089 6293.178 5186.128 0.0352838 0.0332404 0.0357510 0.1612903
8760 hours 39307861 6269.598 5196.775 0.0350580 0.0332404 0.0355287 0.1612903 
plot_cross_validation_metric(df_cv_mod3, metric = 'rmse')

performance_metrics(df.cv_holiday)%>%
  kbl(caption = "Table 8: Model 4 CV performance metrics")%>%
  kable_classic(full_width = F, html_font = "Cambria")
Table 8: Model 4 CV performance metrics
horizon mse rmse mae mape mdape smape coverage
1380 hours 7085812 2661.919 1847.714 0.0127661 0.0088902 0.0128621 0.1693548
1392 hours 6830350 2613.494 1828.086 0.0125604 0.0088902 0.0126436 0.1935484
1404 hours 7929313 2815.904 2004.096 0.0137685 0.0089556 0.0138474 0.1935484
1416 hours 8326729 2885.607 2134.955 0.0147409 0.0099573 0.0148185 0.2258065
1428 hours 8925242 2987.514 2227.183 0.0154213 0.0099573 0.0155174 0.2338710
1440 hours 9782313 3127.669 2321.821 0.0161352 0.0109473 0.0162752 0.2338710
1452 hours 9628944 3103.054 2311.548 0.0160471 0.0109473 0.0161895 0.2258065
1464 hours 9701966 3114.798 2331.786 0.0161688 0.0121212 0.0163127 0.2258065
2100 hours 8151873 2855.149 2072.723 0.0142187 0.0109473 0.0143113 0.2903226
2124 hours 10653239 3263.930 2488.605 0.0170862 0.0125826 0.0171767 0.2419355
2136 hours 10717861 3273.815 2522.003 0.0172941 0.0137316 0.0173834 0.2338710
2148 hours 11607770 3407.018 2606.955 0.0180729 0.0128247 0.0182391 0.2580645
2160 hours 11352351 3369.325 2607.621 0.0180847 0.0137316 0.0182502 0.2338710
2172 hours 11216270 3349.070 2586.079 0.0179203 0.0128247 0.0180888 0.2258065
2184 hours 11792074 3433.959 2619.637 0.0181315 0.0128247 0.0183149 0.2258065
2208 hours 11505619 3391.993 2605.975 0.0179717 0.0128247 0.0181463 0.2177419
2844 hours 10081178 3175.087 2346.764 0.0160457 0.0128247 0.0161706 0.2903226
2856 hours 10113834 3180.225 2369.322 0.0161690 0.0128247 0.0162876 0.2822581
2868 hours 10650577 3263.522 2476.660 0.0169122 0.0128247 0.0170360 0.2580645
2880 hours 10977307 3313.202 2621.293 0.0180028 0.0151552 0.0181192 0.2338710
2892 hours 12358409 3515.453 2777.737 0.0191477 0.0151552 0.0193032 0.2338710
2904 hours 14357527 3789.133 2931.750 0.0202899 0.0172276 0.0205209 0.2258065
2916 hours 14253307 3775.355 2913.183 0.0201432 0.0150671 0.0203787 0.2258065
2928 hours 14524051 3811.043 2968.882 0.0204882 0.0172276 0.0207292 0.2258065
3564 hours 12279925 3504.272 2647.313 0.0180714 0.0150671 0.0182375 0.2903226
3588 hours 15215440 3900.697 3084.814 0.0211192 0.0181413 0.0212932 0.2177419
3600 hours 15354601 3918.495 3130.387 0.0214060 0.0182110 0.0215772 0.2096774
3612 hours 17190227 4146.110 3253.759 0.0224584 0.0182110 0.0227317 0.2258065
3624 hours 16918182 4113.172 3218.015 0.0222109 0.0187418 0.0224809 0.2580645
3636 hours 16781197 4096.486 3194.597 0.0220248 0.0152658 0.0223010 0.2580645
3648 hours 17269927 4155.710 3217.739 0.0221652 0.0156254 0.0224531 0.2580645
3672 hours 17037972 4127.708 3227.050 0.0221259 0.0156254 0.0224043 0.2580645
4308 hours 14889876 3858.740 2929.773 0.0199010 0.0156254 0.0201010 0.3548387
4332 hours 17594367 4194.564 3322.505 0.0226795 0.0203274 0.0228883 0.2822581
4344 hours 15710286 3963.620 3077.467 0.0209027 0.0172238 0.0210921 0.3225806
4356 hours 19101619 4370.540 3391.115 0.0233096 0.0187418 0.0236241 0.3064516
4368 hours 17969227 4239.013 3358.235 0.0230029 0.0222755 0.0232267 0.2983871
4380 hours 17227279 4150.576 3303.528 0.0225634 0.0222755 0.0227665 0.2903226
4392 hours 19851147 4455.463 3518.704 0.0241879 0.0175517 0.0245044 0.2580645
4416 hours 20588467 4537.452 3617.151 0.0247918 0.0234997 0.0251231 0.2580645
5016 hours 18019475 4244.935 3322.038 0.0225760 0.0175517 0.0228247 0.2983871
5040 hours 19584840 4425.476 3542.102 0.0240430 0.0234997 0.0242296 0.2741935
5052 hours 18301392 4278.013 3338.617 0.0225972 0.0197556 0.0227711 0.3225806
5064 hours 21050377 4588.069 3610.164 0.0247866 0.0234997 0.0250675 0.2903226
5076 hours 21696987 4658.003 3789.650 0.0259998 0.0259075 0.0262356 0.2419355
5088 hours 20711486 4550.987 3754.801 0.0255614 0.0259075 0.0257638 0.2258065
5100 hours 24205695 4919.928 3995.778 0.0273792 0.0259075 0.0277396 0.2258065
5124 hours 24254011 4924.836 4004.360 0.0274321 0.0259075 0.0277935 0.2258065
5760 hours 21769420 4665.771 3698.630 0.0251265 0.0197556 0.0254030 0.2903226
5772 hours 20421540 4519.020 3499.653 0.0237404 0.0197556 0.0239631 0.3440860
5784 hours 21911436 4680.965 3639.825 0.0247012 0.0191914 0.0249144 0.3306452
5796 hours 23601327 4858.120 3958.128 0.0269945 0.0259075 0.0272247 0.2500000
5808 hours 25180261 5017.994 4119.196 0.0281903 0.0223888 0.0284723 0.2258065
5820 hours 27535914 5247.467 4224.078 0.0290345 0.0223888 0.0294426 0.2258065
5832 hours 27644336 5257.788 4235.336 0.0290310 0.0223888 0.0294559 0.2258065
5844 hours 27326546 5227.480 4203.571 0.0288354 0.0209749 0.0292542 0.2258065
6480 hours 23858964 4884.564 3851.119 0.0261895 0.0205783 0.0264908 0.2661290
6504 hours 26782552 5175.186 4169.212 0.0282877 0.0248724 0.0285277 0.2473118
6516 hours 26994411 5195.615 4210.219 0.0285457 0.0260384 0.0287818 0.2177419
6528 hours 29529574 5434.112 4456.319 0.0305953 0.0224780 0.0310035 0.1693548
6540 hours 29242776 5407.659 4454.195 0.0306165 0.0260384 0.0310294 0.1612903
6552 hours 29074404 5392.069 4442.977 0.0303074 0.0260384 0.0307263 0.1612903
6564 hours 30811671 5550.826 4527.305 0.0308395 0.0260384 0.0313125 0.1612903
6588 hours 29796426 5458.610 4469.971 0.0303514 0.0252500 0.0307951 0.1612903
7224 hours 26723551 5169.483 4124.258 0.0278534 0.0252500 0.0281878 0.2258065
7236 hours 26469106 5144.814 4052.574 0.0273652 0.0252500 0.0276596 0.2473118
7248 hours 27369799 5231.615 4161.627 0.0281096 0.0224780 0.0284085 0.2338710
7260 hours 28774453 5364.182 4437.877 0.0301684 0.0295566 0.0304749 0.1774194
7272 hours 31675690 5628.116 4639.708 0.0316485 0.0273066 0.0320463 0.1612903
7284 hours 36247338 6020.576 4872.729 0.0333665 0.0295566 0.0339473 0.1612903
7296 hours 36729861 6060.517 4903.854 0.0334686 0.0295566 0.0340743 0.1612903
7308 hours 36430203 6035.744 4890.075 0.0333955 0.0289497 0.0339958 0.1612903
7944 hours 31867621 5645.141 4469.368 0.0302531 0.0252500 0.0306949 0.2258065
7968 hours 34731632 5893.355 4790.511 0.0323881 0.0301323 0.0327692 0.2150538
7980 hours 35127034 5926.806 4856.017 0.0328047 0.0319223 0.0331780 0.2096774
7992 hours 39436339 6279.836 5174.038 0.0353709 0.0289497 0.0359697 0.1693548
8004 hours 39089330 6252.146 5143.341 0.0352034 0.0319223 0.0357977 0.1612903
8016 hours 38828669 6231.265 5147.085 0.0349462 0.0319223 0.0355429 0.1612903
8028 hours 40680158 6378.100 5251.867 0.0356294 0.0319223 0.0362751 0.1612903
8052 hours 39581249 6291.363 5210.728 0.0352145 0.0319223 0.0358262 0.1612903
8688 hours 35425308 5951.916 4810.196 0.0323424 0.0319223 0.0328037 0.2258065
8712 hours 38865763 6234.241 5151.744 0.0347855 0.0349362 0.0352124 0.2150538
8724 hours 36237637 6019.770 4868.439 0.0327822 0.0264664 0.0331871 0.2338710
8736 hours 42574661 6524.926 5351.314 0.0365124 0.0349362 0.0371695 0.1774194
8748 hours 39604089 6293.178 5186.128 0.0352838 0.0332404 0.0357510 0.1612903
8760 hours 39307861 6269.598 5196.775 0.0350580 0.0332404 0.0355287 0.1612903 
plot_cross_validation_metric(df_cv_mod4, metric = 'rmse')

Cross-validation results for monthly data is not very reasonable. Maybe because we set the CV freq to ‘Daily’. All models seems to fit pretty well and have similar RMSE as well as other performing metrics.

reference