Question 8.1

Consider the the number of pigs slaughtered in Victoria, available in the aus_livestock dataset.

Part a

Use the ETS() function to estimate the equivalent model for simple exponential smoothing. Find the optimal values of \(\alpha\) and \(\ell_0\), and generate forecasts for the next four months.

In the graph above forecasted using all the data going back to 1973, but only show the last 4 years for clarity in seeing the forecasted months. The orange line shows all the forecasted values that the ETS model created. Using this ETS() approach we got that the optimal \(\alpha\) is equal to 0.322 and \(\ell_0\) equal to 100,646.6. We can feel confident that future pig slaughters in the 80% level, which encompasses most of the peaks and dips from previous observations.

Part b

Compute a 95% prediction interval for the first forecast using \(\hat y \pm 1.96s\) where \(s\) is the standard deviation of the residuals. Compare your interval with the interval produced by R.

Month Forecasted 95 Lo 95 Hi Calculated Lo 95 Calculated Hi 95
2019 Jan 95186.56 76854.79 113518.33 76871.01 113502.1
2019 Feb 95186.56 75927.17 114445.95 76871.01 113502.1
2019 Mar 95186.56 75042.22 115330.9 76871.01 113502.1
2019 Apr 95186.56 74194.54 116178.58 76871.01 113502.1

By manually calculating the 95% confidence intervals you lose the increase and decrease for each additional t+1. If you look at the table above the first set of Hi/Lo intervals are calculated by the ETS model and they increase and decrease over the course of the 4 forecasted months, however, when you calculate it manually they stay the same over the course of the forecasted months.

Question 8.5

Data set global_economy contains the annual Exports from many countries. Select one country to analyse.

Part a

Plot the Exports series and discuss the main features of the data.

The graph above shows US exports from 1960 to 2017. We can clearly see an upward trend and an 10-year cycle of highs and lows.

Part b

Use an ETS(A,N,N) model to forecast the series, and plot the forecasts.

This model seems to do a pretty good job at following historic data and the model seems like it could be an accurate prediction given the 80 and 90% levels. However there are other ways to check model accuracy.

Part c

Compute the RMSE values for the training data.

ETS(A,N,N) Model
.type ME RMSE MAE MPE MAPE MASE RMSSE ACF1
Training 0.125 0.627 0.465 1.371 5.097 0.99 0.992 0.239

The RMSE for the initial model in part B, the ETS(A,N,N) model is 0.627. This is relatively low and can be used to compare this model to a different model.

Part d

Compare the results to those from an ETS(A,A,N) model. (Remember that the trended model is using one more parameter than the simpler model.) Discuss the merits of the two forecasting methods for this data set.

Model accuracy metrics
Country .model .type ME RMSE MAE MPE MAPE MASE RMSSE ACF1
United States part_b Training 0.125 0.627 0.465 1.371 5.097 0.990 0.992 0.239
United States part_d Training 0.011 0.615 0.466 -0.057 5.188 0.992 0.973 0.238

Comparing the ETS(A,N,N) and the ETS(A,A,N) model RMSE we see that the ETS(A,A,N) model has a slighter better RMSE of 0.615 compared to the RMSE of 0.627 for the ETS(A,N,N). Since the RMSE’s are fairly close the next step is to compare the forecasts to understand sensibility, i.e. which forecasted line makes the most sense.

Part e

Compare the forecasts from both methods. Which do you think is best?

Both methods very accurately predict the historical data, but when it comes to the future forecasts they differ slightly. The SES predicts that the future values will be static at around 12% of GDP for the next 4 years. On the other hand the Holt method continues the upward trend and assumes that it will continue upwards at the same rate for the next 4 years. It seems that neither of these forecasts are sensible as it seems like exports are decreasing for 2017, however, I think that the SES is a more sensible forecast of the two.

Part f

Calculate a 95% prediction interval for the first forecast for each model, using the RMSE values and assuming normal errors. Compare your intervals with those produced using R.

95% CI levels automated and manually calculated
model Year mean lo95 hi95 man_lo95 man_hi95
ETS(A,N,N) 2017 11.891 10.640 13.142 10.662 13.120
ETS(A,A,N) 2017 12.007 10.757 13.257 10.801 13.212

In the table above we see that when calculating the manually 96% CI levels using the RMSE for each model we get a very similar result.

Question 8.6

Let’s explore what the GDP looks like:

There seems to be an exponential growth in GDP for China. Let’s first control for population growth as well by calculating GDP per capita.

The population had no effect on the exponential growth. This data might be better served by a log transformation. (I will continue to use GDP per capita going forward as it makes more sense than the raw GDP number)

By transforming the data, we have a much nicer line now showing an upward trend in GDP per capita in China from 1960 to 2017.

Let’s first compare the forecasts using the none transformed data:

Using the non transformed data we see that the Holt and Damped methods are much more sensible that the SES method. Assuming that the exponential trend will continue, and there is no reason to not think that it can based on the Chinese economy, the Damped or Holt methods are best. I prefer the damped method because there is no reason to assume that the growth will happen at the same rate.

Now let’s look at the logged GDP per capita:

In the logged GDP per capita we can see a clearer difference between the Holt and Damped methods. I still prefer the Damped method as it doesn’t continue the upward trend at the same rate. We can compare accuracy scores for all the models now:

Accuracy scores for all models
.model AIC BIC AICc RMSE
SES 902.507827 908.689156 902.952272 298.3580419
Holt 819.631406 829.933621 820.785252 141.0845813
Damped 821.546796 833.909454 823.193855 140.9817124
SES_log -2.289588 3.891741 -1.845144 0.1222504
Holt_log -34.947806 -24.645591 -33.793960 0.0891261
Damped_log -32.607286 -20.244628 -30.960227 0.0893882

If we do not transform the data Holt’s method is the best forecasting method, while the best method for the logged data is the Damped method. However, I would say that both methods are fairly close in accuracy scores so it comes down to the assumption of how constant China’s GDP growth will be.

Question 8.7

Find an ETS model for the Gas data from aus_production and forecast the next few years. Why is multiplicative seasonality necessary here? Experiment with making the trend damped. Does it improve the forecasts?

The first thing that stands out for this data is the variation in the seasonal trend. As time increases we see that the variation in the seasonal trend also increases. For this reason the ETS model should include a multiplicative season component.

Looking at the graph there seems to be no difference when comparing the Multiplicative Holt-Winters’ method with the Holt-Winters’ damped method. Let’s check if there is a difference using the AIC/BIC/AICc metrics.

Model performance metrics
model AIC AICc BIC
Multiplicative 1680.929 1681.794 1711.389
Damped 1684.028 1685.091 1717.873

Comparing the model score metrics we see that the Multiplicative Holt-Winters’ method performs slightly better than the Holt-Winters’ damped method. Given that dampening the trend component does not improve the model there is little reason to complicate the model and using the simpler standard Multiplicative Holt-Winters’ method is advised.

Question 8.8

Recall your retail time series data (from Exercise 8 in Section 2.10).

Part a

Why is multiplicative seasonality necessary for this series?

Looking at the graph above we see that a multiplicative season component in an ETS model makes sense because the variation in the seasonality increases as the time increases.

Part b

Apply Holt-Winters’ multiplicative method to the data. Experiment with making the trend damped.

It’s very difficult to distinguish between the two lines on the graph above. For this reason we cannot discern any noticeable difference between the two models and their fit of the data.

Part c

Compare the RMSE of the one-step forecasts from the two methods. Which do you prefer?

Accuracy scores
.model RMSE
Multiplicative 1.342
Damped 1.356

Comparing the RMSE for the one-step forecast we can see that the Multiplicative Holt-Winters’ method fits the data slightly better than the Holt-Winters’ damped method. For this reason I prefer the Multiplicative Holt-Winters’ method.

Part d

Check that the residuals from the best method look like white noise.

Looking at the first plot above we see that the residuals are white noise and look random. We can also see that they are normally distributed in the histogram above.

Part e

Now find the test set RMSE, while training the model to the end of 2010. Can you beat the seasonal naïve approach from Exercise 7 in Section 5.11?

RMSE for the SNAIVE and Holt-Winter Methods
model rmse
train_snaive 9.129
train_holt 3.987

Based on the graph above we can clearly see that the Multiplicative Holt-Winters’ method is better than the simpler SNAIVE method.

Additionally, comparing the RMSE for the SNAIVE of 9.129 with that of the Multiplicative Holt-Winters’ method of 3.987, we can see that the Multiplicative Holt-Winters’ method does a much better job of predicting the future forecasts than the simple SNAIVE method from chapter 5. This decrease in RMSE warrants using a more complex method.

Question 8.9

For the same retail data, try an STL decomposition applied to the Box-Cox transformed series, followed by ETS on the seasonally adjusted data. How does that compare with your best previous forecasts on the test set?

First we want to autoplot the STL decomposition of the myseries_train data split in Exercise 8.8. Using the methods we learned in Chapter 3 on decomposition we find that the lambda is 0.269. When creating our ETS model we now have to remember that since the transformation normalized the variation in the seasonality we no longer have to use an ETS(M,A,M) model but can use an ETS(A,A,A) model.

We can now look at the new Holt model after using a Box-Cox transformation and compare it to the Multiplicative Holt-Winters’ method.

In the graph the Multiplicative Holt-Winters’ method seems to perform better than the Box-Cox transformation model, ETS(A,A,A) (the orange line). But lets make sure that this is accurate and compare the RMSE’s.

RMSE for the Holt-Winter and Holt-Winter Box-Cox Methods
model rmse
train_snaive 9.129
train_holt 3.987
train_boxcox 5.537

Comparing the RMSE’s of the three methods looked at, SNIAVE, Holt-Winters, and Holt-Winter Box-Cox, we can see that the best model to use for forecasting retail employment in the US is the Multiplicative Holt-Winters’ method.