library(fpp3)
library(fredr)
unrate_raw <- fredr(
series_id = "UNRATE",
observation_start = as.Date("2000-01-01"),
observation_end = as.Date("2020-12-01")
)
unrate <- unrate_raw %>%
select(date, value) %>%
mutate(date = yearmonth(date)) %>%
as_tsibble(index = date)
n <- nrow(unrate)
split_point <- floor(0.8 * n)
train <- unrate %>% slice(1:split_point)
test <- unrate %>% slice((split_point + 1):n)ADEC7406 Module 9 Discussion
Part I
Preparation
I chose to useUNRATE (U.S. Unemployment Rate) as the data set for this discussion.
Model Fitting
fit <- train %>%
model(
ETS = ETS(value),
ARIMA = ARIMA(value)
)
fit %>%
select(ETS) %>%
report()Series: value
Model: ETS(A,Ad,N)
Smoothing parameters:
alpha = 0.8436003
beta = 0.2110355
phi = 0.9322862
Initial states:
l[0] b[0]
4.02554 -0.008788952
sigma^2: 0.0229
AIC AICc BIC
313.8253 314.2583 333.6452 fit %>%
select(ARIMA) %>%
report()Series: value
Model: ARIMA(1,2,1)(0,0,2)[12]
Coefficients:
ar1 ma1 sma1 sma2
-0.1382 -0.7671 -0.4277 -0.2627
s.e. 0.0827 0.0493 0.0687 0.0646
sigma^2 estimated as 0.01874: log likelihood=111.49
AIC=-212.99 AICc=-212.68 BIC=-196.52fc <- fit %>%
forecast(new_data = test)
fc %>%
autoplot(train, level = NULL) +
autolayer(test, value, colour = "black", linetype = "dashed") +
labs(
title = "ETS vs ARIMA Forecasts on UNRATE",
subtitle = "Black dashed line = actual test data",
x = "Year",
y = "Unemployment Rate"
) +
theme_minimal() +
theme(legend.position = "bottom")
acc <- accuracy(fc, test)
acc %>%
select(.model, RMSE, MAE, MAPE, ACF1)# A tibble: 2 × 5
.model RMSE MAE MAPE ACF1
<chr> <dbl> <dbl> <dbl> <dbl>
1 ARIMA 2.67 1.46 21.8 0.817
2 ETS 2.42 1.71 32.0 0.790I use monthly FRED data on the U.S. unemployment rate (UNRATE) from January 2000 to December 2020. The sample is divided into training and test sets, and both an ETS model and an ARIMA model are estimated on the training data. The selected ETS model is ETS(A,Ad,N), while the selected ARIMA model is ARIMA(1,2,1)(0,0,2)[12].
The ETS model suggests an additive error structure with a damped trend and no explicit seasonality. Its forecasts are relatively stable and slightly upward-sloping over the test period. By contrast, the ARIMA model uses second differencing together with non-seasonal AR and MA terms and seasonal MA terms, producing a downward-sloping forecast path. Thus, the two models imply different underlying data-generating structures and generate visibly different forecasts.
However, their out-of-sample forecast accuracy is almost identical. The RMSE, MAE, and MAPE values are nearly the same for both models, with ARIMA only performing marginally better. This suggests that, for this dataset, neither model has a meaningful forecasting advantage over the other. At the same time, the very large forecast errors reflect the unusual shock in 2020, which neither model could anticipate from past historical patterns alone.
Therefore, the models are not equivalent in a theoretical sense. ETS and ARIMA are based on different modeling frameworks: ETS relies on exponential smoothing components such as level and trend, while ARIMA models serial dependence through autoregressive, differencing, and moving-average dynamics. Although their forecast performance is very similar in this application, similar accuracy does not mean that the models are equivalent.
Part II
New Regressors
indpro_raw <- fredr(
series_id = "INDPRO",
observation_start = as.Date("2000-01-01"),
observation_end = as.Date("2020-12-01")
)
fedfunds_raw <- fredr(
series_id = "FEDFUNDS",
observation_start = as.Date("2000-01-01"),
observation_end = as.Date("2020-12-01")
)
xregs <- indpro_raw %>%
select(date, indpro = value) %>%
left_join(
fedfunds_raw %>% select(date, fedfunds = value),
by = "date"
) %>%
mutate(date = yearmonth(date)) %>%
as_tsibble(index = date)
# Now, we merge the two "new" regressors with the original UNRATE data
unrate_xreg <- unrate %>%
left_join(xregs, by = "date")
# Need to redo the train-test split after adding the new regressors
train_xreg <- unrate_xreg %>% slice(1:split_point)
test_xreg <- unrate_xreg %>% slice((split_point + 1):n)Dynamic Regression Models
fit_dr <- train_xreg %>%
model(
DR_INDPRO = ARIMA(value ~ indpro),
DR_BOTH = ARIMA(value ~ indpro + fedfunds)
)
fit_dr %>%
select(DR_INDPRO) %>%
report()Series: value
Model: LM w/ ARIMA(1,2,1)(0,0,2)[12] errors
Coefficients:
ar1 ma1 sma1 sma2 indpro
-0.1488 -0.7836 -0.3935 -0.2370 -0.0222
s.e. 0.0816 0.0488 0.0746 0.0674 0.0202
sigma^2 estimated as 0.01885: log likelihood=112.12
AIC=-212.24 AICc=-211.8 BIC=-192.48fit_dr %>%
select(DR_BOTH) %>%
report()Series: value
Model: LM w/ ARIMA(2,1,1)(0,0,2)[12] errors
Coefficients:
ar1 ar2 ma1 sma1 sma2 indpro fedfunds
0.8203 0.1462 -0.7817 -0.4028 -0.2429 -0.0222 -0.0953
s.e. 0.0862 0.0791 0.0565 0.0757 0.0687 0.0199 0.0637
sigma^2 estimated as 0.01849: log likelihood=116.14
AIC=-216.27 AICc=-215.52 BIC=-189.89Forecast
fc_dr <- fit_dr %>%
forecast(new_data = test_xreg)
acc_dr <- accuracy(fc_dr, test_xreg)
acc_dr %>%
select(.model, RMSE, MAE, MAPE, ACF1)# A tibble: 2 × 5
.model RMSE MAE MAPE ACF1
<chr> <dbl> <dbl> <dbl> <dbl>
1 DR_BOTH 2.28 1.50 26.7 0.781
2 DR_INDPRO 2.63 1.38 19.8 0.819Comparison
acc_all <- bind_rows(acc, acc_dr)
acc_all %>%
select(.model, RMSE, MAE, MAPE, ACF1) %>%
arrange(RMSE)# A tibble: 4 × 5
.model RMSE MAE MAPE ACF1
<chr> <dbl> <dbl> <dbl> <dbl>
1 DR_BOTH 2.28 1.50 26.7 0.781
2 ETS 2.42 1.71 32.0 0.790
3 DR_INDPRO 2.63 1.38 19.8 0.819
4 ARIMA 2.67 1.46 21.8 0.817fc_all <- bind_rows(fc, fc_dr)
fc_all %>%
autoplot(train_xreg, level = NULL) +
autolayer(test_xreg, value, colour = "black", linetype = "dashed") +
labs(
title = "ETS, ARIMA, and Dynamic Regression Forecasts on UNRATE",
subtitle = "Black dashed line = actual test data",
x = "Year",
y = "Unemployment Rate"
) +
theme_minimal() +
theme(legend.position = "bottom")
For Part II, I built two dynamic regression models on the training data by augmenting the baseline ARIMA framework with external monthly regressors from FRED. The first model includes industrial production only, while the second includes both industrial production and the federal funds rate. In the fpp3 framework, these are estimated as linear regression models with ARIMA errors.
The results show that the dynamic regression models improve upon the baseline ARIMA model. In particular, the baseline ARIMA model has an RMSE of 2.672, while the dynamic regression models reduce RMSE to 2.632 for DR_INDPRO and 2.281 for DR_BOTH. This suggests that the external regressors provide useful predictive information beyond the internal time-series dynamics of unemployment alone.
Comparing models across multiple performance metrics, DR_BOTH is preferred overall because it has the lowest RMSE, which is often the main criterion in forecast evaluation since it penalizes larger forecast errors more heavily. However, DR_INDPRO has the lowest MAE and MAPE, indicating that the simpler dynamic regression model produces smaller typical forecast errors on average. Therefore, the results suggest a tradeoff: adding both regressors improves overall fit in terms of RMSE, while using only industrial production yields better average absolute performance.
These findings are consistent with the idea that an external regressor should only improve a basic ARIMA model when it contains additional predictive content. Here, industrial production clearly contributes useful information, and adding the federal funds rate further improves performance under RMSE. At the same time, the gains are not uniform across all metrics, which is reasonable given the unusual test period that includes the 2020 COVID shock. Thus, dynamic regression is preferred to the baseline ARIMA model in this application, with DR_BOTH as the overall preferred model and DR_INDPRO as the best model under MAE and MAPE.