Assignment: Homework Time Series
Austin Kulig
4/28/2021
Homework Time Series
Pick a time series of your choosing. Then analyze using techniques you have learned in this course and others. Discuss at least three models and evaluate their performance. Upload to either RPubs or GitHub.
Loading the Data
In this exercise, I analyzed US Exports to Japan from 2010 through the first 6 months of 2019. I conciously decided to exclude data from 2020 and 2021, given extremes invoked by the COVID-19 pandemic.
Export.data = read.csv("/Users/austin/Desktop/Graduate\ School\ /Boston\ College/Spring\ Semster\ 2021/Predictive\ Analytics/Homework/Homework\ 3/EXPJP.csv")
Export.data$DATE = as.Date(Export.data$DATE)
head(Export.data)## DATE EXPJP
## 1 2010-01-01 4840.293
## 2 2010-02-01 4625.432
## 3 2010-03-01 5073.197
## 4 2010-04-01 4493.851
## 5 2010-05-01 5193.090
## 6 2010-06-01 5009.662Train and Test Set
After loading data from FRED (Federal Reserve Economic Data), I split the data set into a training and testing set for validaton. The training set included export data from January 2010 through December 2018. The test set included data from the first 6-months of 2019.
Exports.ts = ts(Export.data$EXPJP, start = 2010, frequency = 12)
Exports.train = ts(Exports.ts[1:108], start = 2010, frequency = 12)
Exports.test = ts(Exports.ts[109:114], start = 2019, frequency = 12) #6 month test setnaive and snaive Models
I intially generated naive and snaive models to achieve a “baseline” performance for a predictive model. The naive model generated a MASE = 1.4740600 and a RMSE = 790.3406. In contrast, the snaive model generated a MASE = 0.6545306 and a RMSE = 380.6046. Neither model performed exceptionally, as expectated, but a baseline was calculated to compare to ARIMA and ETs models.
#naive model
naive.model = naive(Exports.train)
naive.forecast = forecast(naive.model, h = 6)
summary(naive.model)##
## Forecast method: Naive method
##
## Model Information:
## Call: naive(y = Exports.train)
##
## Residual sd: 436.3202
##
## Error measures:
## ME RMSE MAE MPE MAPE MASE
## Training set 18.03966 436.3202 358.9227 -0.001582951 6.540532 0.8010057
## ACF1
## Training set -0.5527945
##
## Forecasts:
## Point Forecast Lo 80 Hi 80 Lo 95 Hi 95
## Jan 2019 6770.537 6211.370 7329.704 5915.365 7625.709
## Feb 2019 6770.537 5979.755 7561.318 5561.141 7979.933
## Mar 2019 6770.537 5802.031 7739.042 5289.335 8251.738
## Apr 2019 6770.537 5652.203 7888.871 5060.193 8480.881
## May 2019 6770.537 5520.202 8020.872 4858.314 8682.759
## Jun 2019 6770.537 5400.863 8140.210 4675.802 8865.272
## Jul 2019 6770.537 5291.120 8249.953 4507.964 9033.109
## Aug 2019 6770.537 5188.974 8352.100 4351.745 9189.328
## Sep 2019 6770.537 5093.036 8448.037 4205.021 9336.053
## Oct 2019 6770.537 5002.296 8538.778 4066.246 9474.828naive.accuracy = accuracy(naive.forecast, Exports.test)
print(naive.accuracy)## ME RMSE MAE MPE MAPE MASE
## Training set 18.03966 436.3202 358.9227 -0.001582951 6.540532 0.8010057
## Test set -660.51173 790.3406 660.5117 -11.392477259 11.392477 1.4740600
## ACF1 Theil's U
## Training set -0.5527945 NA
## Test set -0.6842137 0.9097343#snaive model
snaive.model = snaive(Exports.train)
snaive.forecast = forecast(snaive.model, h = 6)
summary(snaive.model)##
## Forecast method: Seasonal naive method
##
## Model Information:
## Call: snaive(y = Exports.train)
##
## Residual sd: 527.953
##
## Error measures:
## ME RMSE MAE MPE MAPE MASE ACF1
## Training set 152.89 527.953 448.0901 2.259267 7.971917 1 0.4773745
##
## Forecasts:
## Point Forecast Lo 80 Hi 80 Lo 95 Hi 95
## Jan 2019 5673.693 4997.094 6350.292 4638.924 6708.462
## Feb 2019 5389.072 4712.473 6065.671 4354.303 6423.841
## Mar 2019 6494.875 5818.276 7171.474 5460.106 7529.644
## Apr 2019 5839.202 5162.603 6515.801 4804.434 6873.971
## May 2019 6128.856 5452.257 6805.455 5094.088 7163.625
## Jun 2019 6395.641 5719.042 7072.240 5360.872 7430.410
## Jul 2019 6381.861 5705.262 7058.460 5347.092 7416.630
## Aug 2019 6324.507 5647.908 7001.106 5289.738 7359.276
## Sep 2019 6452.364 5775.765 7128.963 5417.595 7487.133
## Oct 2019 6550.576 5873.977 7227.175 5515.807 7585.345
## Nov 2019 6748.105 6071.506 7424.704 5713.336 7782.874
## Dec 2019 6770.537 6093.938 7447.136 5735.768 7805.306
## Jan 2020 5673.693 4716.838 6630.549 4210.309 7137.077
## Feb 2020 5389.072 4432.217 6345.928 3925.688 6852.456
## Mar 2020 6494.875 5538.019 7451.730 5031.491 7958.259
## Apr 2020 5839.202 4882.347 6796.058 4375.818 7302.586
## May 2020 6128.856 5172.001 7085.712 4665.472 7592.241
## Jun 2020 6395.641 5438.786 7352.497 4932.257 7859.025
## Jul 2020 6381.861 5425.005 7338.716 4918.477 7845.245
## Aug 2020 6324.507 5367.652 7281.362 4861.123 7787.891
## Sep 2020 6452.364 5495.509 7409.220 4988.980 7915.748
## Oct 2020 6550.576 5593.720 7507.431 5087.192 8013.960
## Nov 2020 6748.105 5791.250 7704.961 5284.721 8211.489
## Dec 2020 6770.537 5813.681 7727.392 5307.153 8233.921snaive.accuracy = accuracy(snaive.forecast, Exports.test)
print(snaive.accuracy)## ME RMSE MAE MPE MAPE MASE ACF1
## Training set 152.890 527.9530 448.0901 2.259267 7.971917 1.0000000 0.4773745
## Test set 123.135 380.6046 293.2887 1.806279 4.644855 0.6545306 -0.3103664
## Theil's U
## Training set NA
## Test set 0.3139288#Graph the results
autoplot(Exports.ts) +
autolayer(naive.forecast, series = "naive Forecast") +
autolayer(snaive.forecast, series = "snaive Forecast")
autolayer(Exports.test, series = "Actual Exports")## mapping: group = ~series, colour = ~series, x = ~timeVal, y = ~seriesVal
## geom_line: na.rm = FALSE, orientation = NA
## stat_identity: na.rm = FALSE
## position_identityARIMA Models
In the subsequent model iteration, I generated an ARIMA model. The model returned a MASE = 1.2217134 and a RMSE = 683.5534. The ARIMA model did not perform better than the snaive model. A BoxCox transformation was applied to the time series in an attempt to improve performance.
#ARIMA
ARIMA.model = auto.arima(Exports.train)
ARIMA.forecast = forecast(ARIMA.model, h = 6)
summary(ARIMA.model)## Series: Exports.train
## ARIMA(0,1,2)(0,0,2)[12]
##
## Coefficients:
## ma1 ma2 sma1 sma2
## -0.8257 0.2103 0.1894 0.3474
## s.e. 0.1052 0.1042 0.0951 0.1163
##
## sigma^2 estimated as 97902: log likelihood=-766.62
## AIC=1543.23 AICc=1543.83 BIC=1556.6
##
## Training set error measures:
## ME RMSE MAE MPE MAPE MASE
## Training set 34.13779 305.5643 243.8053 0.3606312 4.421321 0.5440988
## ACF1
## Training set 0.005800664ARIMA.accuracy = accuracy(ARIMA.forecast, Exports.test)
print(ARIMA.accuracy)## ME RMSE MAE MPE MAPE MASE
## Training set 34.13779 305.5643 243.8053 0.3606312 4.421321 0.5440988
## Test set -512.45225 683.5534 547.4377 -8.9479426 9.475205 1.2217134
## ACF1 Theil's U
## Training set 0.005800664 NA
## Test set -0.672504198 0.8027465#Graph the results
autoplot(Exports.ts) +
autolayer(ARIMA.forecast, series = "ARIMA Forecast") +
autolayer(Exports.test, series = "Actual Exports")
After applying the BoxCox transformation, the ARIMA model returned a MASE = 1.2201116 and a RMSE = 682.3265. The BoxCox transformation only mildly improved the ARIMA model perfromance.
#ARIMA
ARIMA.model.BoxCox = auto.arima(Exports.train,lambda = "auto")
ARIMA.forecast.BoxCox = forecast(ARIMA.model.BoxCox, h = 6, lambda = ARIMA.model.BoxCox$lambda)
summary(ARIMA.model)## Series: Exports.train
## ARIMA(0,1,2)(0,0,2)[12]
##
## Coefficients:
## ma1 ma2 sma1 sma2
## -0.8257 0.2103 0.1894 0.3474
## s.e. 0.1052 0.1042 0.0951 0.1163
##
## sigma^2 estimated as 97902: log likelihood=-766.62
## AIC=1543.23 AICc=1543.83 BIC=1556.6
##
## Training set error measures:
## ME RMSE MAE MPE MAPE MASE
## Training set 34.13779 305.5643 243.8053 0.3606312 4.421321 0.5440988
## ACF1
## Training set 0.005800664ARIMA.accuracy.BoxCox = accuracy(ARIMA.forecast.BoxCox, Exports.test)
print(ARIMA.accuracy.BoxCox)## ME RMSE MAE MPE MAPE MASE
## Training set 38.63873 307.9119 245.7165 0.445950 4.454190 0.5483641
## Test set -502.85064 682.3265 546.7200 -8.794895 9.456045 1.2201116
## ACF1 Theil's U
## Training set -0.001176827 NA
## Test set -0.667670400 0.8019268#Graph the results
autoplot(Exports.ts) +
autolayer(ARIMA.forecast.BoxCox, series = "ARIMA Forecast") +
autolayer(Exports.test, series = "Actual Exports")
ETS Models
I implamented an ETS model in the third and final model type. An ETS(M,N,M) model retuned a MASE = 0.9486295 and a RMSE = 478.1777. The ETS model performs better then the ARIMA model, but still performs worse than the snaive model. A BoxCox transformation was applied in an attempt to improve model performance.
#ETS
ETS.model = ets(Exports.train)
ETS.forecast = forecast(ETS.model, h = 6)
summary(ETS.model)## ETS(M,N,M)
##
## Call:
## ets(y = Exports.train)
##
## Smoothing parameters:
## alpha = 0.379
## gamma = 1e-04
##
## Initial states:
## l = 5432.9648
## s = 1.0219 1.0029 1.0348 1.0087 1.0305 0.9919
## 1.0067 1.0068 0.9719 1.0496 0.9357 0.9385
##
## sigma: 0.0543
##
## AIC AICc BIC
## 1751.130 1756.347 1791.362
##
## Training set error measures:
## ME RMSE MAE MPE MAPE MASE ACF1
## Training set 26.54745 277.005 227.7026 0.239579 4.145831 0.5081625 -0.08014135ETS.accuracy = accuracy(ETS.forecast, Exports.test)
print(ETS.accuracy)## ME RMSE MAE MPE MAPE MASE
## Training set 26.54745 277.0050 227.7026 0.239579 4.145831 0.5081625
## Test set -315.49010 478.1777 425.0715 -5.532836 7.235859 0.9486295
## ACF1 Theil's U
## Training set -0.08014135 NA
## Test set -0.41243667 0.5575208#Graph the results
autoplot(Exports.ts) +
autolayer(ETS.forecast, series = "ETS Forecast") +
autolayer(Exports.test, series = "Actual Exports")
After the BoxCox transformation was applied to an ETS(A,N,A) model, the model returned a MASE = 1.0273572 and a RMSE = 540.3687. The BoxCox transformation did not help the model performance.
#ETS with BoxCox Transformation
ETS.model.BoxCox = ets(Exports.train, lambda = "auto")
ETS.forecast.BoxCox = forecast(ETS.model.BoxCox, h = 6, lambda = ETS.model.BoxCox$lambda)
summary(ETS.model.BoxCox)## ETS(A,N,A)
##
## Call:
## ets(y = Exports.train, lambda = "auto")
##
## Box-Cox transformation: lambda= -0.0712
##
## Smoothing parameters:
## alpha = 0.4234
## gamma = 1e-04
##
## Initial states:
## l = 6.3817
## s = 0.0099 0.002 0.0157 -0.0031 0.0089 -0.0043
## 0.0074 0.001 -0.0156 0.0305 -0.0274 -0.0251
##
## sigma: 0.0287
##
## AIC AICc BIC
## -245.9376 -240.7202 -205.7056
##
## Training set error measures:
## ME RMSE MAE MPE MAPE MASE ACF1
## Training set 39.67913 272.9766 219.7162 0.4970804 3.97306 0.4903393 -0.1786186ETS.accuracy = accuracy(ETS.forecast.BoxCox, Exports.test)
print(ETS.accuracy)## ME RMSE MAE MPE MAPE MASE
## Training set 39.67913 272.9766 219.7162 0.4970804 3.973060 0.4903393
## Test set -401.26350 540.3687 460.3486 -6.9571646 7.871234 1.0273572
## ACF1 Theil's U
## Training set -0.1786186 NA
## Test set -0.4147222 0.6462742#Graph the results
autoplot(Exports.ts) +
autolayer(ETS.forecast.BoxCox, series = "ETS with BoxCox Forecast") +
autolayer(Exports.test, series = "Actual Exports")
Conclusion
Ultimately, the snaive model returned the best results (MASE = 0.6545306 and RMSE = 380.6046), when compared to the test data.