Neural Network Discussion

In this week, we will be discussing Neural Networks, or neuralnets. These are based on simple mathematical models of the brain and allow for xomplex nonlinear relationships between the response and predictor variables. One can think of a neuranet as a network of “neurons”, which are organized in layers. Each input has attached coefficients, or “weights”. Forecasts are obtained by a linear combination of inputs, and weights are selected in the neuralnet framework using a “learning algorithm” minimizing a cost function. For our purposes, inputs and the regression coefficients are lagged values of the time series. This time, we will be observing Consumer Price Index (CPI) information for the time period of 12/2009-12/2019

CPIAUCSL_1_ <- read_excel("C:/Users/abact/Downloads/CPIAUCSL (1).xls")
View(CPIAUCSL_1_)
cpi <- ts(CPIAUCSL_1_$CPIAUCSL, start=c(2009, 12), frequency = 12)
require(forecast)
require(fpp2)
require(ggplot2)
autoplot(cpi) + xlab("Time, Year")

summary(cpi)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  217.2   228.8   237.0   236.9   244.2   258.4 

We see that there seems to be strong upward trend, and should not be too affected by seasonality. Returning to neuralnets, they consist of an initial input layer of weighted regression values. One can use variables they believe are correlated to the response variable for this layer. Again, for our purposes, we will be using lagged observations. Weights are automatically determined for these inuts by a learning algortihm. There is then, a “hidden layer” that these inputs affect, should the model require it. Should the hidden layer be required, it would then be classified as a “multi-layer feed-forward network”, where each node layer receives inputs from the previous layers. This is basically a series of layer inputs to further layer inputs. Again, each node is combined with a weight.

The hidden layer is calculated as the summation of all weighted input nodes wi,j xi where wj is the weight for input 1 applied to node j. As such, a model with no hidden layer is structurally akin to a moving average model. We can also consider seasonal lags where p=12 so that seasonality can be considered.

Using neural network autoregression, we can automate the number of lagged inputs, the seasonal lags considered, and the number of nodes in the hidden layer. This results in the form NNAR(p,P,k)m.

cpi.nnar <- nnetar(cpi, lambda=0)
nnar.pred <- forecast(cpi.nnar, h=12)
summary(nnar.pred)

Forecast method: NNAR(1,1,2)[12]

Model Information:

Average of 20 networks, each of which is
a 2-2-1 network with 9 weights
options were - linear output units 

Error measures:
                      ME      RMSE       MAE           MPE      MAPE       MASE      ACF1
Training set 0.000414096 0.4239639 0.3292742 -0.0001493837 0.1383185 0.07868945 0.3083911

Forecasts:

autoplot(nnar.pred, PI=TRUE)

Can’t seem to get prediction intervals to be working, but i believe that this model shows a somewhat accurate one-year forecast.

We see that we get the automated model NNAR(1,1,2)12. This results in a single lag, single seasonal lag, and two hidden layers. It has an extremely small RMSE and ME. Of course, this model does not take into account the negative shock that the coronavirus had, especially given the mass layoffs that occurred.

We may want to attempt a model that incorporates the unemployment rate. This would follow the theory laid down by the Phillips Curve. If this model can accurately predict the recent turbulence in wages more than the neuralnet model can, then that would be interesting.

cpi <- read.csv("C:/Users/abact/Downloads/CPIAUCSL (3).csv")
cpi <- ts(cpi$CPIAUCSL, start=c(2009,12), frequency=12)
unem <- read.csv("C:/Users/abact/Downloads/UNRATE (4).csv")
unem <- ts(unem$UNRATE, start=c(2009,12), frequency=12)

phillip.nnar <- nnetar(cpi, lambda=0, xreg=unem)
phillip.pred <- forecast(phillip.nnar,xreg=unem, h=12)
summary(phillip.pred)

Forecast method: NNAR(1,1,2)[12]

Model Information:

Average of 20 networks, each of which is
a 3-2-1 network with 11 weights
options were - linear output units 

Error measures:
                       ME      RMSE     MAE           MPE      MAPE       MASE      ACF1
Training set 0.0002767027 0.4400133 0.34096 -0.0002382096 0.1421984 0.08272387 0.3175262

Forecasts:

autoplot(phillip.pred, PI=TRUE)

We do come across a difficult question. We want to forecast the accuracy of our model with accurate, up-to-date input variables, but can’t calculate the accuracy between observed and forecast obsesrvations. That being said, the model does an admirable job calculating the most recent values, including the negative shock. Redoing the model with up-to-date observations…

cpi.nnar <- nnetar(cpi, lambda=0)
nnar.pred <- forecast(cpi.nnar, h=12)
summary(nnar.pred)

Forecast method: NNAR(1,1,2)[12]

Model Information:

Average of 20 networks, each of which is
a 2-2-1 network with 9 weights
options were - linear output units 

Error measures:
                      ME      RMSE       MAE           MPE      MAPE       MASE      ACF1
Training set 0.000414641 0.4936522 0.3679964 -0.0002474709 0.1527091 0.08928346 0.3648081

Forecasts:

autoplot(nnar.pred, PI=TRUE)

I would say that the Phillips Curve model does better in forecasting random shocks.

Anyway, that was fun. The Phillips Curve does well, but not so much better that it’s not easier to just use lagged variables.