About me

Background

Research question: How to model and forecast Exchange rate of EUR/USD using MLPs neural network?

Time series analysis and dynamic modeling are important to study topics with a variety of applications in business, economics, finance, and computer science. The purpose of time series analysis is to examine the projected path of time series observations, build a model to characterize the data structure, and then predict the time series’ future values. In finance, exchange rate forecasting is a challenging application of current time series forecasting that is critical to the performance of many businesses and financial organizations. The rates are nonlinear, chaotic, noisy, and non-stationary. On finance-related time series, traditional time series analysis, such as ARIMA models, suffers from low. Because they show a heteroscdasticity behavior. As a result, using Neural Networks (NN) to estimate exchange rates has been proposed as a real alternative. In this study, we will utilize an MLP-NN to predict the next step-ahead exchange rate of EUR/USD. From February 2015 to September 2016, daily data was collected (400 observations). The first 300 must be utilized as training data, while the remaining ones must be used as a testing set.

Important librarys

The packages being used in this study series are here in listed:

## Load Packages
library(zoo,warn.conflicts=FALSE)
library(lubridate,warn.conflicts=FALSE)
library(mgcv,warn.conflicts=FALSE)
library(rugarch,warn.conflicts=FALSE)
# visualization
suppressPackageStartupMessages(library(ggplot2)) 
# getting financial data
suppressPackageStartupMessages(library(quantmod)) 
# calculating returns
suppressPackageStartupMessages(library(PerformanceAnalytics)) 
# GARCH modeling
suppressPackageStartupMessages(library(rugarch)) 
# ARCH test
suppressPackageStartupMessages(library(FinTS)) 
# ARMA modeling
suppressPackageStartupMessages(library(forecast)) 
# structural changes
suppressPackageStartupMessages(library(strucchange)) 
# ARMA order identification
suppressPackageStartupMessages(library(TSA)) 
library(tseries)
library(timeSeries)
library(xts)
library(pastecs)
library(tidyr)
library(dplyr)
library(splines)
library(tidyverse)
rm(list=ls()) 
library(FinTS)
library(rugarch)
library(tseries)
library(dynlm) 
library(vars)
library(nlWaldTest) 
library(broom) 
library(readxl)
# For Multilayer Perceptrons (MLP)
library(nnfor)
library(neuralnet)

Importing and cleaning data

#importing the data
exchangeEUR <- read.csv("exchangeEUR20152016.csv", head = TRUE)

#cleaning the data
#dates to date format
exchangeEUR$YYYY.MM.DD <- as.Date(exchangeEUR$YYYY.MM.DD,format='%m/%d/%Y')
#exchange rate to timeseries format

#exchangerate <- ts(exchangeEUR$USD.EUR,start=c(2015,2,2),freq=252)
dim(exchangeEUR)
## [1] 400   3
head(exchangeEUR)
##   YYYY.MM.DD Wdy USD.EUR
## 1 2015-02-02 Mon  1.1336
## 2 2015-02-03 Tue  1.1461
## 3 2015-02-04 Wed  1.1418
## 4 2015-02-05 Thu  1.1432
## 5 2015-02-06 Fri  1.1330
## 6 2015-02-09 Mon  1.1316
tail(exchangeEUR)
##     YYYY.MM.DD Wdy USD.EUR
## 395 2016-08-29 Mon  1.1181
## 396 2016-08-30 Tue  1.1150
## 397 2016-08-31 Wed  1.1146
## 398 2016-09-01 Thu  1.1194
## 399 2016-09-02 Fri  1.1157
## 400 2016-09-06 Tue  1.1236

We have a total number of 400 daily observations (5 days per week) and 3 variables, below the types of each variable:

# structure of dataset
str(exchangeEUR) 
## 'data.frame':    400 obs. of  3 variables:
##  $ YYYY.MM.DD: Date, format: "2015-02-02" "2015-02-03" ...
##  $ Wdy       : Factor w/ 5 levels "Fri","Mon","Thu",..: 2 4 5 3 1 2 4 5 3 1 ...
##  $ USD.EUR   : num  1.13 1.15 1.14 1.14 1.13 ...

Exploratory time series Analysis

Descriptive statistics

We present the descriptive statistics for each variable:

summary(exchangeEUR)
##    YYYY.MM.DD          Wdy        USD.EUR     
##  Min.   :2015-02-02   Fri:78   Min.   :1.052  
##  1st Qu.:2015-06-24   Mon:74   1st Qu.:1.093  
##  Median :2015-11-18   Thu:83   Median :1.113  
##  Mean   :2015-11-18   Tue:84   Mean   :1.109  
##  3rd Qu.:2016-04-13   Wed:81   3rd Qu.:1.127  
##  Max.   :2016-09-06            Max.   :1.158

For more descriptive statistics, use stat.desc() from the package {pastecs}:

stat.desc(exchangeEUR)
##                YYYY.MM.DD Wdy      USD.EUR
## nbr.val      4.000000e+02  NA 4.000000e+02
## nbr.null     0.000000e+00  NA 0.000000e+00
## nbr.na       0.000000e+00  NA 0.000000e+00
## min          1.646800e+04  NA 1.052300e+00
## max          1.705000e+04  NA 1.157800e+00
## range        5.820000e+02  NA 1.055000e-01
## sum          6.703136e+06  NA 4.437879e+02
## median       1.675750e+04  NA 1.113300e+00
## mean         1.675784e+04  NA 1.109470e+00
## SE.mean      8.443313e+00  NA 1.113374e-03
## CI.mean.0.95 1.659894e+01  NA 2.188811e-03
## var          2.851581e+04  NA 4.958403e-04
## std.dev      1.688663e+02  NA 2.226747e-02
## coef.var     1.007685e-02  NA 2.007037e-02

Visualize the Time Series

ggplot(exchangeEUR, aes(exchangeEUR$YYYY.MM.DD,exchangeEUR$USD.EUR) ) +
  geom_point() +geom_line(colour = "blue")+xlab("Date")+ ylab("EUR")+ggtitle("
  Exchange rate daily of EUR")

The price development in efficient markets as, for example, exchange rates, can often be represented by a random walk. As we can see in the figure above fluctuation of the series over time. From the behavior of the series we can say that the series is stationary. Because we are not a seasonality or a trend patterns, We can confirm this after by the Augmented Dickey Fuller (ADF) test or by the analysis of the plots of ACF and PACF.

From the above data table, it is clear that we only need the USD.EUR column and convert the series into time series and plot the series. Figure below shows the time plot of the exchange rate.

rate <- exchangeEUR[3]

## Create a daily Date object - helps my work on dates

inds <- seq(as.Date("2015-02-02"), as.Date("2016-09-06"), by = "day")

## Create a time series object

rate.ts <- ts(rate, start = c(2015, as.numeric(format(inds[1], "%j"))),
           frequency = 365)


plot.ts(rate.ts, main = "Figure : Time Plot of Exchange Rate of EUR/USD", col = "blue")

#exchangerate <- exchangeEUR[,"USD.EUR"]
autoplot(rate.ts,main="Time series plot of exchange rate",xlab='
         Date: from Feb. 02, 2015, to Sep. 06, 2016', ylab='Exchange',col = "blue")

Stationarity and Differencing

Time series patterns

A seasonal time series has three components: a trend, a seasonal component, and an error component (or cyclic). The process of decomposing a time series is breaking it down into these three elements (Coghlan, 2018). The Structure of the time series can be summarized as follows:

  • Trend: pattern exists when there is a long-term increase or decrease in the data.

  • Seasonal: pattern exists when a series is influenced by seasonal factors (e.g., the quarter of the year, the month, or day of the week).

  • Cyclic: pattern exists when data exhibit rises and falls that are not of fixed period (duration usually of at least 2 years).

We need to perform a stationarity test to check if the data is stationary. The Augmented Dickey-Fuller unit root test was used as follows:

  • Null hypothesis \(H_0\): Unit root exists i.e. the series is not stationary.

  • Alternative hypothesis \(H_a\): Stationary.

adf.test(rate.ts, alternative = "stationary")
## 
##  Augmented Dickey-Fuller Test
## 
## data:  rate.ts
## Dickey-Fuller = -3.7108, Lag order = 7, p-value = 0.02366
## alternative hypothesis: stationary

The expectation was that the series is stationary at \(95\%\) confidence level, the ADF test statistic = -3.711, p-value = 0.024 implies that the p-vale is less than \(0.05\); thus, reject the null hypothesis of existence of unit root. Therefore, the series is stationary.

Analyzes the plots of the ACF and the PACF

autoplot(acf(rate.ts, plot = FALSE), main="
         Autocorrelogram of the exchange rate of EUR/USD series")

ggPacf((rate.ts),main="Partial autocorrelogram of the exchange rate of EUR/USD series")

The ACF of difference drops to zero relatively quickly, this proves that the series is stationary!

In the next step, we fixed a breakpoint which will be used to split the series dataset in two parts; training (\(75\%\)) and test (\(25\%\)).

A train and test set was created. The range was as follows:

Training Set Range: 02 February 2015 - 13 April 2016

Test Set Range: 14 April 2016 - 06 September 2016

Split the data into training and test

# Create training and test set
# Daily Seasonality (frequency set to 1 for daily data)
rate.ts <- ts(rate.ts,frequency=1)

# Delimit training range (75%)
rate.train <- window(rate.ts,end=c(300,1))

# Delimit testing range (25%)
rate.test <- window(rate.ts,start=c(301,1))

Modeling

Build MLPs Model

The Basics of Neural Network

A simple neural network model

A simple neural network model

The backpropgation error approach can be utilized in conjunction with the learning rules. The error at the output unit is calculated using the learning rule. This mistake is backpropagated to all units, resulting in an error proportional to the contribution of each unit to the total error at the output unit. The weight at each connection is then optimized based on the mistakes at each unit. For a better understanding, the structure of a simple neural network model is shown in the diagram below.

Time series forecasting with Multilayer Perceptrons (MLP) and Extreme Learning Machines is simplified with the nnfor package for R.

Univariate forecasting using function mlp()

The main function is mlp(), and at its simplest form you only need to input a time series to be modelled. nnfor is different from other R neural network implementations in that it includes code for automatically designing networks with respectable predicting performance while also giving the expert user in-depth control. With parsimony in mind, the automatic specification was created. This not only improves the robustness of the resulting networks, but it also cuts training time.

fit1 <- mlp(rate.train)
print(fit1)
## MLP fit with 5 hidden nodes and 20 repetitions.
## Univariate lags: (1)
## Forecast combined using the median operator.
## MSE: 1e-04.

The output indicates that the resulting network has 5 hidden nodes, it was trained 20 times and the different forecasts were combined using the median operator. The mlp() function automatically generates ensembles of networks, the training of which starts with different random initial weights.

You can get a visual summary by using the plot() function.

plot(fit1,main="MLP of Exchange rate")

The grey input nodes are autoregressions. If any other regressors were included, they would be shown in light blue or in magenta (seasonality).

The mlp() function accepts several arguments to fine-tune the resulting network. The hd argument defines a fixed number of hidden nodes. If it is a single number, then the neurons are arranged in a single hidden node. If it is a vector, then these are arranged in multiple layers.

fit2 <- mlp(rate.train, hd = c(10,5))
print(fit2)
## MLP fit with (10,5) hidden nodes and 20 repetitions.
## Univariate lags: (1)
## Forecast combined using the median operator.
## MSE: 1e-04.
plot(fit2)

The number of hidden nodes can be either preset, using the argument hd or automatically specified, as defined with the argument hd.auto.type. By default this is hd.auto.type="set" and uses the input provided in hd (default is 5). You can set this to hd.auto.type="valid" to test using a validation sample (\(25\%\) of the time series), or hd.auto.type="cv" to use 5-fold cross-validation. The number of hidden nodes to evaluate is set by the argument hd.max.

fit3 <- mlp(rate.train, hd.auto.type="valid",hd.max=8)
print(fit3)
## MLP fit with 4 hidden nodes and 20 repetitions.
## Univariate lags: (1)
## Forecast combined using the median operator.
## MSE: 1e-04.
plot(fit3)

Forecasting

To produce forecasts, we use the function forecast(), which requires a trained network object and the forecast horizon h.

frc1 <- forecast(fit1,h=100)
#print(frc1)
# Multi-Steps Forecast
plot(frc1,main="Forcasts from MLP",ylab="Exchange",xlab=
       "Date: from Feb. 02, 2015, to Sep. 06, 2016")
lines(rate.test,lty=3)

autoplot(frc1,main="Forcasts for MLP fit with 5 hidden nodes",ylab="Exchange",xlab=
           "Date: from Feb. 02, 2015, to Sep. 06, 2016")

frc2 <- forecast(fit2,h=100)
#print(frc2)
# Multi-Steps Forecast
plot(frc2,main="Forcasts from MLP",ylab="Exchange",xlab=
       "Date: from Feb. 02, 2015, to Sep. 06, 2016")
lines(rate.test,lty=3)

autoplot(frc2,main="Forcasts from MLP fit with (10,5) hidden nodes",ylab="Exchange",xlab=
           "Date: from Feb. 02, 2015, to Sep. 06, 2016")

frc3 <- forecast(fit3,h=100)
#print(frc3)
# Multi-Steps Forecast
plot(frc3,main="Forcasts from MLP",ylab="Exchange",xlab=
       "Date: from Feb. 02, 2015, to Sep. 06, 2016")
lines(rate.test,lty=3)

autoplot(frc3,main="Forcasts from MLP fit with 6 hidden nodes",ylab="Exchange",xlab=
           "Date: from Feb. 02, 2015, to Sep. 06, 2016")

Accuracy of the predictions for the Exchange rate

# Forecast  Accuracy for MLP fit with 5 hidden nodes
accuracy(frc1,rate.test)[1,1:5]
##            ME          RMSE           MAE           MPE          MAPE 
## -1.257315e-06  7.989220e-03  6.219192e-03 -9.786502e-05  5.630358e-01
# Forecast  Accuracy for MLP fit with (10,5) hidden nodes
accuracy(frc2,rate.test)[1,1:5]
##            ME          RMSE           MAE           MPE          MAPE 
## -9.946981e-06  7.974448e-03  6.225215e-03 -9.140153e-04  5.635841e-01
# Forecast  Accuracy for MLP fit with 6 hidden nodes
accuracy(frc3,rate.test)[1,1:5]
##           ME         RMSE          MAE          MPE         MAPE 
## 4.287588e-06 7.990393e-03 6.221881e-03 4.106961e-04 5.632755e-01

Prediction Comparison

Models ME RMSE MAE MAPE MAPE
MLP with 5 hidden nodes Test 1.279e-6 7.988e-3 6.218e-3 1.253e-4 5.629e-1
MLP with (10,5) hidden nodes Test 2.594e-6 7.981e-3 6.226e-3 2.299e-4 5.636e-1
MLP fit with 6 hidden nodes Test -3.846e-6 7.988e-3 6.22e-3 -3.229e-4 5.632e-1

From the table we can see the MLP fit with 6 hidden nodes is the best model compared to the other models because have small ME, RMSE, MAE, MAPE, and MAPE.

Univariate forecasting using function Neural Networks Autoregression

Finally, you can pass arguments directly to the neuralnet() function that is used to train the networks by using the nnetar.

model = nnetar(rate.train) 
print(model)
## Series: rate.train 
## Model:  NNAR(1,1) 
## Call:   nnetar(y = rate.train)
## 
## Average of 20 networks, each of which is
## a 1-1-1 network with 4 weights
## options were - linear output units 
## 
## sigma^2 estimated as 6.378e-05
forecast = forecast(model, h=100)   
plot(forecast)
lines(rate.test,lty=3)

autoplot(forecast,main="Forcasts from NNAR(1,1)",ylab="Exchange",xlab=
           "Date: from Feb. 02, 2015, to Sep. 06, 2016")

# Forecast  Accuracy from NNAR(1,1)
accuracy(forecast,rate.test)[1,1:5]
##            ME          RMSE           MAE           MPE          MAPE 
##  3.642982e-08  7.986339e-03  6.230291e-03 -5.214203e-03  5.636677e-01

Prediction Comparison

Models ME RMSE MAE MAPE MAPE
MLP fit with 6 hidden nodes Test -3.846e-6 7.988e-3 6.22e-3 -3.229e-4 5.632e-1
NNAR(1,1) Test -2.436e-8 7.986e-3 6.23e-3 -5.22e-3 5.637e-1

From the table we can see the NNAR(1,1) is the best model compared to the MLP model because have small ME, RMSE, MAE, MAPE, and MAPE.

Conclusion

In our study, we studied the exchange rate of EUR/USD time series. The purpose of this study was to model, and predict the series. The modeling consisted in finding models that will estimate them with the minimum error. We used the MLP neural network model, After modeling we predicted the next 100 values of the series. The prediction remains constant over time this tells you that the model is not capable of following test set fluctuation due to non-linearity. Generally the prediction does not contradict the structure of the series. However we are conscious that the models are not unique, and that it is possible to find other models with better estimation performances. All the difficulty of the time series is in the modeling, and with a more thorough study and with advanced technical and theoretical means, it is possible to “find” the best model for the series.

References:

  1. Forecasting Kshs to USD Exhange Rate using R studio
  2. Creating & Visualizing Neural Network in R
  3. Tutorial for the nnfor R package
  4. Neural Networks Autoregression Forecasting Model in R
  5. Package ‘timedelay’