Time series forecasting - with deep learning

Sigrid Keydana, Trivadis GmbH
2017-05-23

 

 

Time series forecasting: the classical approach

A time series

 

cola_df <- read_csv("monthly-sales-of-tasty-cola.csv", col_names = c("month", "sales"), skip = 1, 
                    col_types = cols(month = col_date("%y-%m")))
ggplot(cola_df, aes(x = month, y = sales)) + geom_line() + ggtitle("Monthly sales of Tasty Cola")

plot of chunk unnamed-chunk-2

Another one ...

 

traffic_df <- read_csv("internet-traffic-data-in-bits-fr.csv", col_names = c("hour", "bits"), skip = 1)
ggplot(traffic_df, aes(x = hour, y = bits)) + geom_line() + ggtitle("Internet traffic")

plot of chunk unnamed-chunk-3

And another.

 

win_df <- read_csv("winning-times-for-the-mens-400-m.csv", col_names = c("year", "seconds"), skip = 1)
ggplot(win_df, aes(x = year, y = seconds)) + geom_line() + ggtitle("Men's 400m winning times")

plot of chunk unnamed-chunk-4

Sometimes we may look at several time series together.

 

deaths_df <- read_csv("deaths-from-homicides-and-suicid.csv", col_names = c("year", "homicide", "suicide"), skip = 1)
deaths_df <- gather(deaths_df, key = 'type', value = 'deaths', homicide:suicide)
ggplot(deaths_df, aes(x = year, y = deaths, color = type)) + geom_line() + scale_colour_manual(values=c("green","blue")) + ggtitle("Australia: Homicides and suicides")

plot of chunk unnamed-chunk-5

Sometimes there's more to the picture than we might think.

 

So far, this is nothing but a univariate time series of lynx population.

data("lynx")
autoplot(lynx) + ggtitle("Lynx population over time")

plot of chunk unnamed-chunk-6

However ...

 

Source: Rudolfo’s Usenet Animal Pictures Gallery (link no longer exists)

Lynx and hare

lynx_df <- read_delim("lynxhare.csv", delim = ";") %>% select(year, hare, lynx) %>%   filter(between(year, 1890, 1945)) %>% mutate(hare = scale(hare), lynx = scale(lynx))
lynx_df <- gather(lynx_df, key = 'species', value = 'number', hare:lynx)
ggplot(lynx_df, aes(x = year, y = number, color = species)) + geom_line() + scale_colour_manual(values=c("green","red")) + ggtitle("Lynx and hare populations over time")

plot of chunk unnamed-chunk-7

Concepts in classical time series modeling

 

  • Stationarity
  • Decomposition
  • Autocorrelation

Wait. This will be about deep learning

 

… why would the classical approach even matter?

Stationarity (1)

 

  • We want to forecast future values of a time series
  • We need fundamental statistical properties like mean, variance …
  • What is the mean, or the variance, of a time series?

Stationarity (2)

 

  • If a time series \( y_{t} \) is stationary, then for all \( s \), the distribution of (\( y_{t} \),…, \( y_{t+s} \)) does not depend on \( t \).
  • By ergodicity, after we remove any trend and seasonality effects, we may assume that the residual series is stationary in the mean: \( \mu(t)=t \)

Differencing

  • The trend is usually removed using differencing (forming the differences of neighboring values)
set.seed(7777)
trend <- 1:100 + rnorm(100, sd = 5)
diff <- diff(trend)
df <- data_frame(time_id = 1:100,
                 trend = trend,
                 diff = c(NA, diff))
df <- df %>% gather(key = 'trend_diff', value = 'value', -time_id)
ggplot(df, aes(x = time_id, y = value, color = trend_diff)) + geom_line()

plot of chunk unnamed-chunk-8

Time series decomposition

 

autoplot(stl(ts(cola_df$sales, frequency=12), s.window = 12))

plot of chunk unnamed-chunk-9

Autocorrelation - Case 1: White noise

If consecutive values were not related, there'd be no way of forecasting future values

s1 <- ts(rnorm(100))
ts1 <- autoplot(s1)
acf1 <- ggfortify:::autoplot.acf(acf(s1, plot = FALSE), conf.int.fill = '#00cccc', conf.int.value = 0.95)
do.call('grid.arrange', list('grobs' = list(ts1, acf1), 'ncol' = 2, top = "White noise"))

plot of chunk unnamed-chunk-10

Autocorrelation - Case 2: Linear trend

 

s2 <- ts(1:100 + rnorm(100, 2, 4))
ts2 <- autoplot(s2)
acf2 <- ggfortify:::autoplot.acf(acf(s2, plot = FALSE), conf.int.fill = '#00cccc', conf.int.value = 0.95)
do.call('grid.arrange', list('grobs' = list(ts2, acf2), 'ncol' = 2, top = "Series with a trend"))

plot of chunk unnamed-chunk-11

Autocorrelation - Case 3: Seasonality

 

s3 <- ts(rep(1:5,20) + rnorm(100, sd= 0.5))
ts3 <- autoplot(s3)
acf3 <- ggfortify:::autoplot.acf(acf(s3, plot = FALSE), conf.int.fill = '#00cccc', conf.int.value = 0.95)
do.call('grid.arrange', list('grobs' = list(ts3, acf3), 'ncol' = 2, top = "Series with seasonality"))

plot of chunk unnamed-chunk-12

 

 

Forecasting internet traffic, the classical way

Here's the traffic time series again.

 

ggplot(traffic_df, aes(x = hour, y = bits)) + geom_line() + ggtitle("Internet traffic")