Module 10 Discussion

Pre

Loading data

library(fpp3)
library(dplyr)
library(tidyr)
library(ggplot2)

Part I:

Comparison between HTS and GTS

Both HTS and GTS have time series data sets that should be consistent with each other when they are aggregated together, although HTS and GTS have different structures of aggregation. The structure of HTS is represented by a tree structure where there is only one tree. Each lower-level time series can only belong to one upper-level time series as its parent. For example, store sales are aggregated up to regional level, then state level, then national level. The structure of GTS can be described as having more than one classification system for each observation. There is no tree structure for HTS; instead, there are several cross-tabulated dimensions. For instance, data about tourists may include information about tourists in a particular state and the purpose of their visit. Thus, one observation can contribute to more than one intermediate sum. In conclusion, HTS is simply a pure parent-child tree structure. GTS, on the other hand, has multiple classifications for each observation.

Forecasting vs. reconciliation

The forecasting model is the type of model that will be used to create the forecasts for the time series data. This includes models such as ARIMA, ETS, and others. The reconciliation process is the technique that is used once the forecasts have been created. Since forecasts created separately in various levels in the hierarchy might not sum up properly, the reconciliation process helps in making sure that all the forecasts become coherent. Coherent forecasts are those forecasts that respect the rules of aggregation within the hierarchy. Hence, the forecasting model answers: “How can I create the forecasts?” The reconciliation model answers: “How can I ensure all forecasts sum up?”

Methods of reconciliation: Advantages and disadvantages

Top-Down

The top-down approach first involves making forecasts for the aggregate time series at the highest level. Subsequently, these forecasts are allocated to the lower levels based on past ratios.

Advantages - Easy and quick to implement - Efficient for time series that are fairly stable - Useful if bottom-level series are erratic

Disadvantages - Poor performance at the lowest levels - Takes no account of any unique characteristics of each bottom series - Inaccurate at lower levels

Bottom-Up Approach

The bottom-up approach is applied to fit models exclusively to the lowest level of detail, and the forecasts are then aggregated to form forecasts for higher levels.

Advantages - Utilizes the finest level of detail available - Highly intuitive - Automatically coherent by definition - Generally works well if the lowest level series are accurate

Disadvantages - Can become noisy if the lowest level series are highly variable - Forecast aggregations can carry over low-level noise - More difficult to handle with many low-level series

Middle-Out

Middle-out forecasting starts from a middle-level point. The forecasts can be aggregated up and disaggregated down from the middle level.

Advantages - A balance between top-down and bottom-up approaches
- Helpful in situations where the middle level provides the most stable and relevant data for analysis
- Can filter out noisy information on the bottom level while being more detailed than top-down forecasting

Disadvantages - Needs the selection of a suitable middle level - Statistically less effective than MinT - Complicated compared to top-down or bottom-up forecasting

MinT

MinT (minimum trace) matches forecasts on each level and then corrects them based on their covariance structure of forecast errors.

Advantages - Often provides the highest precision - Utilizes the information available at every level - Provides coherent forecasts in a statistical manner

Disadvantages - Takes more computational power - Difficult to understand - Relies on a good understanding of forecast error relationships

In conclusion, top-down is the simplest approach, bottom-up considers more details, middle-out is a combination of both, and MinT is usually the best one to use.

Part II:

In this case study, I utilized a hierarchical time series from the tourism dataset available in fpp3. An ETS model was applied, and reconciliation methods included top-down, middle-out, and MinT. Accuracy was then assessed based on a test set.

Create hierarchy

library(fpp3)
library(dplyr)
library(ggplot2)

data("tourism")

# Build a TRUE hierarchy: Total -> State -> Region
# Filter to one purpose so the structure stays hierarchical, not grouped
tourism_hier <- tourism |>
  filter(Purpose == "Holiday") |>
  select(Quarter, State, Region, Trips) |>
  as_tsibble(index = Quarter, key = c(State, Region)) |>
  aggregate_key(State / Region, Trips = sum(Trips))

tourism_hier

# A tsibble: 6,800 x 4 [1Q]
# Key:       State, Region [85]
   Quarter State        Region        Trips
     <qtr> <chr*>       <chr*>        <dbl>
 1 1998 Q1 <aggregated> <aggregated> 11806.
 2 1998 Q2 <aggregated> <aggregated>  9276.
 3 1998 Q3 <aggregated> <aggregated>  8642.
 4 1998 Q4 <aggregated> <aggregated>  9300.
 5 1999 Q1 <aggregated> <aggregated> 11172.
 6 1999 Q2 <aggregated> <aggregated>  9608.
 7 1999 Q3 <aggregated> <aggregated>  8914.
 8 1999 Q4 <aggregated> <aggregated>  9026.
 9 2000 Q1 <aggregated> <aggregated> 11071.
10 2000 Q2 <aggregated> <aggregated>  9196.
# ℹ 6,790 more rows

Train Test split

n_total <- tourism_hier |>
  distinct(Quarter) |>
  nrow()

n_test <- 8   # last 8 quarters for testing

split_date <- tourism_hier |>
  distinct(Quarter) |>
  slice(n_total - n_test) |>
  pull(Quarter)

train <- tourism_hier |> filter(Quarter <= split_date)
test  <- tourism_hier |> filter(Quarter > split_date)

train

# A tsibble: 6,120 x 4 [1Q]
# Key:       State, Region [85]
   Quarter State        Region        Trips
     <qtr> <chr*>       <chr*>        <dbl>
 1 1998 Q1 <aggregated> <aggregated> 11806.
 2 1998 Q2 <aggregated> <aggregated>  9276.
 3 1998 Q3 <aggregated> <aggregated>  8642.
 4 1998 Q4 <aggregated> <aggregated>  9300.
 5 1999 Q1 <aggregated> <aggregated> 11172.
 6 1999 Q2 <aggregated> <aggregated>  9608.
 7 1999 Q3 <aggregated> <aggregated>  8914.
 8 1999 Q4 <aggregated> <aggregated>  9026.
 9 2000 Q1 <aggregated> <aggregated> 11071.
10 2000 Q2 <aggregated> <aggregated>  9196.
# ℹ 6,110 more rows

test

# A tsibble: 680 x 4 [1Q]
# Key:       State, Region [85]
   Quarter State           Region        Trips
     <qtr> <chr*>          <chr*>        <dbl>
 1 2016 Q1 <aggregated>    <aggregated> 12593.
 2 2016 Q2 <aggregated>    <aggregated>  9980.
 3 2016 Q3 <aggregated>    <aggregated>  9940.
 4 2016 Q4 <aggregated>    <aggregated> 10085.
 5 2017 Q1 <aggregated>    <aggregated> 12406.
 6 2017 Q2 <aggregated>    <aggregated> 10471.
 7 2017 Q3 <aggregated>    <aggregated> 10499.
 8 2017 Q4 <aggregated>    <aggregated> 11211.
 9 2016 Q1 ACT             <aggregated>   163.
10 2016 Q1 New South Wales <aggregated>  3898.
# ℹ 670 more rows

Plot Training:

train |>
  filter(is_aggregated(State), is_aggregated(Region)) |>
  ggplot(aes(x = Quarter, y = Trips)) +
  geom_line(color = "steelblue", linewidth = 0.8) +
  labs(
    title = "Total Holiday Tourism Training Data",
    x = "Quarter",
    y = "Trips"
  ) +
  theme_minimal()

ETS Fit

fit_base <- train |>
  model(ets = ETS(Trips))

report(fit_base)

# A tibble: 85 × 11
   State         Region     .model  sigma2 log_lik   AIC  AICc   BIC   MSE  AMSE
   <chr*>        <chr*>     <chr>    <dbl>   <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
 1 ACT         … Canberra … ets    6.28e-2   -408.  831.  833.  847. 1398. 1422.
 2 ACT         … <aggregat… ets    6.28e-2   -408.  831.  833.  847. 1398. 1422.
 3 New South Wa… Blue Moun… ets    4.21e-2   -370.  754.  756.  770.  454.  532.
 4 New South Wa… Capital C… ets    4.10e+2   -370.  745.  745.  752.  399.  416.
 5 New South Wa… Central C… ets    2.58e-2   -382.  779.  781.  795.  604.  596.
 6 New South Wa… Central N… ets    2.39e-2   -390.  793.  795.  809.  771.  774.
 7 New South Wa… Hunter   … ets    1.50e-2   -413.  841.  842.  857. 1359. 1302.
 8 New South Wa… New Engla… ets    3.44e-2   -377.  761.  761.  768.  505.  507.
 9 New South Wa… North Coa… ets    3.52e+3   -445.  904.  905.  920. 3225. 3289.
10 New South Wa… Outback N… ets    7.01e-2   -328.  670.  672.  686.  157.  157.
# ℹ 75 more rows
# ℹ 1 more variable: MAE <dbl>

Fitted values on training

train_fitted <- augment(fit_base)

Total Level

train_fitted |>
  filter(is_aggregated(Region), is_aggregated(State)) |>
  ggplot(aes(x = Quarter)) +
  geom_line(aes(y = Trips, colour = "Actual")) +
  geom_line(aes(y = .fitted, colour = "Fitted"), linetype = "dashed") +
  labs(
    title = "Top-Level Training Fit: Actual vs Fitted",
    y = "Trips",
    colour = ""
  )

State level

train_fitted |>
  filter(!is_aggregated(State), is_aggregated(Region)) |>
  ggplot(aes(x = Quarter)) +
  geom_line(aes(y = Trips, colour = "Actual")) +
  geom_line(aes(y = .fitted, colour = "Fitted"), linetype = "dashed") +
  facet_wrap(vars(State), scales = "free_y") +
  labs(
    title = "State-Level Training Fit: Actual vs Fitted",
    y = "Trips",
    colour = ""
  )

Region level

train_fitted |>
  filter(!is_aggregated(State), is_aggregated(Region)) |>
  ggplot(aes(x = Quarter)) +
  geom_line(aes(y = Trips, colour = "Actual"), linewidth = 0.6) +
  geom_line(aes(y = .fitted, colour = "Fitted"), linetype = "dashed", linewidth = 0.6) +
  facet_wrap(vars(State), scales = "free_y", ncol = 3) +
  labs(
    title = "State-Level Training Fit: Actual vs Fitted",
    x = "Quarter",
    y = "Trips",
    colour = ""
  ) +
  theme_minimal() +
  theme(
    plot.title = element_text(face = "bold"),
    legend.position = "bottom"
  )

Reconcile Forecasts

fc_reconciled <- fit_base |>
  reconcile(
    top_down   = top_down(ets),
    middle_out = middle_out(ets, split = "State"),
    mint       = min_trace(ets, method = "mint_shrink")
  ) |>
  forecast(h = 8)

fc_reconciled

# A fable: 2,720 x 6 [1Q]
# Key:     State, Region, .model [340]
   State  Region   .model   Quarter
   <chr*> <chr*>   <chr>      <qtr>
 1 ACT    Canberra ets      2016 Q1
 2 ACT    Canberra ets      2016 Q2
 3 ACT    Canberra ets      2016 Q3
 4 ACT    Canberra ets      2016 Q4
 5 ACT    Canberra ets      2017 Q1
 6 ACT    Canberra ets      2017 Q2
 7 ACT    Canberra ets      2017 Q3
 8 ACT    Canberra ets      2017 Q4
 9 ACT    Canberra top_down 2016 Q1
10 ACT    Canberra top_down 2016 Q2
# ℹ 2,710 more rows
# ℹ 2 more variables: Trips <dist>, .mean <dbl>

Total Series Data

train_total <- train |>
  filter(is_aggregated(State), is_aggregated(Region)) |>
  as_tibble()

test_total <- test |>
  filter(is_aggregated(State), is_aggregated(Region)) |>
  as_tibble()

fc_total <- fc_reconciled |>
  filter(is_aggregated(State), is_aggregated(Region)) |>
  as_tibble() |>
  select(Quarter, .model, .mean)

Reconcile Forecast vs Test Data

ggplot() +
  geom_line(data = train_total, aes(x = Quarter, y = Trips, colour = "Train")) +
  geom_line(data = test_total, aes(x = Quarter, y = Trips, colour = "Test")) +
  geom_line(data = fc_total, aes(x = Quarter, y = .mean, colour = "Forecast")) +
  facet_wrap(vars(.model), scales = "free_y") +
  labs(
    title = "Reconciled Forecasts vs Test Data (Total Level)",
    x = "Quarter",
    y = "Trips",
    colour = ""
  )

Forecast Plots:

Top-Down

fc_total |>
  filter(.model == "top_down") |>
  ggplot() +
  geom_line(
    data = train_total,
    aes(x = Quarter, y = Trips, colour = "Train")
  ) +
  geom_line(
    data = test_total,
    aes(x = Quarter, y = Trips, colour = "Test")
  ) +
  geom_line(
    aes(x = Quarter, y = .mean, colour = "Forecast")
  ) +
  labs(
    title = "Top-Down Forecast vs Test Data",
    x = "Quarter",
    y = "Trips",
    colour = ""
  )

Middle-out

fc_total |>
  filter(.model == "middle_out") |>
  ggplot() +
  geom_line(
    data = train_total,
    aes(x = Quarter, y = Trips, colour = "Train")
  ) +
  geom_line(
    data = test_total,
    aes(x = Quarter, y = Trips, colour = "Test")
  ) +
  geom_line(
    aes(x = Quarter, y = .mean, colour = "Forecast")
  ) +
  labs(
    title = "Middle-Out Forecast vs Test Data",
    x = "Quarter",
    y = "Trips",
    colour = ""
  )

MinT

fc_total |>
  filter(.model == "mint") |>
  ggplot() +
  geom_line(
    data = train_total,
    aes(x = Quarter, y = Trips, colour = "Train")
  ) +
  geom_line(
    data = test_total,
    aes(x = Quarter, y = Trips, colour = "Test")
  ) +
  geom_line(
    aes(x = Quarter, y = .mean, colour = "Forecast")
  ) +
  labs(
    title = "MinT Forecast vs Test Data",
    x = "Quarter",
    y = "Trips",
    colour = ""
  )

Accuracy Results

accuracy_results <- fc_reconciled |>
  accuracy(test) |>
  arrange(RMSE)

accuracy_results

# A tibble: 340 × 12
   .model     State       Region     .type     ME  RMSE   MAE    MPE  MAPE  MASE
   <chr>      <chr*>      <chr*>     <chr>  <dbl> <dbl> <dbl>  <dbl> <dbl> <dbl>
 1 middle_out South Aust… Eyre Peni… Test  -0.719  3.40  2.79  -3.24  8.45   NaN
 2 ets        South Aust… Eyre Peni… Test  -0.860  3.44  2.81  -3.63  8.59   NaN
 3 mint       South Aust… Eyre Peni… Test  -0.968  3.46  2.80  -3.96  8.56   NaN
 4 mint       Northern T… Barkly   … Test   1.16   3.47  3.18  -5.82 46.6    NaN
 5 ets        Northern T… Barkly   … Test   1.16   3.48  3.18  -5.54 46.7    NaN
 6 top_down   Northern T… Barkly   … Test   1.17   3.48  3.19  -5.51 47.0    NaN
 7 middle_out Northern T… Barkly   … Test   1.37   3.53  3.25  -3.16 46.9    NaN
 8 top_down   South Aust… Eyre Peni… Test  -1.51   3.67  3.02  -5.47  9.27   NaN
 9 ets        South Aust… Adelaide … Test  -2.32   4.95  4.24 -20.8  28.8    NaN
10 mint       South Aust… Adelaide … Test  -2.30   4.96  4.25 -20.7  28.8    NaN
# ℹ 330 more rows
# ℹ 2 more variables: RMSSE <dbl>, ACF1 <dbl>

Best Model

best_model <- accuracy_results |>
  slice(1) |>
  select(.model, RMSE, MAE, MAPE)

best_model

# A tibble: 1 × 4
  .model      RMSE   MAE  MAPE
  <chr>      <dbl> <dbl> <dbl>
1 middle_out  3.40  2.79  8.45

Conclusion

From the given accuracy table, a comparison has been made between the top-down, middle-out, and MinT forecasting methods based on the test set. The most accurate model is determined by which has the smallest value of RMSE. Typically, the top-down approach is straightforward but might miss finer details, while the middle-out approach takes into account both top and bottom details. The MinT method usually performs better since it considers all levels and forecast error correlations. In this particular case, the middle-out forecasting model performs the best.