Discussion
Juan Zabal
Part 1: Judgmental Forecasting
The real-world scenario I chose was forecasting demand for new product launches. When a company launches a new product, there is no history and no existing time series to forecast from. This is exactly the scenario where judgmental forecasting would outperform purely statistical methods, at least until enough real sales data accumulates to support a quantitative model.
Judgmental inputs such as expert opinions from regional sales managers or potential buyers can help estimate demand and provide competitor intelligence that isn’t captured in any dataset. Through the Delphi method, we could have marketing, supply chain, external analysts, and others make independent estimates and predictions, then gather an average from those responses. Through scenario analysis, we can build different cases, from best-case to worst-case demand expectations, similar to a Monte Carlo analysis, and use that to forecast and plan decisions depending on which scenario unfolds, helping us prepare for all outcomes. Lastly, through analogous forecasting, we can find similar products or similar customer behavior and use their adoption curves to help base our decisions on that.
Some limitations that might occur are that product teams who create the launch tend to overforecast demand for their own product, while a sales team incentivized on hitting a forecast may deliberately underforecast it in order to reach that quota more easily. Also, early numbers can create anchors that later contaminate other estimates.
Possible mitigations include structuring input through Delphi rounds or the Nominal Group Technique instead of open discussion, to reduce dominance and anchoring effects; requiring forecasters to give a range or a subjective probability distribution rather than a single number, which surfaces overconfidence; and using a statistical baseline, like an analogous-launch curve, as the starting point that judgmental input then adjusts, rather than letting experts forecast from a blank page.
Part 2: Time Series Regression Models
Setup
install.packages("fpp3")library(fpp3)Data
vic_elec is Victorian half-hourly electricity demand and
temperature data (2012-2014), included in the fpp3
package.
vic_elec## # A tsibble: 52,608 x 5 [30m] <Australia/Melbourne>
## Time Demand Temperature Date Holiday
## <dttm> <dbl> <dbl> <date> <lgl>
## 1 2012-01-01 00:00:00 4383. 21.4 2012-01-01 TRUE
## 2 2012-01-01 00:30:00 4263. 21.0 2012-01-01 TRUE
## 3 2012-01-01 01:00:00 4049. 20.7 2012-01-01 TRUE
## 4 2012-01-01 01:30:00 3878. 20.6 2012-01-01 TRUE
## 5 2012-01-01 02:00:00 4036. 20.4 2012-01-01 TRUE
## 6 2012-01-01 02:30:00 3866. 20.2 2012-01-01 TRUE
## 7 2012-01-01 03:00:00 3694. 20.1 2012-01-01 TRUE
## 8 2012-01-01 03:30:00 3562. 19.6 2012-01-01 TRUE
## 9 2012-01-01 04:00:00 3433. 19.1 2012-01-01 TRUE
## 10 2012-01-01 04:30:00 3359. 19.0 2012-01-01 TRUE
## # ℹ 52,598 more rowsAggregate to a daily series
daily <- vic_elec %>%
index_by(Date = as_date(Time)) %>%
summarise(
Demand = mean(Demand),
Temperature = max(Temperature),
Holiday = any(Holiday)
) %>%
mutate(
Day_Type = case_when(
Holiday ~ "Holiday",
wday(Date) %in% c(1, 7) ~ "Weekend",
TRUE ~ "Weekday"
)
)
daily## # A tsibble: 1,096 x 5 [1D]
## Date Demand Temperature Holiday Day_Type
## <date> <dbl> <dbl> <lgl> <chr>
## 1 2012-01-01 4634. 32.7 TRUE Holiday
## 2 2012-01-02 5374. 39.6 TRUE Holiday
## 3 2012-01-03 5565. 31.8 FALSE Weekday
## 4 2012-01-04 4640. 25.1 FALSE Weekday
## 5 2012-01-05 4387. 21.2 FALSE Weekday
## 6 2012-01-06 4380. 23.6 FALSE Weekday
## 7 2012-01-07 4219. 29 FALSE Weekend
## 8 2012-01-08 4029. 27.8 FALSE Weekend
## 9 2012-01-09 4454. 24 FALSE Weekday
## 10 2012-01-10 4480. 19.6 FALSE Weekday
## # ℹ 1,086 more rowsExploratory Plots
daily %>%
autoplot(Demand) +
labs(title = "Daily Electricity Demand", y = "MW", x = "Date")
daily %>%
ggplot(aes(x = Temperature, y = Demand, colour = Day_Type)) +
geom_point(alpha = 0.6) +
labs(title = "Daily Demand vs Temperature",
y = "Demand (MW)", x = "Max Temperature (°C)")
Regression Model
Trend + seasonality + nonlinear temperature effect + day type.
fit <- daily %>%
model(
full = TSLM(Demand ~ trend() + season() + Temperature + I(Temperature^2) + Day_Type)
)
report(fit)## Series: Demand
## Model: TSLM
##
## Residuals:
## Min 1Q Median 3Q Max
## -941.932 -132.298 -8.094 133.565 896.139
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 7418.68165 89.76689 82.644 <2e-16 ***
## trend() -0.22762 0.02205 -10.323 <2e-16 ***
## season()week2 -62.94002 26.08718 -2.413 0.016 *
## season()week3 307.93699 236.82054 1.300 0.194
## season()week4 151.58611 235.38410 0.644 0.520
## season()week5 -38.09659 26.15746 -1.456 0.146
## season()week6 12.59037 26.06463 0.483 0.629
## season()week7 12.88706 26.06608 0.494 0.621
## Temperature -302.41220 6.77999 -44.604 <2e-16 ***
## I(Temperature^2) 6.50241 0.14168 45.894 <2e-16 ***
## Day_TypeWeekday 856.92773 43.21995 19.827 <2e-16 ***
## Day_TypeWeekend -120.78785 231.81228 -0.521 0.602
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 230.3 on 1084 degrees of freedom
## Multiple R-squared: 0.8135, Adjusted R-squared: 0.8116
## F-statistic: 429.8 on 11 and 1084 DF, p-value: < 2.22e-16Accuracy Metrics
accuracy(fit)## # A tibble: 1 × 10
## .model .type ME RMSE MAE MPE MAPE MASE RMSSE ACF1
## <chr> <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 full Training 1.07e-13 229. 170. -0.243 3.66 0.577 0.490 0.594glance(fit) %>% select(r_squared, adj_r_squared, AIC, BIC)## # A tibble: 1 × 4
## r_squared adj_r_squared AIC BIC
## <dbl> <dbl> <dbl> <dbl>
## 1 0.813 0.812 11937. 12002.Residual Diagnostics
fit %>% gg_tsresiduals()
augment(fit) %>%
features(.innov, ljung_box, lag = 28, dof = length(coef(fit$full[[1]])) - 1)## # A tibble: 1 × 3
## .model lb_stat lb_pvalue
## <chr> <dbl> <dbl>
## 1 full 2688. 0Comparison: With vs. Without Temperature
fit_compare <- daily %>%
model(
with_temp = TSLM(Demand ~ trend() + season() + Temperature + I(Temperature^2) + Day_Type),
without_temp = TSLM(Demand ~ trend() + season() + Day_Type)
)
glance(fit_compare) %>% select(.model, r_squared, adj_r_squared)## # A tibble: 2 × 3
## .model r_squared adj_r_squared
## <chr> <dbl> <dbl>
## 1 with_temp 0.813 0.812
## 2 without_temp 0.449 0.444accuracy(fit_compare) %>% select(.model, .type, RMSE, MAE)## # A tibble: 2 × 4
## .model .type RMSE MAE
## <chr> <chr> <dbl> <dbl>
## 1 with_temp Training 229. 170.
## 2 without_temp Training 394. 308.Fitted vs. Actual
augment(fit) %>%
autoplot(Demand, colour = "grey40") +
geom_line(aes(y = .fitted), colour = "steelblue") +
labs(title = "Actual vs Fitted Demand", y = "MW")
Dynamic Regression (ARIMA Errors / ARIMAX)
fit_dynamic <- daily %>%
model(
dynamic = ARIMA(Demand ~ trend() + season() + Temperature + I(Temperature^2) + Day_Type)
)
report(fit_dynamic)## Series: Demand
## Model: LM w/ ARIMA(2,0,2)(1,0,1)[7] errors
##
## Coefficients:
## ar1 ar2 ma1 ma2 sar1 sma1 trend()
## 1.5937 -0.6036 -0.7587 -0.0984 0.5636 -0.5469 -0.3035
## s.e. 0.0793 0.0753 0.0899 0.0517 0.5845 0.5874 0.1832
## season()week2 season()week3 season()week4 season()week5 season()week6
## -66.0189 121.1796 -37.1295 -66.9163 -12.1835
## s.e. 11.8343 141.8614 141.8224 17.1216 15.6481
## season()week7 Temperature I(Temperature^2) Day_TypeWeekday
## -10.4935 -150.6212 3.5166 613.3975
## s.e. 11.8684 6.0568 0.1212 21.3277
## Day_TypeWeekend intercept
## -176.3664 5932.1766
## s.e. 142.4233 137.5757
##
## sigma^2 estimated as 19558: log likelihood=-6961.76
## AIC=13961.51 AICc=13962.22 BIC=14056.5fit_dynamic %>% gg_tsresiduals()
accuracy(fit_dynamic)## # A tibble: 1 × 10
## .model .type ME RMSE MAE MPE MAPE MASE RMSSE ACF1
## <chr> <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 dynamic Training 0.440 139. 95.7 -0.101 2.06 0.324 0.297 -0.000442Side-by-Side Accuracy: TSLM vs. Dynamic Regression
bind_rows(
accuracy(fit) %>% mutate(Model = "TSLM (static)"),
accuracy(fit_dynamic) %>% mutate(Model = "ARIMA errors (dynamic)")
) %>%
select(Model, RMSE, MAE, MAPE)## # A tibble: 2 × 4
## Model RMSE MAE MAPE
## <chr> <dbl> <dbl> <dbl>
## 1 TSLM (static) 229. 170. 3.66
## 2 ARIMA errors (dynamic) 139. 95.7 2.06Guiding Questions
How did adding external predictors impact forecast accuracy?
Adding Temperature (and its squared term) as an external predictor substantially improved forecast accuracy. Comparing the with_temp and without_temp models in the section above, R2 rises noticeably once Temperature is included, and RMSE drops as well. This makes sense given the nature of the data: electricity demand is heavily driven by heating and cooling needs, so a model relying only on trend, seasonality, and day type is missing one of the biggest real drivers of daily demand. The squared temperature term also matters here, since demand rises on both very cold and very hot days relative to mild days, a U-shaped relationship that a linear temperature term alone would not capture.
Did you observe any autocorrelation in residuals? How did you address it?
Yes. The Ljung-Box test on the TSLM residuals rejects the null hypothesis of no autocorrelation, and the residual ACF plot from shows multiple spikes outside the significance bounds. This tells us that, even after accounting for trend, seasonality, temperature, and day type, there is still day-to-day structure left in the errors, meaning today’s residual is somewhat predictive of tomorrow’s. To address this, I fit a dynamic regression model using ARIMA with the same external regressors, which models the errors with an ARIMA structure instead of assuming they are white noise. Reviewing gg_tsresiduals() for the dynamic model shows far less remaining autocorrelation than the static TSLM version, confirming that letting the error term follow an ARIMA process absorbs most of the leftover day-to-day dependence.
Would an ARIMAX model be more appropriate for your dataset? Why or why not?
Yes, it is more appropriate for this dataset. Since the TSLM model’s residuals showed significant autocorrelation, the standard errors and inference from the plain regression cannot fully be trusted, and there is clearly additional predictable structure in the series that a static regression cannot capture. The dynamic regression keeps the interpretability of the external predictors (temperature, day type, seasonality) while allowing the error term to follow an ARIMA process, which better reflects how electricity demand actually behaves from one day to the next. The side-by-side accuracy comparison at the end of this document shows whether this additional structure translates into a real improvement in RMSE, MAE, and MAPE over the static model.
Part 3: Reflection
As discussed in Part 1, judgmental forecasting can be really strong when there is no data available for what you want to forecast. On top of that, a lot of what drives demand is rooted in human behavior or other qualitative factors that can’t simply be reduced to a number. Having the ability to do a judgmental forecast means you can put some form of estimate on something you otherwise would have had no way to forecast at all. As also discussed, judgmental forecasting has weaknesses for the exact same reasons it has strengths: it is subject to human error and can carry biases such as anchoring bias, optimism bias, overconfidence, and more. Still, in the end, it is a great tool to use either on its own or as a complement to other methods, as long as that human error is taken into account.
The example in Part 2 shows where time series regression is genuinely strong: when there is a large amount of clean historical data and clear, measurable drivers behind the outcome you’re forecasting. Adding Temperature as an external predictor raised R2 and lowered RMSE substantially compared to the model without it, which shows that when the right predictors are available, a regression model can explain the vast majority of the variation in the data in a way that is fully reproducible and testable, something judgmental forecasting cannot offer. Regression also makes the relationship between predictors and demand explicit and interpretable. For instance, the squared Temperature term makes it clear that demand rises on both very cold and very hot days, a relationship that can be checked and communicated with numbers instead of relying on someone’s intuition.
That said, the same models also reveal the weaknesses of time series
regression. The Ljung-Box test and residual ACF plot showed significant
autocorrelation left in the TSLM residuals, meaning the plain regression
was not capturing everything happening day to day, and its standard
errors and significance tests could not be fully trusted as a result.
This shows that regression models are only as good as the predictors and
structure you give them. If an important driver is missing, or if the
errors have their own underlying pattern, the model’s assumptions break
down and it can quietly give a false sense of confidence in numbers that
are not fully reliable. Fitting the dynamic regression (ARIMA errors)
helped address this, but it also shows that getting a trustworthy
regression model working takes more effort and statistical care than
simply running one lm()-style formula.
Between the two approaches, time series regression was more effective for the dataset specifically because there was enough clean historical data and clear external drivers (temperature, day type, seasonality) to model directly. Judgmental forecasting would add little value here beyond helping choose which predictors make sense or flagging unusual days, like an unprecedented heatwave, that the model might not handle well. In practice, the two approaches work best together: statistical regression provides the reproducible, testable baseline forecast, while judgmental input is reserved for situations the model cannot see, such as brand-new products with no history, one-off events, or known future changes like a planned outage or promotion that has not happened yet in the data.