Intro to CRAFT

Conditional Regime Analog Forecasting with Trajectories

CRAFT turns recent multivariate trajectory memory into conditional, probabilistic futures. It does not try to boss time around. It asks a quieter and more useful question: when history looked like this, what tended to happen next?

“The future enters into us, in order to transform itself in us, long before it happens.” – Rainer Maria Rilke

“Time discovers truth.” – Seneca

“All models are wrong, but some are useful.” – George E. P. Box

What you can do with CRAFT

CRAFT stands for Conditional Regime Analog Forecasting with Trajectories.

It builds probabilistic forecasts for multivariate time series by converting level histories into lag and lead trajectory embeddings, detecting compact SVD-based changepoint regimes, estimating conditional transitions from present regimes to future regimes, sampling historical future-trajectory analogues, and fitting smooth empirical forecast distributions.

That sounds like a small machine shop because, well, it is. But the idea is simple:

1. Describe the recent past as a trajectory.
2. Find the regime structure that trajectory belongs to.
3. Estimate which future-regime profiles historically followed.
4. Resample plausible future trajectories.
5. Wrap the sampled futures in distribution functions you can query.

The package is especially useful when a lonely point forecast feels too smug. CRAFT keeps uncertainty visible: trajectories, probabilities, quantiles, level distributions, return distributions, and diagnostics all remain available for inspection.

The CRAFT package works this way

Trajectory preparation. trajectory_embedding() converts each numeric series into backward-looking cumulative-change windows and forward-looking cumulative-change windows. The lag windows describe what was known at each origin; the lead windows describe what happened next.

Regime detection. svd_changepoint_regimes() compresses trajectory matrices with singular value decomposition and detects changepoint-based regimes on the resulting factors. This keeps the regime map compact even when several assets and horizons are involved.

Conditional transitions. craft_fit() learns how past-regime labels map into future-regime labels. The transition model is intentionally modest: it estimates conditional probabilities and leaves the drama to the data.

Analog sampling. Historical future trajectories are resampled according to the latest regime probabilities. Optional sampler weights and tail penalties can make the analogue pool more selective.

Forecast distributions. trajectory_forecast() fits smooth empirical distributions to sampled trajectory draws. CRAFT exposes both return-space and level-space distributions, so you can ask quantile, density, probability, and simulation questions.

Historical futures walk into a regime map …

In this mini demo, we create three synthetic level series with a little shared structure. Not enough to form a committee, just enough to make dependence worth noticing.

1) Build a tiny multivariate history

set.seed(42)
n <- 140

market_pulse <- as.numeric(stats::arima.sim(
  model = list(ar = 0.72),
  n = n,
  sd = 0.005
))

rate_pulse <- as.numeric(stats::arima.sim(
  model = list(ar = 0.55),
  n = n,
  sd = 0.004
))

returns <- data.frame(
  asset_a = 0.0008 + market_pulse + rnorm(n, 0, 0.006),
  asset_b = 0.0005 + 0.55 * market_pulse - 0.25 * rate_pulse + rnorm(n, 0, 0.007),
  asset_c = 0.0006 - 0.20 * market_pulse + 0.65 * rate_pulse + rnorm(n, 0, 0.006)
)

series <- as.data.frame(lapply(returns, function(x) 100 * cumprod(1 + x)))

tail(round(series, 2), 3)

matplot(
  series,
  type = "l",
  lty = 1,
  lwd = 2,
  col = c("black", "gray35", "gray65"),
  xlab = "Time",
  ylab = "Level",
  main = "Synthetic multivariate history"
)
legend(
  "topleft",
  legend = names(series),
  col = c("black", "gray35", "gray65"),
  lty = 1,
  lwd = 2,
  bty = "n"
)

Three synthetic level series used in the CRAFT demo.

2) Fold the history into trajectories

A trajectory window is a compact profile of cumulative changes. Backward-looking windows feed the regime detector; forward-looking windows become the pool of historical analogues.

trajectories <- trajectory_embedding(series, trajectory_window = 6)

names(trajectories)
#> [1] "past_trajectories"   "future_trajectories" "latest_trajectory"  
#> [4] "time_index"
dim(trajectories$past_trajectories)
#> [1] 128  18
dim(trajectories$future_trajectories)
#> [1] 128  18

head(round(trajectories$past_trajectories[, 1:6], 4), 3)
#>      asset_a_cum_lag_1 asset_a_cum_lag_2 asset_a_cum_lag_3 asset_a_cum_lag_4
#> [1,]           -0.0053            0.0001            0.0043            0.0007
#> [2,]           -0.0008           -0.0060           -0.0006            0.0035
#> [3,]           -0.0055           -0.0062           -0.0115           -0.0061
#>      asset_a_cum_lag_5 asset_a_cum_lag_6
#> [1,]           -0.0061           -0.0233
#> [2,]           -0.0001           -0.0069
#> [3,]           -0.0019           -0.0056

3) Fit a smooth distribution from trajectory draws

Before fitting the full model, we can use trajectory_forecast() directly. Here we take the realized future trajectories for one asset and estimate a predictive interface with density, distribution, quantile, and random-generation functions. Very civilized. Almost suspiciously civilized.

asset_a_cols <- grep(
  "^asset_a_cum_lead_",
  colnames(trajectories$future_trajectories),
  value = TRUE
)

asset_a_draws <- as.data.frame(
  trajectories$future_trajectories[, asset_a_cols, drop = FALSE]
)

asset_a_fc <- trajectory_forecast(
  asset_a_draws,
  probs = seq(0.05, 0.95, length.out = 40),
  min_unique = 5,
  verbose = FALSE
)

names(asset_a_fc)
#> [1] "asset_a_cum_lead_1" "asset_a_cum_lead_2" "asset_a_cum_lead_3"
#> [4] "asset_a_cum_lead_4" "asset_a_cum_lead_5" "asset_a_cum_lead_6"
asset_a_fc[["asset_a_cum_lead_6"]]$qfun(c(0.05, 0.50, 0.95))
#> [1] -0.057988172  0.003995577  0.040557745

4) Fit CRAFT end to end

Now we let craft_fit() run the full sequence: trajectory embedding, regime labelling, transition probabilities, future-trajectory sampling, and forecast distribution fitting.

The settings below are deliberately small so the article does not ask your laptop fan to write a resignation letter.

fit <- craft_fit(
  series,
  window = 6,
  n_draws = 120,
  n_factors = 1,
  min_segment = 6,
  max_regimes_per_factor = 3,
  n_testing = 0,
  return_train_probs = FALSE,
  verbose = FALSE,
  seed = 123
)

class(fit)
#> [1] "craft_fit"                  "regime_forecast_v5"        
#> [3] "regime_forecast_transition" "regime_forecast"
fit$valid_joint_acc
#> [1] 0
names(fit$return_dists)
#> [1] "asset_a" "asset_b" "asset_c"

The fitted object keeps both the machinery and the useful bits: trajectory matrices, regime models, latest labels, transition probabilities, sampled future trajectories, fitted return distributions, fitted level distributions, and diagnostics.

dim(fit$trajectory_draws)
#> [1] 120  18
names(fit$diagnostics)
#>  [1] "latest_unseen_counts"          "latest_unseen_total"          
#>  [3] "posterior_entropy"             "sampler_weight_effective_n"   
#>  [5] "trajectory_columns_used_names" "expected_trajectory_columns"  
#>  [7] "class_balance"                 "transition"                   
#>  [9] "valid_split"                   "temperature"                  
#> [11] "factor_selection"              "min_segment"                  
#> [13] "tail_penalty"

round(fit$sampler_weights, 3)
#> regime_dim_1 
#>            1
round(fit$diagnostics$posterior_entropy, 3)
#> regime_dim_1 
#>        0.868

5) Predict again, because the future keeps moving

craft_predict() reuses the fitted model. With no newdata, it resamples from the latest trajectory already stored in the fit. With newdata, it transforms a fresh history and runs the same conditional analogue logic again.

pred <- craft_predict(fit, n_draws = 75, seed = 7)

class(pred)
#> [1] "craft_prediction"           "regime_forecast_prediction"
dim(pred$trajectory_draws)
#> [1] 75 18

horizon_name <- tail(names(pred$return_dists$asset_a), 1)
pred$return_dists$asset_a[[horizon_name]]$qfun(c(0.05, 0.50, 0.95))
#> [1] -0.08379133 -0.00327543  0.06054972
pred$level_dists$asset_a[[horizon_name]]$qfun(c(0.05, 0.50, 0.95))
#> [1] 80.16185 87.20643 92.79068

You can also request only the part you need. This is helpful when a dashboard wants sampled trajectories while another report wants regime probabilities. Different rooms, same house.

labels <- craft_predict(fit, type = "labels")
probs <- craft_predict(fit, type = "probabilities")
draws <- craft_predict(fit, type = "trajectory_draws", n_draws = 10, seed = 9)

lapply(probs, function(x) round(x[1, ], 3))
#> $regime_dim_1
#>       1     2
#> 1 0.711 0.289
dim(draws)
#> [1] 10 18

6) Minimal plots, tiny but telling

The sampled future trajectories can be summarized as a forecast corridor. Below, the values are cumulative percentage changes from the forecast origin for asset_a.

asset <- "asset_a"
cols <- paste0(asset, "_cum_lead_", seq_len(fit$window))
asset_draws <- pred$trajectory_draws[, cols, drop = FALSE]

qs <- apply(
  asset_draws,
  2,
  stats::quantile,
  probs = c(0.05, 0.25, 0.50, 0.75, 0.95),
  na.rm = TRUE
)

matplot(
  seq_len(fit$window),
  t(100 * qs),
  type = "l",
  lty = c(2, 3, 1, 3, 2),
  lwd = c(1.5, 1.5, 3, 1.5, 1.5),
  col = c("gray50", "gray25", "black", "gray25", "gray50"),
  xlab = "Horizon",
  ylab = "Cumulative return (%)",
  main = "CRAFT analogue forecast corridor"
)
abline(h = 0, col = "gray70", lty = 3)
legend(
  "topleft",
  legend = c("5%", "25%", "50%", "75%", "95%"),
  col = c("gray50", "gray25", "black", "gray25", "gray50"),
  lty = c(2, 3, 1, 3, 2),
  lwd = c(1.5, 1.5, 3, 1.5, 1.5),
  bty = "n"
)

CRAFT analogue forecast corridor for asset_a on the cumulative-return scale.

A few practical knobs

CRAFT is intentionally inspectable. A few settings are worth knowing early:

• window controls the lag and lead trajectory length.
• n_factors and var_threshold control how much SVD structure is used for regimes.
• min_segment and max_regimes_per_factor control changepoint/regime granularity.
• sampler_weight can favor analogues linked to stronger validation performance.
• tail_penalty_lambda can reduce the appetite for extreme historical analogues.
• n_testing enables internal backtesting when you want a quick model-health check.

For example, a more cautious sampling setup might look like this:

fit_cautious <- craft_fit(
  series,
  window = 6,
  n_draws = 500,
  n_factors = 2,
  sampler_weight = "weighted_bal_acc",
  tail_penalty_lambda = 0.25,
  tail_penalty_threshold = 3,
  n_testing = 5,
  verbose = FALSE,
  seed = 123
)

What to look at when the model returns

• Inspect fit$valid_joint_acc and transition diagnostics. They tell you whether the regime transition model is learning anything more useful than a shrug.
• Inspect fit$diagnostics$posterior_entropy. High entropy means the latest regime probabilities are diffuse. That is not necessarily bad; sometimes the model is being appropriately humble.
• Inspect fit$trajectory_draws. These are the sampled analogue futures. If they look absurd, the distributions fitted on top of them will be beautifully absurd, which is still absurd.
• Use fit$level_dists when you want forecast levels and fit$return_dists when you want cumulative-change forecasts.

Conclusion

CRAFT is a memory-based probabilistic forecasting workflow: profile the recent past, find comparable regime structure, sample what historically came next, and wrap the result in forecast distributions that can answer quantile, density, probability, and simulation questions.

It will not make time behave. Time has never accepted calendar invites from models. But CRAFT gives the next trajectory a disciplined set of historical analogues, and that is often a better conversation than a lonely point forecast pretending to be certain.

Enzoi.