Intro to unfold

“The future enters into us, in order to transform itself in us, long before it happens.” - Rainer Maria Rilke

“Life is a process of becoming, a combination of states we have to go through.” - Anais Nin

“Time discovers truth.” - Seneca

“I am large, I contain multitudes.” - Walt Whitman

unfolding the hidden dimensions of future

The unfold algorithm reframes time series forecasting as a geometric learning problem. It first slices multiple series into non-overlapping windows and builds a 4D distance tensor that encodes pairwise temporal differences across sequences. A variational auto-mapper then compresses each window into a latent representation and decodes it to reconstruct or predict the next horizon. Forecasting proceeds by repeatedly encoding the most recent window and decoding its latent embedding into future values, effectively “unfolding” the hidden geometry of the series into stochastic predictive paths.

The unfold package works this way:

• Preparation and Input Validation. The unfold() function begins by validating the input and adapting parameters to the structure of the time series. It ensures that the input ts_set is a data frame where each column represents an individual univariate time series. A key parameter, horizon, which determines how far into the future the algorithm will attempt to forecast, is checked against a maximum feasible value derived from the available length of the series and the expected number of training windows. If the requested horizon exceeds this limit, it is automatically capped and a message is issued. The function also sets a random seed for reproducibility and verifies the validity of optional arguments such as the choice of metric ("euclidean", "mahalanobis", or "cosine") and output transformation ("linear" or "tanh").

• Reframing the Time Series and Constructing the Distance Tensor. Each individual time series is reframed into a collection of non-overlapping windows of length equal to the specified horizon. This transformation produces a 3D array of shape [n_windows, horizon, n_series], where each slice represents a segment of the time series data. To capture the geometric structure of the system, a 4D distance tensor is computed across all windows and series using the distances4d() function. This tensor, shaped [n_windows, n_windows, n_series, n_series], measures how different each pair of windows is across all variables. It supports multiple distance metrics and employs techniques such as diagonal masking, upper-triangle zeroing, and memory capping to remain efficient, even with high-dimensional data.

• Training the Variational Mapper. The core of unfold() lies in a variational autoencoder (VAE) model that learns to translate the geometric distance representation into future values. The model takes as input the distance tensor (minus the last window) and learns to predict the next raw window in the reframed series. It consists of an encoder that maps input distances into a latent space of dimension latent_dim, and a decoder that reconstructs future values. The encoder and decoder are multi-layer perceptrons with customizable hidden dimensions (enc_hidden and dec_hidden) and dropout regularization. The training process includes MSE reconstruction loss and optional KL divergence regularization, with support for gradient clipping, early stopping, and validation splitting. The train_variational_mapper() function handles this training process and returns both the fitted model and the training history.

• Generating Forecasts via Sampling and Projection. Once trained, the model is used to generate probabilistic forecasts. A final distance tensor is computed from the latest window and passed through the model’s generative component, producing n = 1000 samples of future growth rates for each variable and time step. These forecasts, initially expressed as relative changes, are converted into projected levels resulting in a large matrix of possible future paths, shaped [1000, horizon, n_series]. For each series and each time step, the algorithm estimates a smooth probability density function using a Gaussian mixture approximation (gmix()), controlled by the n_bases parameter.

• Visualization and Output Structure. To aid interpretation, unfold() provides a rich set of visual outputs. The projected densities are plotted for each series using the plot_graph() function, showing the range of plausible futures in a visually intuitive way. A separate ggplot chart of the training and validation losses is also generated to help assess model performance over epochs. All results are compiled into a single structured list, which includes the model object, the training log, the forecast functions, visualizations, and time elapsed. This makes the output convenient for further analysis, reporting, or interactive exploration.

The process behind unfold

What happens when you unfold hidden geometry? A practical example

In this example, we use unfold on a dataset to explore how patterns bend, stretch, and evolve over time. Our goal is to forecast 60 future steps by leveraging the full pipeline—distance tensors, variational mapping, and sample simulations.

# Load required libraries
library(unfold)

# Load a dummy dataset with sufficient observations.
set.seed(42)

# Execute unfold
results <- unfold(dummy_set[, c(2, 4)], 
                 horizon = 30,
                 metric = "euclidean", 
                 latent_dim = 5, 
                 enc_hidden = c(512, 256), 
                 dec_hidden = c(256, 512), 
                 p_drop = 0.1, 
                 out_kind = "tanh", 
                 epochs = 100, 
                 batch_size = 64, 
                 lr = 0.001, 
                 beta = 1, 
                 beta_warmup = 0, 
                 grad_clip = NULL, 
                 valid_split = 0.3, 
                 verbose = FALSE, 
                 alpha = 0.1, 
                 dates = NULL,  
                 patience = 30, 
                 n_bases = 10, 
                 seed = 42)

# Display the computation time and forecast plot
results$time_log

[1] "6S"

results$graph_plot

$MSFT.Close

$MARA.Close

The result is a list containing:

• Description: A textual summary of the variational model used: architecture layers, latent space size, activation functions, dropout rate, and training configuration. This helps users quickly understand the internal setup.

• Model: The trained variational mapper object, which includes the encoder-decoder network, latent space dynamics, and prediction capability. This model can be reused to make further inferences on new inputs.

• 4D Distance Tensor: A high-dimensional array capturing time-dependent pairwise distances across all time series segments. It reflects the evolving geometry of the data and serves as the model’s main input.

• Loss Plot: A ggplot2 graph showing training and validation loss across epochs. It helps users evaluate convergence and possible overfitting/underfitting patterns during training.

• Forecast Functions (pred_funs): A list of probabilistic forecast functions (one per time series), each providing smoothed projections for every future time step. These functions are useful for simulation, sampling, and density estimation.

• Forecast Graphs (graph_plot): Visuals that overlay past time series values with probabilistic forecast distributions, giving an intuitive picture of uncertainty and expected trajectories.

• Time Log: A human-readable duration that tracks how long the entire process took, useful for benchmarking or debugging long runs.

Let’s inspect the structure of train and validation loss inside results$model$history:

results$loss_plot

The loss plot in unfold displays the training and validation loss over epochs, offering insights into how well the variational mapper learns to reconstruct future time series from distance-based embeddings. The training loss shows how well the model fits the known data, while the validation loss indicates its generalization to unseen patterns. A smooth decline in both suggests effective learning; divergence may signal overfitting. The loss is computed using mean squared error (MSE) between predicted and actual windows, which captures reconstruction accuracy, and optionally weighted by a KL-divergence term to regularize the latent space. This combination ensures a balance between precise forecasting and meaningful latent representation.

Let’s inspect the description of the model

results$description

[1] "Variational Auto-Mapper (VAM). Latent dimension: 32. Encoder hidden layers: 512 -> 256. Decoder hidden layers: 256 -> 512. Dropout: 0.1. Output kind: linear. Target shape: [60 x 3]. Total parameters: 541,940"

Now, let’s explore the predictive functions for the last forecast point (t30) for the time series included in the dummy_set:

# List available predictive functions
names(results$pred_funs$MSFT.Close$t30)

[1] "rfun" "dfun" "pfun" "qfun"

Using these functions at each forecasted time point, you can compute confidence intervals, extract quantiles, and generate random samples from the predicted distribution—enabling both uncertainty estimation and full probabilistic forecasting. For example:

# Calculate the 95% confidence interval for t30 predictions
results$pred_funs$MSFT.Close$t30$qfun(c(0.025, 0.975))

[1] 467.9539 555.0898

# Generate 1000 random forecast samples for t30 and compute their mean
mean(results$pred_funs$MARA.Close$t30$rfun(1000))

[1] 16.06398

# Probability of positive growth at T10
1 - results$pred_funs$MARA.Close$t30$pfun(tail(dummy_set$MARA.Close, 1))

[1] 0.7088431

# Probability of a value between 495 and 520 at t30
results$pred_funs$MSFT.Close$t30$pfun(520) - results$pred_funs$MSFT.Close$t30$pfun(495)

[1] 0.3854748

# Odds for value 470 over value 550 at t30
results$pred_funs$MSFT.Close$t30$dfun(470)/results$pred_funs$MSFT.Close$t30$dfun(550)

[1] 0.5461458

Final Thoughts

When it comes to forecasting, it’s easy to feel like you’re peering into a foggy crystal ball—except this one hums with tensors, distances, and latent dimensions. unfold doesn’t promise prophecy, but it does offer a statistically different way to guess into the unknown. Just remember: time doesn’t like to be pinned down. It wiggles, it folds, it throws curveballs. We just try to unfold it—one probability distribution at a time. And if your forecast misses the mark? Don’t worry. Somewhere, a butterfly just flapped its wings and changed everything (yeah, it’s a joke).

Enzoi!