1.1 MCMC Algorithm Selection

Highlander delegates MCMC sampling to the LaplacesDemon package, which provides dozens of algorithms (see ?LaplacesDemon::LaplacesDemon). The default is CHARM (Componentwise Hit-And-Run Metropolis), which is a reliable general-purpose choice. Drawing on MCMC literature and the LaplacesDemon tutorial, the following guidelines can help when selecting an alternative via the Algorithm field of the LDargs argument:

4–8 parameters: CHARM (default) performs well at low dimensions. NUTS (No-U-Turn Sampler) and HMCDA are very efficient when the log-posterior is differentiable and smooth. RAM (Robust Adaptive Metropolis) and Slice sampling are also strong choices, and the t-walk (twalk) requires no gradient information and is easy to apply. AIES also works well in this domain, and is popular in astronomy.

8–20 parameters: Gradient-based samplers become increasingly valuable as dimension grows. NUTS and HMCDA are recommended when a differentiable log-posterior is available. DEMC (Differential Evolution Markov Chain) handles correlated parameters well in this range. AFSS (Automated Factor Slice Sampler) is the LaplacesDemon-recommended general-purpose algorithm and adapts automatically. CHARM remains viable but may converge more slowly than gradient-based alternatives. AIES also works well in this domain, and is popular in astronomy.

20–40 parameters: NUTS continues to perform well when gradients are accessible. DEMC and AFSS (used with parameter blocks) are strong gradient-free options. AMWG (Adaptive Metropolis-within-Gibbs) with carefully selected blocks scales efficiently at this dimensionality. CHARM can still be used but is likely to mix slowly; blockwise versions such as AMM or AMWG are preferable.

40-100 parameters: fitting will be getting very hard, especially if likelihoods are not auto-differentiable (if they are, then probably you should be using Stan or similar at this point). The best approach will take some user experimentation, but CHARM is often not a bad starting point.

To switch algorithm, pass Algorithm via LDargs:

Highlander(..., LDargs=list(Algorithm='**SEE BELOW**', Specs = list(**SEE BELOW**), Iterations=1000, Thinning=1))

The LaplacesDemon::Juxtapose function can compare algorithm efficiencies on a simplified version of your problem before committing to a full run.

1.2 Notation

Throughout this document we use the following notation:

• \(\theta \in \mathbb{R}^d\) — the parameter vector of dimension \(d\).
• \(\pi(\theta)\) — the unnormalised target (posterior) density, evaluated on the log scale as \(\log\pi(\theta)\) (LP in LaplacesDemon).
• \(q(\theta' \mid \theta)\) — the proposal density.
• \(\alpha(\theta, \theta')\) — the Metropolis–Hastings acceptance probability.
• \(U(\theta) = -\log\pi(\theta)\) — the potential energy function (used in Hamiltonian samplers).
• \(K(p) = \frac{1}{2}p^T M^{-1} p\) — the kinetic energy, where \(p\) is the auxiliary momentum and \(M\) is a mass matrix.

1.3 Using the Algorithm Argument

The algorithm is selected by passing a character string to the Algorithm argument of LaplacesDemon():

library(LaplacesDemon)

Fit <- LaplacesDemon(
  Model,
  Data        = MyData,
  Initial.Values = iv,
  Iterations  = 10000,
  Thinning    = 10,
  Algorithm   = "CHARM",               # <-- select algorithm here
  Specs       = list(alpha.star = NA)  # <-- algorithm-specific specs
)

Each algorithm may require a different Specs list. The required and optional elements of Specs for every algorithm are documented below.

2.1 Random-Walk Metropolis (RWM)

Description. The simplest MH variant. A zero-mean, isotropic Gaussian perturbation is added to the current state:

\[ \theta' = \theta^{(t)} + \epsilon, \qquad \epsilon \sim \mathcal{N}(0,\,\Sigma). \]

For well-tuned proposals the optimal acceptance rate is approximately 23.4% for high-dimensional targets (Roberts et al. 1997).

Specs

When to use. Good default choice for low- to moderate-dimensional problems when no gradient information is available. Requires careful tuning of \(\Sigma\) for efficiency.

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "RWM",
  Specs     = list(B = NULL))

2.2 Independence Metropolis (IM)

Description. Proposals are drawn from a fixed distribution \(q(\theta')\) that does not depend on the current state. This is equivalent to importance sampling within an MH accept/reject step. It is most efficient when \(q\) closely approximates \(\pi\), and can be used with a multivariate normal or t proposal centred on the posterior mode.

Specs

When to use. Useful when a good approximation to \(\pi\) is available (e.g., from LaplaceApproximation()). Poor mixing when \(q\) deviates substantially from \(\pi\).

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "IM",
  Specs     = list(mu = posterior_mode))

2.3 Metropolis-within-Gibbs (MWG)

Description. Each parameter (or block) is updated in turn with a univariate Metropolis–Hastings step, conditional on all other parameters. The proposal variance for each component is tuned adaptively to reach a target acceptance rate of approximately 44%. This is the default algorithm in LaplacesDemon.

Specs

When to use. Very robust; works well even with correlated parameters because it updates one dimension at a time. May be slow for strongly correlated, high-dimensional posteriors because it cannot exploit inter-parameter correlations within a single proposal.

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "MWG",
  Specs     = list(B = NULL))

2.4 Adaptive Metropolis-within-Gibbs (AMWG)

Description. Extends MWG with componentwise adaptation of the log-scale proposal standard deviations following Roberts and Rosenthal (2009). Every Periodicity iterations the log-scale step is incremented or decremented to push each component’s acceptance rate towards 44%.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "AMWG",
  Specs     = list(B = NULL, n = 0, Periodicity = 50))

2.5 Adaptive Metropolis (AM)

Description. The Adaptive Metropolis algorithm of Haario, Saksman and Tamminen (2001) learns a full multivariate Gaussian proposal covariance from the history of the chain. After a burn-in of Adaptive iterations the proposal covariance is updated every Periodicity iterations:

\[ \Sigma_t = s_d \left[\operatorname{Cov}(\theta^{(0)},\ldots,\theta^{(t-1)}) + \varepsilon I_d\right], \]

where \(s_d = 2.38^2/d\) is the theoretically optimal scaling for a Gaussian target (Gelman et al. 1996).

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "AM",
  Specs     = list(Adaptive = 500, Periodicity = 10))

2.6 Adaptive-Mixture Metropolis (AMM)

Description. A mixture of an adaptive multivariate proposal (similar to AM) and a static componentwise proposal. Each iteration, the componentwise proposal is used with probability \(w\) and the adaptive multivariate proposal with probability \(1-w\). This helps maintain ergodicity while benefiting from adaptation (Roberts & Rosenthal 2009).

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "AMM",
  Specs     = list(Adaptive = 500, B = NULL, n = 0,
                   Periodicity = 10, w = 0.05))

2.7 Delayed Rejection Metropolis (DRM)

Description. When a proposal is rejected, a second-stage proposal of smaller scale is attempted before the chain remains at the current state. This reduces the probability of long sequences of rejections without violating detailed balance (Mira 2001).

Specs DRM requires no Specs (pass NULL or omit).

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "DRM",
  Specs     = NULL)

2.8 Delayed Rejection Adaptive Metropolis (DRAM)

Description. Combines delayed rejection (DRM) with the adaptive multivariate proposal of AM. After Adaptive iterations the proposal covariance is updated every Periodicity iterations, and rejected proposals trigger a second, smaller attempt (Haario et al. 2006).

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "DRAM",
  Specs     = list(Adaptive = 500, Periodicity = 10))

2.9 Robust Adaptive Metropolis (RAM)

Description. Vihola’s (2011) RAM algorithm adapts the proposal Cholesky factor \(S\) so that the mean acceptance rate converges to a target \(\alpha^*\). The adaptation rule is:

\[ S_{t+1} S_{t+1}^T = S_t \!\left(I + \eta_t (\alpha_t - \alpha^*) \frac{u_t u_t^T}{\|u_t\|^2}\right) S_t^T, \]

where \(u_t = \theta' - \theta^{(t)}\) and \(\eta_t = \min(1, d\,t^{-\gamma})\). RAM is guaranteed to converge under mild conditions.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "RAM",
  Specs     = list(alpha.star = 0.234, B = NULL,
                   Dist = "N", gamma = 0.66, n = 0))

2.10 Multiple-Try Metropolis (MTM)

Description. Instead of a single proposal, \(K\) candidate points are drawn from the proposal distribution. One candidate \(\theta'\) is selected with probability proportional to \(\pi(\theta')\,w(\theta',\theta)\), and accepted via a modified MH ratio that maintains detailed balance (Liu et al. 2000). For a fixed computational budget it can achieve better mixing than standard MH.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "MTM",
  Specs     = list(K        = 4,
                   CPUs     = 1,
                   Packages = NULL,
                   Dyn.libs = NULL))

2.11 Random Dive Metropolis-Hastings (RDMH)

Description. A multiplicative random-walk sampler on \(\mathbb{R}\), proposed by Dutta (2012). The proposal is \(\theta'_j = \theta_j \cdot \exp(\epsilon_j)\), \(\epsilon_j \sim \mathcal{N}(0, \sigma^2)\). This makes it scale-free and avoids the sign problem of additive proposals when parameters are constrained to be positive.

Specs RDMH requires no Specs (pass NULL or omit).

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "RDMH",
  Specs     = NULL)

3.1 Componentwise Hit-And-Run Metropolis (CHARM)

Description. The default algorithm used by Highlander when calling LaplacesDemon(). At each iteration a random direction \(v\) is drawn uniformly from the unit sphere and a step of size \(\sigma\) is proposed along \(v\):

\[ \theta' = \theta^{(t)} + \sigma v. \]

Unlike standard MWG, CHARM mixes coordinate directions, which can aid mixing when parameters are correlated. When alpha.star is provided, the scale \(\sigma\) is tuned to approach that acceptance rate.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "CHARM",
  Specs     = list(alpha.star = NA))

3.2 Hit-And-Run Metropolis (HARM)

Description. Similar to CHARM but the direction is drawn from a fixed (non-adapting) distribution. Smith (1984) originally proposed hit-and-run for uniform sampling over convex sets; the HARM variant adapts it to general log-concave targets using Gaussian proposals along the chosen direction.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "HARM",
  Specs     = list(alpha.star = NA, B = NULL))

3.3 Adaptive Directional Metropolis-within-Gibbs (ADMG)

Description. Bai (2009) proposed updating each parameter along an adaptively-chosen direction within the Gibbs framework. The direction for each component is adapted using past samples to align with the local curvature of the posterior, reducing the correlations that slow componentwise MH.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "ADMG",
  Specs     = list(n = 0, Periodicity = 1))

4.1 Gibbs Sampler (Gibbs)

Description. The user supplies a function FC that computes and returns updated parameters from their full conditionals. This is the exact Gibbs sampler of Geman and Geman (1984); no Metropolis–Hastings correction is needed because samples are drawn exactly.

Specs

FC <- function(parm, Data) {
  # draw each parameter from its full conditional
  parm[1] <- rnorm(1, ...)
  return(parm)
}

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "Gibbs",
  Specs     = list(FC = FC, MWG = NULL))

4.2 Griddy-Gibbs (GG)

Description. When full conditionals are not available analytically, Ritter and Tanner (1992) proposed evaluating the unnormalised conditional on a user-defined grid and drawing via inverse CDF. This is exact up to the grid resolution and avoids Metropolis correction.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "GG",
  Specs     = list(dparm    = NULL,
                   Grid     = seq(-0.1, 0.1, length.out = 5),
                   CPUs     = 1,
                   Packages = NULL,
                   Dyn.libs = NULL))

4.3 Adaptive Griddy-Gibbs (AGG)

Description. Extends GG by adapting the grid width using the running variance of each parameter. This removes the burden of hand-tuning the grid while preserving the exact-Gibbs character of the sampler.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "AGG",
  Specs     = list(dparm    = NULL,
                   Grid     = seq(-0.1, 0.1, length.out = 5),
                   smax     = Inf,
                   CPUs     = 1,
                   Packages = NULL,
                   Dyn.libs = NULL))

5.1 Affine-Invariant Ensemble Sampler (AIES)

Description. Goodman and Weare’s (2010) ensemble sampler maintains \(N\) walkers \(\{\theta^{(k)}\}_{k=1}^N\). For walker \(k\), a complement walker \(j \ne k\) is selected at random and the proposal is

\[ \theta' = \theta^{(j)} + Z \left(\theta^{(k)} - \theta^{(j)}\right), \quad Z \sim g(z), \]

where \(g\) is chosen so the sampler is affine-invariant. The default choice gives \(Z\) with PDF \(g(z) \propto z^{-1/2}\) on \([1/\beta, \beta]\) with \(\beta = 2\).

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "AIES",
  Specs     = list(Nc       = 16,
                   Z        = NULL,
                   beta     = 2,
                   CPUs     = 1,
                   Packages = NULL,
                   Dyn.libs = NULL))

5.2 Differential Evolution Markov Chain (DEMC)

Description. DEMC (Ter Braak & Vrugt 2008) runs \(N_c \ge 3\) chains in parallel. The proposal for chain \(i\) uses the difference of two randomly-chosen other chains:

\[ \theta'_i = \theta_i^{(t)} + \gamma (\theta_r^{(t)} - \theta_s^{(t)}) + e, \quad e \sim \mathcal{N}(0, \epsilon I), \]

where \(\gamma = 2.38/\sqrt{2d}\) by default. A “snooker” update (probability \(w\)) projects along the line connecting the current chain to a randomly-chosen history point.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "DEMC",
  Specs     = list(gamma = NULL, Nc = 3, w = 0.1, Z = NULL))

5.3 Interchain Adaptation (INCA)

Description. Craiu, Rosenthal and Yang (2009) proposed INCA, which uses parallel chains to adapt a shared proposal covariance. Regional information from nearby chains is combined to improve the proposal for each chain, yielding faster mixing than independent AM chains.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "INCA",
  Specs     = list(Adaptive = 500, Periodicity = 10))

5.4 Metropolis-Coupled MCMC (MCMCMC)

Description. Geyer’s (1991) parallel tempering runs \(T\) chains at different temperatures \(\{1/\lambda_t\}\). The cold chain (\(t=1\)) targets \(\pi(\theta)\); hotter chains explore more freely. Periodically, adjacent chains attempt a swap:

\[ \alpha_{\text{swap}} = \min\!\left(1,\; \frac{\pi(\theta_t)^{1/\lambda_{t+1}} \pi(\theta_{t+1})^{1/\lambda_t}} {\pi(\theta_t)^{1/\lambda_t} \pi(\theta_{t+1})^{1/\lambda_{t+1}}} \right). \]

This helps the cold chain escape local modes.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "MCMCMC",
  Specs     = list(lambda   = 1,
                   CPUs     = 2,
                   Packages = NULL,
                   Dyn.libs = NULL))

6.1 Hamiltonian Monte Carlo (HMC)

Description. The classic HMC sampler. Gradient \(\nabla_\theta \log\pi(\theta)\) must be evaluated at each leapfrog step, making it expensive per iteration but typically much more efficient per independent sample than gradient-free methods.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "HMC",
  Specs     = list(epsilon = 0.1, L = 20, m = NULL))

6.2 Adaptive Hamiltonian Monte Carlo (AHMC)

Description. Adapts the mass matrix and step sizes from the running covariance of the chain, removing the need to hand-tune these HMC parameters.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "AHMC",
  Specs     = list(epsilon = 0.1, L = 20, m = NULL, Periodicity = 10))

6.3 Hamiltonian Monte Carlo with Dual-Averaging (HMCDA)

Description. Hoffman and Gelman’s (2014) HMCDA uses Nesterov’s dual-averaging scheme to automatically tune the step size \(\epsilon\) to achieve a target acceptance rate \(\delta\) (typically 0.65), and adapts the trajectory length \(\lambda = L\epsilon\) to a fixed user-supplied target.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "HMCDA",
  Specs     = list(A       = 500,
                   delta   = 0.65,
                   epsilon = NULL,
                   lambda  = 0.1,
                   Lmax    = 1000))

6.4 No-U-Turn Sampler (NUTS)

Description. NUTS (Hoffman & Gelman 2014) eliminates the need to choose \(L\) by automatically stopping the leapfrog trajectory when it would start to backtrack (“make a U-turn”). Combined with dual-averaging for \(\epsilon\), NUTS is nearly parameter-free and often outperforms hand-tuned HMC.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "NUTS",
  Specs     = list(A = 500, delta = 0.6,
                   epsilon = NULL, Lmax = 1000))

6.5 Tempered Hamiltonian Monte Carlo (THMC)

Description. A variant of HMC that heats the momentum in the first half of the leapfrog trajectory and cools it in the second half, controlled by a Temperature parameter. Higher temperatures encourage exploration of multimodal posteriors.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "THMC",
  Specs     = list(epsilon = 0.1, L = 20, m = NULL, Temperature = 1))

6.6 Metropolis-Adjusted Langevin Algorithm (MALA)

Description. MALA uses a single gradient-guided Langevin step

\[ \theta' = \theta^{(t)} + \frac{\epsilon^2}{2} \nabla\log\pi(\theta^{(t)}) + \epsilon\,\xi, \quad \xi \sim \mathcal{N}(0, I), \]

followed by an MH correction. Atchadé (2006) proposed an adaptive truncated version that clips the drift to prevent divergence. The optimal acceptance rate for MALA is approximately 57.4%.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "MALA",
  Specs     = list(A = Inf, alpha.star = 0.574,
                   delta = 1, epsilon = c(1e-6, 1e-7),
                   gamma = 1))

6.7 Stochastic Gradient Langevin Dynamics (SGLD)

Description. Welling and Teh (2011) proposed SGLD for big-data settings. At each iteration, a random minibatch of size rows is drawn from a large dataset stored on disk, and a noisy gradient is used in a Langevin step with step size \(\epsilon_t\). No MH correction is needed when \(\epsilon_t \to 0\).

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "SGLD",
  Specs     = list(epsilon = 1e-4, file = "bigdata.csv",
                   Nc = 10, Nr = 1e6, size = 200))

6.8 Preconditioned Crank-Nicolson (pCN)

Description. Beskos et al. (2008) introduced pCN for infinite-dimensional targets. The proposal is

\[ \theta' = \sqrt{1-\beta^2}\,\theta^{(t)} + \beta\,\xi, \quad \xi \sim \mathcal{N}(0, C), \]

where \(C\) is a prior covariance. pCN is dimension-robust: its acceptance rate does not degrade as dimension increases, making it suitable for large hierarchical models.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "pCN",
  Specs     = list(beta = 0.1))

7.1 Slice Sampler (Slice)

Description. Neal’s (2003) general univariate (or blockwise) slice sampler uses “stepping out” and “shrinkage” to find the slice. For each parameter (or block) an interval of width \(w\) is grown until it brackets the slice, then a point is drawn uniformly from the interval and shrunk if necessary.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "Slice",
  Specs     = list(B = NULL,
                   Bounds = c(-Inf, Inf),
                   m = 100,
                   Type = "Continuous",
                   w = 1))

7.2 Automated Factor Slice Sampler (AFSS)

Description. Tibbits et al. (2014) proposed AFSS, which adapts an orthogonal factor structure of the parameter space to align the slice with the principal axes of the posterior. This greatly reduces the number of steps needed to create the slice in correlated, high-dimensional problems.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "AFSS",
  Specs     = list(A = Inf, B = NULL, m = 100, n = 0, w = 1))

7.3 Univariate Eigenvector Slice Sampler (UESS)

Description. Thompson (2011) proposed slicing along the eigenvectors of the empirical covariance matrix of the chain. Moving along decorrelated directions yields a slice interval that is better aligned with the posterior geometry.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "UESS",
  Specs     = list(A = Inf, B = NULL, m = 100, n = 0))

7.4 Reflective Slice Sampler (RSS)

Description. An extension of the slice sampler that reflects the proposal off the boundary of the slice rather than shrinking, following the trajectory of a billiard ball. This can give better mixing in elongated or curved posteriors.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "RSS",
  Specs     = list(m = 1, w = 1))

7.5 Oblique Hyperrectangle Slice Sampler (OHSS)

Description. Generalises the hyperrectangle slice sampler to oblique (rotated) hyperrectangles, allowing better coverage of posteriors with axis-aligned correlations.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "OHSS",
  Specs     = list(A = Inf, n = 0))

7.6 Elliptical Slice Sampler (ESS)

Description. Murray, Adams and MacKay (2010) proposed ESS for models with a Gaussian prior. A “ellipse” of proposals is defined using the current state and a Gaussian draw; the algorithm searches the ellipse for a point that satisfies the likelihood slice condition. No tuning is required.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "ESS",
  Specs     = list(B = NULL))

8.1 t-walk (twalk)

Description. Christen and Fox (2010) proposed the t-walk, a self-contained, essentially tuning-free sampler that maintains two points \(\theta\) and \(\phi\) and proposes moves using a random mixture of four “kernels”: walk, traverse, blow, and hop. The traverse kernel generates proposals along the line connecting the two points and is particularly effective for correlated posteriors.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "twalk",
  Specs     = list(at = 6, aw = 1.5, n1 = 4, SIV = NULL))

8.2 Refractive Sampler (Refractive)

Description. Boyles and Welling (2012) proposed a sampler inspired by Snell’s law of refraction. A particle crosses the boundary of a “lens” defined by the current log-posterior value and refracts according to the ratio of densities on either side. This creates proposals that preferentially explore high-density regions.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "Refractive",
  Specs     = list(Adaptive = 1, m = 2, w = 0.1, r = 1.3))

8.3 Reversible-Jump (RJ)

Description. Green’s (1995) reversible-jump MCMC allows the dimension of the parameter space to change between iterations, enabling joint inference over model selection and parameters. At each iteration a parameter is either added to or removed from the active set, with a binomial prior on model size.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "RJ",
  Specs     = list(bin.n = d, bin.p = 0.5,
                   parm.p   = rep(0.5, d),
                   selectable = rep(1, d),
                   selected   = rep(1, d)))

9.1 Sequential Metropolis-within-Gibbs (SMWG)

Description. Applies MWG updates to dynamic state parameters in a sequential state-space model. The Dyn matrix specifies dynamic parameter values for each of \(T\) time-periods.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "SMWG",
  Specs     = list(Dyn = Dyn_matrix))

9.2 Sequential Adaptive Metropolis-within-Gibbs (SAMWG)

Description. Extends SMWG with componentwise adaptive proposals for dynamic parameters.

Specs

Fit <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "SAMWG",
  Specs     = list(Dyn = Dyn_matrix, Periodicity = 50))

9.3 Updating Sequential Metropolis-within-Gibbs (USMWG)

Description. An updating variant of SMWG intended for online inference. Past samples stored in a Fit object are retained; only the time-periods from Begin onwards are re-sampled.

Specs

Fit_update <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "USMWG",
  Specs     = list(Begin = T_new, Dyn = Dyn_matrix, Fit = Fit_old))

9.4 Updating Sequential Adaptive Metropolis-within-Gibbs (USAMWG)

Description. Combines the adaptive proposals of SAMWG with the online updating capability of USMWG.

Specs

Fit_update <- LaplacesDemon(Model, MyData, iv,
  Algorithm = "USAMWG",
  Specs     = list(Begin = T_new, Dyn = Dyn_matrix,
                   Fit = Fit_old, Periodicity = 50))

11.1 No gradient available

• Low dimension (\(d \lesssim 20\)): MWG, AMWG, CHARM, HARM — simple, robust, easy to tune.
• Moderate dimension (\(d \sim 20\)–\(100\)): AM, AMM, DRAM, RAM — learn the proposal covariance adaptively.
• Correlated parameters: AMM, DEMC, AIES, twalk — use inter-chain information or direction-aware proposals.
• Multimodal posterior: MCMCMC, THMC, DEMC — tempering or parallel chains help escape local modes.
• Discrete parameters: GG, AGG, Slice — exact or grid-based sampling.
• Variable selection: RJ — reversible-jump for trans-dimensional moves.
• Big data: SGLD — stochastic gradients on minibatches.
• Online updating / state-space models: SMWG, SAMWG, USMWG, USAMWG.

11.2 Gradient available

• General purpose: NUTS — nearly tuning-free, state-of-the-art gradient-based sampler.
• Fixed hyperparameters: HMC — highly efficient when \(\epsilon\) and \(L\) are well tuned.
• Big data: SGLD — gradient estimated on minibatch.
• Multimodal: THMC — tempered momentum for better exploration.
• Infinite-dimensional / functional: pCN — dimension-robust.
• Gaussian likelihood / prior: ESS — requires no tuning.

11.3 The default: CHARM

Highlander sets CHARM as its default LaplacesDemon algorithm because it offers a good balance of generality, robustness, and ease of use. It requires no gradient, adapts its scale automatically, and handles correlated parameters better than standard MWG by mixing directions. For most problems it is a sensible starting point before switching to a more specialised algorithm if needed.