Introduction

This note relates to the paper Spatio-temporal modeling of particulate matter concentration through the SPDE approach by Cameletti et a. (Cameletti, M.; Lindgren, F.; Simpson, D.; Rue, H. Spatio-temporal modeling of particulate matter concentration through the SPDE approach. AStA Advances in Statistical Analysis 2013, 97, 109-131, doi https://doi.org/10.1007/s10182-012-0196-3.)

Here, I aim to demonstrate how the final equation for the joint posterior distribution, shown below, is derived.

\[ \begin{aligned} \pi(\theta, \xi \mid y) \propto & \; (\sigma_\varepsilon^2)^{-dT/2} \exp\left[ -\frac{1}{2\sigma_\varepsilon^2} \sum_{t=1}^T (y_t - z_t\beta - \xi_t)'(y_t - z_t\beta - \xi_t) \right] \\ & \times (\sigma_\omega^2)^{-dT/2} (1-a^2)^{d/2} |\tilde{\Sigma}|^{-T/2} \\ & \times \exp\left[ -\frac{1-a^2}{2\sigma_\omega^2} \xi_1' \tilde{\Sigma}^{-1} \xi_1 - \frac{1}{2\sigma_\omega^2} \sum_{t=2}^T (\xi_t - a\xi_{t-1})' \tilde{\Sigma}^{-1} (\xi_t - a\xi_{t-1}) \right] \\ & \times \prod_{i=1}^{\dim(\theta)} \pi(\theta_i) \end{aligned} \]

Step 1: Starting Point – Equation (7)

The joint posterior is:

\[ \pi(\theta, \xi \mid y) \propto \pi(y \mid \xi, \theta) \; \pi(\xi \mid \theta) \; \pi(\theta) \tag{7} \]

where \(\pi(\theta) = \prod_{i=1}^{\dim(\theta)} \pi(\theta_i)\) under independent priors.

Step 2: Use Conditional Independence and Markov Assumption

Thus:

\[ \pi(y \mid \xi, \theta) = \prod_{t=1}^T \pi(y_t \mid \xi_t, \theta) \] \[ \pi(\xi \mid \theta) = \pi(\xi_1 \mid \theta) \prod_{t=2}^T \pi(\xi_t \mid \xi_{t-1}, \theta) \]

Substitute into (7):

\[ \pi(\theta, \xi \mid y) \propto \left[ \prod_{t=1}^T \pi(y_t \mid \xi_t, \theta) \right] \, \pi(\xi_1 \mid \theta) \, \prod_{t=2}^T \pi(\xi_t \mid \xi_{t-1}, \theta) \, \pi(\theta) \tag{8} \]

Step 3: Insert Gaussian Densities from (5) and (6)

The observation equation (5):

\[ y_t = z_t \beta + \xi_t + \varepsilon_t, \quad \varepsilon_t \stackrel{\text{i.i.d.}}{\sim} N(0, \sigma_\varepsilon^2 I_d) \]

Hence:

\[ \pi(y_t \mid \xi_t, \theta) \propto (\sigma_\varepsilon^2)^{-d/2} \exp\left[ -\frac{1}{2\sigma_\varepsilon^2} (y_t - z_t\beta - \xi_t)'(y_t - z_t\beta - \xi_t) \right] \]

The state equation (6):

\[ \xi_t = a \xi_{t-1} + \omega_t, \quad \omega_t \stackrel{\text{i.i.d.}}{\sim} N(0, \sigma_\omega^2 \tilde{\Sigma}) \]

For \(t \ge 2\):

\[ \pi(\xi_t \mid \xi_{t-1}, \theta) \propto (\sigma_\omega^2)^{-d/2} |\tilde{\Sigma}|^{-1/2} \exp\left[ -\frac{1}{2\sigma_\omega^2} (\xi_t - a\xi_{t-1})' \tilde{\Sigma}^{-1} (\xi_t - a\xi_{t-1}) \right] \]

Initial condition (\(t=1\)):

\[ \xi_1 \sim N\left(0, \frac{\sigma_\omega^2}{1-a^2} \tilde{\Sigma}\right) \]

Thus:

\[ \pi(\xi_1 \mid \theta) \propto (\sigma_\omega^2)^{-d/2} (1-a^2)^{d/2} |\tilde{\Sigma}|^{-1/2} \exp\left[ -\frac{1-a^2}{2\sigma_\omega^2} \xi_1' \tilde{\Sigma}^{-1} \xi_1 \right] \]

Step 4: Multiply All Components

4.1 Likelihood product

\[ \prod_{t=1}^T \pi(y_t \mid \xi_t, \theta) \propto (\sigma_\varepsilon^2)^{-dT/2} \exp\left[ -\frac{1}{2\sigma_\varepsilon^2} \sum_{t=1}^T (y_t - z_t\beta - \xi_t)'(y_t - z_t\beta - \xi_t) \right] \]

4.2 Initial state

\[ \pi(\xi_1 \mid \theta) \propto (\sigma_\omega^2)^{-d/2} (1-a^2)^{d/2} |\tilde{\Sigma}|^{-1/2} \exp\left[ -\frac{1-a^2}{2\sigma_\omega^2} \xi_1' \tilde{\Sigma}^{-1} \xi_1 \right] \]

4.3 Transitions (\(t=2\) to \(T\))

\[ \prod_{t=2}^T \pi(\xi_t \mid \xi_{t-1}, \theta) \propto (\sigma_\omega^2)^{-d(T-1)/2} |\tilde{\Sigma}|^{-(T-1)/2} \exp\left[ -\frac{1}{2\sigma_\omega^2} \sum_{t=2}^T (\xi_t - a\xi_{t-1})' \tilde{\Sigma}^{-1} (\xi_t - a\xi_{t-1}) \right] \]

Step 5: Combine into Final Expression

Gather powers of \(\sigma_\varepsilon^2\):

\[ (\sigma_\varepsilon^2)^{-dT/2} \]

Gather powers of \(\sigma_\omega^2\):

\[ (\sigma_\omega^2)^{-d/2} \cdot (\sigma_\omega^2)^{-d(T-1)/2} = (\sigma_\omega^2)^{-dT/2} \]

Gather powers of \(|\tilde{\Sigma}|\):

\[ |\tilde{\Sigma}|^{-1/2} \cdot |\tilde{\Sigma}|^{-(T-1)/2} = |\tilde{\Sigma}|^{-T/2} \]

Include \((1-a^2)^{d/2}\) from the initial density.

Collect exponential terms:

\[ \exp\left[ -\frac{1}{2\sigma_\varepsilon^2} \sum_{t=1}^T (y_t - z_t\beta - \xi_t)'(y_t - z_t\beta - \xi_t) - \frac{1-a^2}{2\sigma_\omega^2} \xi_1' \tilde{\Sigma}^{-1} \xi_1 - \frac{1}{2\sigma_\omega^2} \sum_{t=2}^T (\xi_t - a\xi_{t-1})' \tilde{\Sigma}^{-1} (\xi_t - a\xi_{t-1}) \right] \]

Finally, multiply by the prior product:

\[ \prod_{i=1}^{\dim(\theta)} \pi(\theta_i) \]

Step 6: Final Equation (as in the paper)

\[ \begin{aligned} \pi(\theta, \xi \mid y) \propto & \; (\sigma_\varepsilon^2)^{-dT/2} \exp\left[ -\frac{1}{2\sigma_\varepsilon^2} \sum_{t=1}^T (y_t - z_t\beta - \xi_t)'(y_t - z_t\beta - \xi_t) \right] \\ & \times (\sigma_\omega^2)^{-dT/2} (1-a^2)^{d/2} |\tilde{\Sigma}|^{-T/2} \\ & \times \exp\left[ -\frac{1-a^2}{2\sigma_\omega^2} \xi_1' \tilde{\Sigma}^{-1} \xi_1 - \frac{1}{2\sigma_\omega^2} \sum_{t=2}^T (\xi_t - a\xi_{t-1})' \tilde{\Sigma}^{-1} (\xi_t - a\xi_{t-1}) \right] \\ & \times \prod_{i=1}^{\dim(\theta)} \pi(\theta_i) \end{aligned} \]