The Gaussian Conditional Autoregressive (CAR) model is defined through the full conditional distributions:
\[ Y_i \mid Y_j, j \neq i \sim N\left(\sum_j b_{ij} Y_j, \tau_i^2\right), \quad i = 1, \dots, n. \]
The joint distribution is multivariate normal:
\[ Y \sim N(0, \Sigma_y), \quad \text{where } \Sigma_y^{-1} = D^{-1}(I - B), \]
with \(B = \{b_{ij}\}\) and \(D = \text{diag}(\tau_1^2, \dots, \tau_n^2)\).
We aim to prove the following result:
Theorem: If \((\Sigma_y^{-1})_{ij} = 0\), then \(Y_i\) and \(Y_j\) are conditionally independent given \(Y_k\) for all \(k \neq i, j\).
For a multivariate normal random vector \(Y \sim N(\mu, \Sigma)\), the conditional independence structure is encoded in the precision matrix \(\Theta = \Sigma^{-1}\):
\[ \boxed{Y_i \perp Y_j \mid Y_{-ij} \iff (\Sigma^{-1})_{ij} = 0} \]
where \(Y_{-ij} = \{Y_k : k \neq i, j\}\).
We will prove this theorem using partitioned matrix theory.
Let \(Y = (Y_1, \dots, Y_n)^T\) be partitioned as:
\[ Y = \begin{pmatrix} Y_1 \\ Y_2 \end{pmatrix} \]
where:
Partition the covariance matrix \(\Sigma\) conformably:
\[ \Sigma = \begin{pmatrix} \Sigma_{11} & \Sigma_{12} \\ \Sigma_{21} & \Sigma_{22} \end{pmatrix} \]
where:
Similarly, partition the precision matrix \(\Theta = \Sigma^{-1}\):
\[ \Theta = \begin{pmatrix} \Theta_{11} & \Theta_{12} \\ \Theta_{21} & \Theta_{22} \end{pmatrix} \]
For multivariate normal distributions, the conditional distribution is:
\[ Y_1 \mid Y_2 = y_2 \sim N\left(\mu_1 + \Sigma_{12}\Sigma_{22}^{-1}(y_2 - \mu_2), \; \Sigma_{11} - \Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}\right) \]
Thus, the conditional covariance matrix is:
\[ \boxed{\text{Cov}(Y_1 \mid Y_2) = \Sigma_{11} - \Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}} \]
The precision matrix of the conditional distribution is the inverse of the conditional covariance:
\[ \boxed{\text{Precision}(Y_1 \mid Y_2) = \left(\Sigma_{11} - \Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}\right)^{-1}} \]
For a symmetric positive definite matrix partitioned as:
\[ \Sigma = \begin{pmatrix} A & B \\ B^T & D \end{pmatrix} \]
the inverse is given by:
\[ \Sigma^{-1} = \begin{pmatrix} (A - BD^{-1}B^T)^{-1} & -A^{-1}B(D - B^T A^{-1}B)^{-1} \\ -(D - B^T A^{-1}B)^{-1}B^T A^{-1} & (D - B^T A^{-1}B)^{-1} \end{pmatrix} \]
Therefore, the upper-left block of \(\Theta = \Sigma^{-1}\) is:
\[ \boxed{\Theta_{11} = (A - BD^{-1}B^T)^{-1}} \]
where \(A = \Sigma_{11}\), \(B = \Sigma_{12}\), and \(D = \Sigma_{22}\).
Comparing Step 4 and Step 5, we have:
\[ \boxed{\text{Precision}(Y_1 \mid Y_2) = \Theta_{11}} \]
That is, the precision matrix of the conditional distribution \(Y_1 \mid Y_2\) is exactly the upper-left \(2 \times 2\) block of the full precision matrix \(\Theta\).
Since \(Y_1 = (Y_i, Y_j)^T\), we can write:
\[ \Theta_{11} = \begin{pmatrix} \theta_{ii} & \theta_{ij} \\ \theta_{ij} & \theta_{jj} \end{pmatrix} = \begin{pmatrix} (\Sigma^{-1})_{ii} & (\Sigma^{-1})_{ij} \\ (\Sigma^{-1})_{ij} & (\Sigma^{-1})_{jj} \end{pmatrix} \]
Therefore:
\[ \boxed{ \text{Precision}(Y_i, Y_j \mid Y_{-ij}) = \begin{pmatrix} (\Sigma^{-1})_{ii} & (\Sigma^{-1})_{ij} \\ (\Sigma^{-1})_{ij} & (\Sigma^{-1})_{jj} \end{pmatrix} } \]
Then the conditional precision matrix becomes:
\[ \text{Precision}(Y_i, Y_j \mid Y_{-ij}) = \begin{pmatrix} (\Sigma^{-1})_{ii} & 0 \\ 0 & (\Sigma^{-1})_{jj} \end{pmatrix} \]
This is a diagonal matrix. Therefore, the conditional covariance matrix is also diagonal:
\[ \text{Cov}(Y_i, Y_j \mid Y_{-ij}) = \begin{pmatrix} 1/(\Sigma^{-1})_{ii} & 0 \\ 0 & 1/(\Sigma^{-1})_{jj} \end{pmatrix} \]
Since the off-diagonal element is zero:
\[ \text{Cov}(Y_i, Y_j \mid Y_{-ij}) = 0 \]
A zero off-diagonal entry in the precision matrix does not mean the variance doesn’t exist. Let’s clarify:
| Entry Type | Value | Meaning |
|---|---|---|
| Off-diagonal \((\Sigma^{-1})_{ij} = 0\) | Zero | Conditional independence (variances exist) |
| Diagonal \((\Sigma^{-1})_{ii} = 0\) | Zero | Infinite conditional variance (improper distribution) |
The diagonal entries \((\Sigma^{-1})_{ii}\) and \((\Sigma^{-1})_{jj}\) remain positive, so the conditional variances exist and are finite:
\[ \text{Var}(Y_i \mid Y_{-ij}) = \frac{1}{(\Sigma^{-1})_{ii}} < \infty \]
The case of a zero diagonal entry occurs in the intrinsic autoregressive (IAR) model:
\[ \Sigma_y^{-1} = \frac{1}{\tau^2}(D_w - W) \]
where \((D_w - W)\mathbf{1} = 0\), making the precision matrix singular and the distribution improper.
In the CAR model from Subsection 4.3.1:
\[ \Sigma_y^{-1} = D^{-1}(I - B) \]
where:
Therefore, for \(i \neq j\):
\[ (\Sigma_y^{-1})_{ij} = -\frac{b_{ij}}{\tau_i^2} \]
Thus:
\[ (\Sigma_y^{-1})_{ij} = 0 \iff b_{ij} = 0 \]
This means that if \(b_{ij} = 0\), then \(Y_i\) does not appear in the full conditional mean of \(Y_j\), and vice versa (due to the symmetry condition \(b_{ij}/\tau_i^2 = b_{ji}/\tau_j^2\)).
This is precisely the local Markov property: variables are conditionally independent given their neighbors, and the neighbor structure is defined by the non-zero entries of \(B\) (or equivalently, the non-zero off-diagonal entries of \(\Sigma_y^{-1}\)).
# Load required library
library(MASS)
# Set seed for reproducibility
set.seed(123)
# Create a 3x3 covariance matrix with a zero precision off-diagonal
# We want (Sigma^{-1})_{12} = 0
# Choose a precision matrix with zero off-diagonal
Theta <- matrix(c(2, 0, 0.5,
0, 3, 0.3,
0.5, 0.3, 1), nrow = 3, byrow = TRUE)
# Ensure symmetry
Theta <- (Theta + t(Theta)) / 2
# Check that Theta is positive definite
eigen(Theta)$values## [1] 3.0496822 2.1918356 0.7584822# Get the covariance matrix
Sigma <- solve(Theta)
# Display matrices
cat("Precision Matrix (Theta):\n")## Precision Matrix (Theta):print(Theta)## [,1] [,2] [,3]
## [1,] 2.0 0.0 0.5
## [2,] 0.0 3.0 0.3
## [3,] 0.5 0.3 1.0cat("\nCovariance Matrix (Sigma):\n")##
## Covariance Matrix (Sigma):print(Sigma)## [,1] [,2] [,3]
## [1,] 0.5739645 0.0295858 -0.2958580
## [2,] 0.0295858 0.3451677 -0.1183432
## [3,] -0.2958580 -0.1183432 1.1834320# The off-diagonal element (1,2) of Theta is:
cat("\n(Theta)_{12} =", Theta[1,2], "\n")##
## (Theta)_{12} = 0# Compute the conditional covariance of (Y1, Y2) given Y3
Sigma_11 <- Sigma[1:2, 1:2]
Sigma_12 <- Sigma[1:2, 3, drop = FALSE]
Sigma_21 <- Sigma[3, 1:2, drop = FALSE]
Sigma_22 <- Sigma[3, 3, drop = FALSE]
cond_cov <- Sigma_11 - Sigma_12 %*% solve(Sigma_22) %*% Sigma_21
cat("\nConditional Covariance of (Y1, Y2) given Y3:\n")##
## Conditional Covariance of (Y1, Y2) given Y3:print(cond_cov)## [,1] [,2]
## [1,] 0.5 0.0000000
## [2,] 0.0 0.3333333# The off-diagonal element should be zero
cat("\nConditional covariance between Y1 and Y2 given Y3 =", cond_cov[1,2], "\n")##
## Conditional covariance between Y1 and Y2 given Y3 = 0We have proven that for a multivariate normal distribution, the upper-left \(2 \times 2\) block of the precision matrix \(\Sigma^{-1}\) is exactly the precision matrix of the conditional distribution of those two variables given all others. Consequently, a zero off-diagonal entry in the precision matrix implies zero conditional covariance, and hence conditional independence.
In the context of the Gaussian CAR model, this means that the non-zero pattern of \(B\) (or equivalently, of \(\Sigma_y^{-1}\)) defines the conditional independence structure, which is exactly the local Markov property of the spatial process.