Introduction

This document provides a comprehensive, step-by-step explanation of weak stationarity (also called second-order stationarity) for a spatial or temporal process. We will use the example of a random phase sine process:

\[Y_t = X \sin(\omega t + \Theta)\]

where:

  • \(X\) is a random variable with \(\mathbb{E}[X] = 0\) and \(\text{Var}(X) = 1\) (so \(\mathbb{E}[X^2] = 1\))
  • \(\Theta \sim U(-\pi, \pi)\) is a uniformly distributed random phase
  • \(X\) and \(\Theta\) are independent
  • \(\omega\) is a fixed frequency

Our goal is to prove that \(Y_t\) is weakly stationary.

Part 1: What is Weak Stationarity?

A process \(Y_t\) is called weakly stationary (or second-order stationary) if it satisfies two conditions:

Condition 1: Constant Mean

The mean function does not depend on time \(t\):

\[\mathbb{E}[Y_t] = \mu \text{ for all } t\]

Condition 2: Lag-Dependent Covariance

The covariance between two points depends only on the time lag \(h\), not on the absolute time \(t\):

\[\text{Cov}(Y_t, Y_{t+h}) = C(h) \text{ for all } t \text{ and } h\]

where \(C(h)\) is a function that depends only on the lag \(h\).

Part 2: Understanding the Random Phase

2.1 The Uniform Distribution

Since \(\Theta \sim U(-\pi, \pi)\), its probability density function (PDF) is:

\[f_\Theta(\theta) = \frac{1}{2\pi} \text{ for } -\pi \leq \theta \leq \pi\]

And \(f_\Theta(\theta) = 0\) elsewhere.

This means that \(\Theta\) is equally likely to take any value between \(-\pi\) and \(\pi\). The interval length is \(2\pi\), so the PDF must be \(1/(2\pi)\) to ensure the total probability integrates to 1:

\[\int_{-\pi}^{\pi} \frac{1}{2\pi} \, d\theta = \frac{1}{2\pi} \cdot [\theta]_{-\pi}^{\pi} = \frac{1}{2\pi} \cdot 2\pi = 1\]

2.2 The General Expectation Formula

For any function \(g(\Theta)\) of the random variable \(\Theta\), the expected value is:

\[\mathbb{E}[g(\Theta)] = \int_{-\infty}^{\infty} g(\theta) f_\Theta(\theta) \, d\theta = \frac{1}{2\pi} \int_{-\pi}^{\pi} g(\theta) \, d\theta\]

This is the fundamental formula we will use repeatedly.

Part 3: Proving the Mean is Constant

We need to show that \(\mathbb{E}[Y_t] = \mu\) for all \(t\), where \(\mu\) is a constant.

Step 3.1: Write the Expectation

\[\mathbb{E}[Y_t] = \mathbb{E}[X \sin(\omega t + \Theta)]\]

Step 3.2: Use Independence

Since \(X\) and \(\Theta\) are independent:

\[\mathbb{E}[Y_t] = \mathbb{E}[X] \cdot \mathbb{E}[\sin(\omega t + \Theta)]\]

Given that \(\mathbb{E}[X] = 0\), we have:

\[\mathbb{E}[Y_t] = 0 \cdot \mathbb{E}[\sin(\omega t + \Theta)] = 0\]

Step 3.3: Verify the Sine Expectation

For completeness, let’s verify that \(\mathbb{E}[\sin(\omega t + \Theta)] = 0\):

\[\mathbb{E}[\sin(\omega t + \Theta)] = \frac{1}{2\pi} \int_{-\pi}^{\pi} \sin(\omega t + \theta) \, d\theta\]

Let \(\alpha = \omega t\) (a constant for fixed \(t\)):

\[\mathbb{E}[\sin(\alpha + \Theta)] = \frac{1}{2\pi} \int_{-\pi}^{\pi} \sin(\alpha + \theta) \, d\theta\]

Use u-substitution: Let \(u = \alpha + \theta\), so \(du = d\theta\). When \(\theta = -\pi\), \(u = \alpha - \pi\); when \(\theta = \pi\), \(u = \alpha + \pi\):

\[\mathbb{E}[\sin(\alpha + \Theta)] = \frac{1}{2\pi} \int_{\alpha - \pi}^{\alpha + \pi} \sin(u) \, du\]

Evaluate the integral:

\[= \frac{1}{2\pi} [-\cos(u)]_{\alpha - \pi}^{\alpha + \pi}\]

\[= \frac{1}{2\pi} (-\cos(\alpha + \pi) + \cos(\alpha - \pi))\]

Using \(\cos(\alpha + \pi) = -\cos(\alpha)\) and \(\cos(\alpha - \pi) = -\cos(\alpha)\):

\[= \frac{1}{2\pi} (-[-\cos(\alpha)] + [-\cos(\alpha)])\]

\[= \frac{1}{2\pi} (\cos(\alpha) - \cos(\alpha)) = 0\]

Therefore: \(\mathbb{E}[Y_t] = 0\) for all \(t\). This is a constant mean.

Part 4: Proving the Covariance Depends Only on the Lag

We need to show that \(\text{Cov}(Y_t, Y_{t+h})\) depends only on \(h\), not on \(t\).

Step 4.1: Write the Covariance Formula

\[\text{Cov}(Y_t, Y_{t+h}) = \mathbb{E}[Y_t Y_{t+h}] - \mathbb{E}[Y_t]\mathbb{E}[Y_{t+h}]\]

Since we already proved that \(\mathbb{E}[Y_t] = 0\) for all \(t\), the second term is zero:

\[\text{Cov}(Y_t, Y_{t+h}) = \mathbb{E}[Y_t Y_{t+h}]\]

Step 4.2: Expand the Product

\[\mathbb{E}[Y_t Y_{t+h}] = \mathbb{E}[X \sin(\omega t + \Theta) \cdot X \sin(\omega(t + h) + \Theta)]\]

\[= \mathbb{E}[X^2 \cdot \sin(\omega t + \Theta) \sin(\omega(t + h) + \Theta)]\]

Step 4.3: Use Independence

Since \(X\) and \(\Theta\) are independent:

\[\mathbb{E}[Y_t Y_{t+h}] = \mathbb{E}[X^2] \cdot \mathbb{E}[\sin(\omega t + \Theta) \sin(\omega(t + h) + \Theta)]\]

Given that \(\mathbb{E}[X^2] = 1\) (since \(\text{Var}(X) = 1\) and \(\mathbb{E}[X] = 0\)):

\[\mathbb{E}[Y_t Y_{t+h}] = \mathbb{E}[\sin(\omega t + \Theta) \sin(\omega(t + h) + \Theta)]\]

Step 4.4: Evaluate the Sine Product Expectation

We need to compute:

\[\mathbb{E}[\sin(\omega t + \Theta) \sin(\omega(t + h) + \Theta)] = \frac{1}{2\pi} \int_{-\pi}^{\pi} \sin(\omega t + \theta) \sin(\omega(t + h) + \theta) \, d\theta\]

Use the Product-to-Sum Identity

Recall the trigonometric identity:

\[\sin A \sin B = \frac{1}{2}[\cos(A - B) - \cos(A + B)]\]

Here:

  • \(A = \omega t + \theta\)
  • \(B = \omega(t + h) + \theta = \omega t + \omega h + \theta\)

So:

\[\sin(\omega t + \theta) \sin(\omega t + \omega h + \theta) = \frac{1}{2}[\cos(\omega t + \theta - \omega t - \omega h - \theta) - \cos(\omega t + \theta + \omega t + \omega h + \theta)]\]

\[= \frac{1}{2}[\cos(-\omega h) - \cos(2\omega t + \omega h + 2\theta)]\]

Since \(\cos(-x) = \cos(x)\):

\[= \frac{1}{2}[\cos(\omega h) - \cos(2\omega t + \omega h + 2\theta)]\]

Substitute Back into the Integral

\[\mathbb{E}[\sin(\omega t + \Theta) \sin(\omega(t + h) + \Theta)] = \frac{1}{2\pi} \int_{-\pi}^{\pi} \frac{1}{2}[\cos(\omega h) - \cos(2\omega t + \omega h + 2\theta)] \, d\theta\]

\[= \frac{1}{4\pi} \int_{-\pi}^{\pi} \cos(\omega h) \, d\theta - \frac{1}{4\pi} \int_{-\pi}^{\pi} \cos(2\omega t + \omega h + 2\theta) \, d\theta\]

Evaluate the First Integral

Since \(\cos(\omega h)\) does not depend on \(\theta\):

\[\int_{-\pi}^{\pi} \cos(\omega h) \, d\theta = \cos(\omega h) \cdot [\theta]_{-\pi}^{\pi} = \cos(\omega h) \cdot 2\pi\]

So the first part becomes:

\[\frac{1}{4\pi} \cdot 2\pi \cos(\omega h) = \frac{1}{2} \cos(\omega h)\]

Evaluate the Second Integral

Let \(u = 2\omega t + \omega h + 2\theta\), so \(du = 2 \, d\theta\) and \(d\theta = du/2\).

When \(\theta = -\pi\), \(u = 2\omega t + \omega h - 2\pi\). When \(\theta = \pi\), \(u = 2\omega t + \omega h + 2\pi\).

So:

\[\int_{-\pi}^{\pi} \cos(2\omega t + \omega h + 2\theta) \, d\theta = \frac{1}{2} \int_{2\omega t + \omega h - 2\pi}^{2\omega t + \omega h + 2\pi} \cos(u) \, du\]

\[= \frac{1}{2} [\sin(u)]_{2\omega t + \omega h - 2\pi}^{2\omega t + \omega h + 2\pi}\]

\[= \frac{1}{2}[\sin(2\omega t + \omega h + 2\pi) - \sin(2\omega t + \omega h - 2\pi)]\]

Since \(\sin(x + 2\pi) = \sin(x)\) and \(\sin(x - 2\pi) = \sin(x)\):

\[= \frac{1}{2}[\sin(2\omega t + \omega h) - \sin(2\omega t + \omega h)] = 0\]

Combine the Results

\[\mathbb{E}[\sin(\omega t + \Theta) \sin(\omega(t + h) + \Theta)] = \frac{1}{2} \cos(\omega h) - 0 = \frac{1}{2} \cos(\omega h)\]

Step 4.5: Final Covariance Result

\[\text{Cov}(Y_t, Y_{t+h}) = \mathbb{E}[Y_t Y_{t+h}] = \frac{1}{2} \cos(\omega h)\]

Notice: This depends only on \(h\), the time lag, and not on \(t\). The \(t\) completely canceled out!

Part 5: Conclusion

We have verified both conditions for weak stationarity:

1. Constant Mean

\[\mathbb{E}[Y_t] = 0 \text{ for all } t\]

2. Lag-Dependent Covariance

\[\text{Cov}(Y_t, Y_{t+h}) = \frac{1}{2} \cos(\omega h) \text{ for all } t \text{ and } h\]

Therefore, the process \(Y_t = X \sin(\omega t + \Theta)\) is weakly stationary.

Part 6: Important Clarifications

6.1 The Random Variable is \(\Theta\), Not \(t\)

When we write:

\[\mathbb{E}[\sin(\omega t + \Theta)] = \frac{1}{2\pi} \int_{-\pi}^{\pi} \sin(\omega t + \theta) \, d\theta\]

  • \(\Theta\) is the random variable (we’re averaging over all possible phases)
  • \(\omega t\) is a constant (for fixed \(t\))
  • We are integrating with respect to \(\theta\), which is the variable of integration

6.2 Treating the Product as One Function

We can define:

\[g(\Theta) = \sin(\omega t + \Theta) \sin(\omega(t + h) + \Theta)\]

Then:

\[\mathbb{E}[g(\Theta)] = \frac{1}{2\pi} \int_{-\pi}^{\pi} g(\theta) \, d\theta\]

This is valid because \(t\) and \(h\) are just parameters. The expectation is still over \(\Theta\).

6.3 The Intuition

The result \(\frac{1}{2} \cos(\omega h)\) comes from two parts:

  • The persistent part: \(\frac{1}{2} \cos(\omega h)\) comes from \(A - B\), which cancels out both \(\theta\) and \(\omega t\). This depends only on the lag \(h\).
  • The oscillating part: \(\frac{1}{2} \cos(2\omega t + \omega h + 2\theta)\) averages to zero because \(\theta\) ranges over a full cycle \((2\pi)\), making the sine wave complete exactly 2 full periods.

Part 7: Visual Demonstration

Let’s visualize a realization of this process to see its stationary behavior.

# Set parameters
set.seed(123)
omega <- 2
t <- seq(0, 10, length.out = 1000)

# Generate one realization
X <- rnorm(1, mean = 0, sd = 1)
theta <- runif(1, -pi, pi)
Y <- X * sin(omega * t + theta)

# Plot
plot(t, Y, type = "l", col = "blue", lwd = 2,
     main = "One Realization of Y_t = X sin(ωt + Θ)",
     xlab = "Time (t)", ylab = "Y_t")
abline(h = 0, col = "red", lty = 2)
grid()

The mean is zero (red dashed line), and the process oscillates around it.

Part 8: Summary Table

Property Mathematical Expression Depends On
Mean \(\mathbb{E}[Y_t] = 0\) Nothing (constant)
Variance \(\text{Var}(Y_t) = \frac{1}{2}\) Nothing (constant)
Autocovariance \(\text{Cov}(Y_t, Y_{t+h}) = \frac{1}{2} \cos(\omega h)\) Only the lag \(h\)

All conditions for weak stationarity are satisfied!