Taking on exercise 22 at the end of chapter 11 (page 469):

Problem:

Show that if \(P\) is the transition matrix of a regular Markov chain, and \(W\) is the matrix each of whose rows is the fixed probability vector corresponding to \(P\), then \(PW = W\), and \(W^k = W\) for all positive integers \(k\).

Solution:

First, to define the transition matrix \(P\) of a regular Markov chain:

P <- matrix(c(0.5, 0.5, 0.2, 0.8), nrow = 2, byrow = TRUE)

Then to calculate \(\pi\) that satisfies \(\pi P = \pi\) and \(\sum \pi_i = 1\), where \(\pi\) represents the stationary distribution of a Markov chain.:

eigen_result <- eigen(t(P))
pi_index <- which.max(Re(eigen_result$values))
pi <- eigen_result$vectors[,pi_index] / sum(eigen_result$vectors[,pi_index])

Now to create matrix \(W\) with each row being the stationary distribution \(\pi\):

W <- matrix(rep(pi, times = nrow(P)), nrow = nrow(P), byrow = TRUE)

Finally, to verify \(PW = W\):

PW <- P %*% W
all.equal(PW, W)
## [1] TRUE

Validate \(W^k = W\) for all positive integers \(k\):

k <- 5 # just an example value for k
Wk <- W
for (i in 2:k) {
  Wk <- Wk %*% W
}
all.equal(Wk, W)
## [1] TRUE