Find the matrices \(P^2\), \(P^3\), \(P^4\), and \(P^n\) for the Markov chain determined by the transition matrix \[ P=\begin{bmatrix} 1&0\\ 0&1 \end{bmatrix} \] Do the same for the transition matrix \[ P=\begin{bmatrix} 0&1\\ 1&0 \end{bmatrix} \] Interpret what happens in each of these processes.

I’m going to start with the second because it’s more interesting and gives more information.

Finding the first four iterations:

 b<-matrix(c(0,1,1,0),nrow=2)
b
##      [,1] [,2]
## [1,]    0    1
## [2,]    1    0
btwo<-b%*%b
btwo
##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1
bthree<-btwo%*%b
bthree
##      [,1] [,2]
## [1,]    0    1
## [2,]    1    0
bfour<-bthree%*%b
bfour
##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1

We can see that the values keep flipping back and forth, with even exponents giving the identity matrix and odd ones giving the original matrix. This will be true of any matrix which, when multiplied by itself, results in the identity matrix.

This will move from the first state to the second every time it is on an odd step and stay in the same state if it is on an even step.

The identity matrix is kind of boring, as it just keeps things in the same state and does not alter the probabilities.