Question 7, Page 414

Find the matrices P2, P3, P4, and Pn for the Markov chain determined by the transition matrix P =(1 0 0 1). Do the same for the transition matrix P =(0 1 1 0). Interpret what happens in each of these processes.

Answer

#library for matrix exponential
library(expm)

## Loading required package: Matrix

## 
## Attaching package: 'expm'

## The following object is masked from 'package:Matrix':
## 
##     expm

#create the transition matrix 
p <- matrix(c(1,0,0,1), nrow=2)
p

##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1

Rows represents input and columns represents output. Row 1 input gives the 1 and 0 outputs. This means something in state row 1 to transition to column 1 probability 1.

\(P^{2}, P^{2}, P^{3}....P^{n}\)

All probabilities in each row adds up to 1. We are looking for matrices in each state. (from first to nth state matrices.)

#start with P 2nd state and go to p n state.
p%^%2

##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1

p%^%3

##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1

p%^%4

##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1

p%^%5

##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1

It is always the same as p. We are multiplying the identity matrix with itself and it will always be the same.

#create the second transition matrix
p_2 <- matrix(c(0,1,1,0), nrow = 2)
p_2

##      [,1] [,2]
## [1,]    0    1
## [2,]    1    0

#start with P_2 2nd state and go to p_2 to nth state. 

p_2%^%2

##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1

p_2%^%3

##      [,1] [,2]
## [1,]    0    1
## [2,]    1    0

p_2%^%4

##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1

p_2%^%5

##      [,1] [,2]
## [1,]    0    1
## [2,]    1    0

We are seeing, at each state, the matrix is either itself or the identity matrix.