MSDS Spring 2018

DATA 605 Fundamental of Computational Mathematics

Jiadi Li

Week 10 Discussion: Pg.422 Markov Chains Ex.4

Ex.4 Find the fundamental matrix N for Example 11.10.

1. print the original matrix

matrix11.10 <- matrix(c(1,0,0,0.5,0.5,0,0,1,0),byrow = TRUE,nrow = 3)
matrix11.10

##      [,1] [,2] [,3]
## [1,]  1.0  0.0    0
## [2,]  0.5  0.5    0
## [3,]  0.0  1.0    0

2. switch rows to get a form such that \(P = \begin{bmatrix}R & Q \\I & O \\\end{bmatrix}\)

matrix.rewrite <- matrix11.10[c(2,1,3),]
matrix.rewrite

##      [,1] [,2] [,3]
## [1,]  0.5  0.5    0
## [2,]  1.0  0.0    0
## [3,]  0.0  1.0    0

3. identify each separated matrix

matrixQ <- matrix.rewrite[c(-1,-2),c(-2,-3)]
matrixQ

## [1] 0

matrixR <- matrix.rewrite[c(-2,-3),c(-3)]
matrixR

## [1] 0.5 0.5

4. compute \(N = (I-Q)^{-1}\)

N <- solve(diag(1)-matrixQ)
N

##      [,1]
## [1,]    1

For an absorbing Markov chain \(P\), the matrix \(N = (I-Q)^{-1}\) is called the \(fundamental matrix\) for \(P\).