1. In the Land of Oz example (Example 11.1), change the transition matrix by making R an absorbing state. This gives \[ P = \left(\begin{array}{} & \fbox{R} & \fbox{N} & \fbox{S} \\ \fbox{R} & 1 & 0 & 0 \\ \fbox{N} & 1/2 & 0 & 1/2 \\ \fbox{S} & 1/4 & 1/4 & 1/2 \end{array} \right) \]

Find the fundamental matrix N, and also Nc and NR. Interpret the results.

Solution:

\(P\) in canonical form:

\[P = \left[\begin{array}{c|c} Q & R \\ \hline{0} & I \end{array} \right] = \left(\begin{array}{ccc|c} & \fbox{N} & \fbox{S} & \fbox{R} \\ \fbox{N} & 0 & 1/2 & 1/2 \\ \fbox{S} & 1/4 & 1/2 & 1/4 \\ \fbox{R} & 0 & 0 & \hline{1} \end{array} \right)\]

We can see that

\[\begin{align} Q &= \begin{pmatrix} 0 & 1/2 \\ 1/4 & 1/2 \end{pmatrix}\\ \\ I &= \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \\ \\ R &= \begin{pmatrix} 1/2 \\ 1/4 \end{pmatrix} \\ \\ N &= (I - Q)^{-1} \\ \\ c &= \begin{pmatrix} 1\\ 1 \end{pmatrix} \end{align}\]

Calculating \(N\) using R

Q <- matrix(c( 0, 0.5, 
               0.25, 0.5), ncol = 2, byrow = T)

I <- diag(2)
I_Q <-  I - Q
N <- solve( I_Q )

print(round(N,4))
##        [,1]   [,2]
## [1,] 1.3333 1.3333
## [2,] 0.6667 2.6667

Interpretation of \(N\):

Given that R (rainy day) is an absorbing state,

(1.) the expected number of nice days (state N) before the rain starts (state R) is 0.6667, if today is a snowy day (state S).

(2.) the expected number of snowy days (state S) before the rain starts (state R) is 2.6667 if it is a nice day today (state N).



Calculating \(Nc\) using R:

c <- matrix(c( 1, 1 ), ncol = 1, byrow = T)

Nc <- N %*% c

print(round(Nc, 4))
##        [,1]
## [1,] 2.6667
## [2,] 3.3333

Interpretation of \(Nc\):

(1.) Starting in state N (nice day), the expected number of days until the next rainy day is 2.6667.

(2.) Starting in state S (snowy day), the expected number of days until the next rainy day is 3.3333.



Calculating \(NR\) using R:

R <- matrix(c( 0.5, 0.25 ), ncol = 1, byrow = T)

NR <- N %*% R

print(round(NR))
##      [,1]
## [1,]    1
## [2,]    1

Interpretation of \(NR\):

(1.) Starting in state N (nice day), there is a probability of 1 of reaching a rainy day (state R).

(2.) Starting in state S (snowy day), there is a probability of 1 of reaching a rainy day (state R).