Discussion Week 10
Julia Ferris
2024-03-28
Chapter 11 Page 467 Question 7
A rat runs through the maze shown in Figure 11.7. At each step it leaves the room it is in by choosing at random one of the doors out of the room.
knitr::include_graphics("/Users/TEMP/Downloads/screenshot.jpg")
Figure 11.7: Maze for exercise 7.
(a) Give the transition matrix P for this Markov chain.
Answer: See the transition matrix below:
P <- matrix(c(0, 0, 1, 0, 0, 0,
0, 0, 1, 0, 0, 0,
0.25, 0.25, 0, 0.25, 0.25, 0,
0, 0, 0.5, 0, 0, 0.5,
0, 0, 0.5, 0, 0, 0.5,
0, 0, 0, 0.5, 0.5, 0), nrow = 6, byrow = TRUE)
rownames(P) <- c("1", "2", "3", "4", "5", "6")
colnames(P) <- c("1", "2", "3", "4", "5", "6")
print(P)## 1 2 3 4 5 6
## 1 0.00 0.00 1.0 0.00 0.00 0.0
## 2 0.00 0.00 1.0 0.00 0.00 0.0
## 3 0.25 0.25 0.0 0.25 0.25 0.0
## 4 0.00 0.00 0.5 0.00 0.00 0.5
## 5 0.00 0.00 0.5 0.00 0.00 0.5
## 6 0.00 0.00 0.0 0.50 0.50 0.0\(\\\)
(b) Show that it is an ergodic chain but not a regular chain.
Answer:
Ergodic chain: It is possible to go from every state to every state.
Regular chain: Movements in a predictable pattern.
This is an ergodic chain because it is possible to go from every room to every other room with a finite number of steps. Also, this is not a regular chain because the rat does not move in a predictable pattern. After a certain amount of time, it will not repeat itself since the rat moves randomly.
\(\\\)
(c) Find the fixed vector.
\((w_1, w_2, w_3, w_4, w_5, w_6)\) * \(\begin{bmatrix} 0 & 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 & 0 \\ 1/4 & 1/4 & 0 & 1/4 & 1/4 & 0 \\ 0 & 0 & 1/2 & 0 & 0 & 1/2 \\ 0 & 0 & 1/2 & 0 & 0 & 1/2 \\ 0 & 0 & 0 & 1/2 & 1/2 & 0 \end{bmatrix}\) = \((w_1, w_2, w_3, w_4, w_5, w_6)\)
\(\\\)
\(w_1 + w_2 + w_3 + w_4 + w_5 + w_6 = 1\)
\(0.25(w_3)\) = \(w_1\)
\(0.25(w_3)\) = \(w_2\)
\(w_1 + w_2 + 0.5(w_4) + 0.5(w_5)\) = \(w_3\)
\(0.25(w_3) + 0.5(w_6)\) = \(w_4\)
\(0.25(w_3) + 0.5(w_6)\) = \(w_5\)
\(0.5(w_4) + 0.5(w_5)\) = \(w_6\)
C:\Users\TEMP\Downloads
We still need to find \(w_4, w_5,\) and \(w_6\) in respect to \(w_3\) . . .
\(w_1 + w_2 + 0.5(w_4) + 0.5(w_5)\) = \(w_3\) = \(0.25(w_3) + 0.25(w_3) + 0.5(w_4) + 0.5(w_5)\) = \(0.5(w_3) + 0.5(w_4) + 0.5(w_5)\)
\(0.5(w_3) = 0.5(w_4) + 0.5(w_5)\)
\(w_3 = w_4 + w_5\)
\(\\\)
Plug in what we know for \(w_4\) and \(w_5\) to get \(w_6\).
\(w_3 = 0.25(w_3) + 0.5(w_6) + 0.25(w_3) + 0.5(w_6) = 0.5(w_3) + w_6\)
$0.5(w_3) = w_6$$
\(\\\)
Solve for \(w_4\) and \(w_5\) based on the value of \(w_6\).
\(0.25(w_3) + 0.5(w_6)\) = \(w_4\) = \(0.25(w_3) + 0.5(0.5(w_3))\) = \(0.25(w_3) + 0.25(w_3)\) = \(0.5(w_3)\)
\(0.25(w_3) + 0.5(w_6)\) = \(w_5\) = \(0.25(w_3) + 0.5(w_6)\) = \(w_4\) = \(0.25(w_3) + 0.5(0.5(w_3))\) = \(0.25(w_3) + 0.25(w_3)\) = \(0.5(w_3)\)
\(\\\)
So now we know . . .
\(w_1 = 0.25(w_3)\)
\(w_2 = 0.25(w_3)\)
\(w_3 = w_3\)
\(w_4 = 0.5(w_3)\)
\(w_5 = 0.5(w_3)\)
\(w_6 = 0.5(w_3)\)
\(\\\)
Solve for \(w_3\) and then solve for the rest.
\(w_1 + w_2 + w_3 + w_4 + w_5 + w_6 = 1\)
\(0.25(w_3)\) + \(0.25(w_3)\) + \(w_3\) + \(0.5(w_3)\) + \(0.5(w_3)\) + \(0.5(w_3)\) = 1
\(3(w_3)\) = 1
\(w_3\) = 1/3
\(\\\)
We know that \(w_3\) is 1/3, so now we know all the values:
\(w_1 = 1/12\)
\(w_2 = 1/12\)
\(w_3 = 1/3\)
\(w_4 = 1/6\)
\(w_5 = 1/6\)
\(w_6 = 1/6\)
\(\\\)
Answer: The fixed vector is (1/12, 1/12, 1/3, 1/6, 1/6, 1/6).
\(\\\)
(d) Find the expected number of steps before reaching Room 5 for the first time, starting in Room 1.
library(matlib)## Warning: package 'matlib' was built under R version 4.3.3I <- diag(6)
W <- matrix(c(1/12, 1/12, 1/3, 1/6, 1/6, 1/6,
1/12, 1/12, 1/3, 1/6, 1/6, 1/6,
1/12, 1/12, 1/3, 1/6, 1/6, 1/6,
1/12, 1/12, 1/3, 1/6, 1/6, 1/6,
1/12, 1/12, 1/3, 1/6, 1/6, 1/6,
1/12, 1/12, 1/3, 1/6, 1/6, 1/6), nrow = 6, byrow = TRUE)
new1 <- I - P + W
Z = inv(new1)\(m_{1,5} = \frac{z_{5,5} - z_{1,5}} {w_5}\)
\(m_{1,5}\) = \(\frac {0.97222222 - -0.09722222} {1/6} = 7\)