Exercise 7

Find the matrices \(P^2, P^3, P^4\), and \(P^n\) for the Markov chain determined by the transition matrix \(P = \begin{pmatrix}1 & 0\ \\0 & 1 \end{pmatrix}\).

Do the same for the transition matrix \(P = \begin{pmatrix}0 & 1\ \\1 & 0 \end{pmatrix}\). Interpret what happens in each of these processes.

Exercise 7 Solution

# Define the first transition matrix in the problem.
transition_matrix_one <- matrix(c(1, 0, 0, 1), 2)

# Define the second transition matrix in the problem.
transition_matrix_two <- matrix(c(0, 1, 1, 0), 2)

Find matrices \(P^2\) through \(P^n\) for the first transition matrix.

Find the matrix \(P^2\).

P2 <- transition_matrix_one%^%2
P2
##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1

Find the matrix \(P^3\).

P3 <- transition_matrix_one%^%3
P3
##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1

Find the matrix \(P^4\).

P4 <- transition_matrix_one%^%4
P4
##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1

Find the matrix \(P^n\).

Pn <- transition_matrix_one%^%5
Pn
##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1

Observations

When \(P = \begin{pmatrix}1 & 0\ \\0 & 1 \end{pmatrix}\), then \(P^n\) is always equal to \(\begin{pmatrix}1 & 0\ \\0 & 1 \end{pmatrix}\) no matter how many times we multiply the matrix by itself.

The below code proves this observation. In this example, we multiply the matrix by itself 500 times and we still get the same result.

# Multiply the matrix P 500 times by itself, to prove that we always get the same result
# no matter how many times we multiply the matrix by itself.
Pn <- transition_matrix_one%^%500
Pn
##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1

Find matrices \(P^2\) through \(P^n\) for the second transition matrix.

Find the matrix \(P^2\).

P2 <- transition_matrix_two%^%2
P2
##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1

Find the matrix \(P^3\).

P3 <- transition_matrix_two%^%3
P3
##      [,1] [,2]
## [1,]    0    1
## [2,]    1    0

Find the matrix \(P^4\).

P4 <- transition_matrix_two%^%4
P4
##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1

Find the matrix \(P^n\).

Pn <- transition_matrix_two%^%5
Pn
##      [,1] [,2]
## [1,]    0    1
## [2,]    1    0

Observations

When \(P = \begin{pmatrix}0 & 1\ \\1 & 0 \end{pmatrix}\) and \(n\) is an even number, then \(P^n\) is equal to \(\begin{pmatrix}1 & 0\ \\0 & 1 \end{pmatrix}\).

When \(n\) is an odd number, then \(P^n\) is equal to \(\begin{pmatrix}0 & 1\ \\1 & 0 \end{pmatrix}\).

The below examples prove that no matter how many times we multiply the matrix by itself, this observation holds true.

# Multiply the matrix P 500 times by itself, to prove that we always
# get the same result when n is an even number.
Pn_even <- transition_matrix_two%^%500
Pn_even
##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1
# Multiply the matrix P 501 times by itself, to prove that we always
# get the same result when n is an odd number.
Pn_odd <- transition_matrix_two%^%501
Pn_odd
##      [,1] [,2]
## [1,]    0    1
## [2,]    1    0