Discussion 10

Prediction using Markov chain

No. 3, p. 414:

In Example 11.5, find P, P2, and P3. What is Pn?

Intuitively, you would expect the probabilities to remain the same for every state, since this in this case the odds of winning are independent of prior races. We can show this experientially using the markovchain package and following Theorem 11.2 in Grimstead.

First, make the transition matrix of probabilities:

raceNames <- c("Win","Place","Show")
raceMat <- matrix(c(.5,.25,.25,.5,.25,.25,.5,.25,.25),
             nrow=3, byrow=TRUE)
row.names(raceMat) <- raceNames; colnames(raceMat) <- raceNames
raceMat

##       Win Place Show
## Win   0.5  0.25 0.25
## Place 0.5  0.25 0.25
## Show  0.5  0.25 0.25

According to Theorem 11.2:

u⁽ⁿ⁾ = uPⁿ

Assuming u starts with equal probability (1/3, 1/3, 1/3), (although we could also start with (0.5, 0.25, 0.25) and get the same result):

First state, as expected the initial probability:

u¹ = uP¹

raceMat1 <- raceMat %^% 1
u <- c(1/3, 1/3, 1/3)
round(u%*%raceMat,3)

##      Win Place Show
## [1,] 0.5  0.25 0.25

Second state, race our transition matrix to the power of 2 and update u:

u² = uP²

raceMat2 <- raceMat %^% 2
u <- c(.5,.25,.25)
raceMat2; cat('\n'); round(u %*% raceMat2, 3)

##       Win Place Show
## Win   0.5  0.25 0.25
## Place 0.5  0.25 0.25
## Show  0.5  0.25 0.25

##      Win Place Show
## [1,] 0.5  0.25 0.25

Third state, race our transition matrix to the power of 3:

u³ = uP³

… and so it would continue to Pⁿ in an independent trials process, with the distribution of the states remaining unhanged.

What explains it? Our transition matrix has the characteristic of idempotency.

raceMat3 <- raceMat %^% 3
u <- c(.5,.25,.25)
raceMat3; cat('\n'); round(u %*% raceMat3, 3)

##       Win Place Show
## Win   0.5  0.25 0.25
## Place 0.5  0.25 0.25
## Show  0.5  0.25 0.25

##      Win Place Show
## [1,] 0.5  0.25 0.25

Here is a diagram of the transition matrix, hatip to the Revolution Analyitics blog.

plotmat(raceMat,pos = c(1,2), 
        lwd = 1, box.lwd = 2, 
        cex.txt = 0.8, 
        box.size = 0.1, 
        box.type = "circle", 
        box.prop = 0.5,
        box.col = "light yellow",
        arr.length=.1,
        arr.width=.1,
        self.cex = .4,
        self.shifty = -.01,
        self.shiftx = .13,
        main = "")