library(knitr)

Smith has a one dollar and needs total of 8 dollars to get out of the jail.

a. He bets 1 dollar each time (timid strategy).

Given data states that \(0\) and \(8\) are absorbing states.

Using Canonical form method,

Where \(Q\) gives the information about probabilities of state change and is a \(t\)X\(t\) square matrix. It is also known as Transition matrix. Matrix \(R\) gives probabilities of absorbing states. Martix \(I\) is identity matrix.

p <- 0.4     #success probability of each bet
q <- 1 -p    #failure probability of each bet
N <- 8       #amount of money required to get out of jail

#Q matrix
Q  <- matrix(NA, nrow=7, ncol=7)
for(i in 1:7){
  for(j in 1:7){
    if (i-j == 1){
      Q[i,j] <- q
    }
    else if(j-i==1){
      Q[i,j] <- p
    }
    else{
      Q[i,j] <- 0
    }
  }
}

#R matrix
R <- matrix(NA, nrow=7, ncol=2)
for(i in 1:7){
  for(j in 1:2){
    if (i == 1 & j == 1){
      R[i,j] <- q
    }
    else if(i==7 & j == 2){
      R[i,j] <- p
    }
    else{
      R[i,j] <- 0
    }
  }
}

#Zero matrix
Z <- matrix(0, nrow=2, ncol=7)

#Identity matrix
I2 <- diag(2)
I7 <- diag(7)

#Fundamental matrix
N1 = solve(I7 - Q)

#absorption probabilities
B1 = N1%*%R

t1 = cbind(Q,R)
t2 = cbind(Z,I2)
ActualMatrix = rbind(t1,t2)
colnames(ActualMatrix) <- c("1","2","3","4","5","6","7","0","8")
rownames(ActualMatrix) <- c("1","2","3","4","5","6","7","0","8")

kable(ActualMatrix, format="pandoc", align = "l", digits = 3, caption = "Transition Matrix", row.names=T)
Transition Matrix
1 2 3 4 5 6 7 0 8
1 0.0 0.4 0.0 0.0 0.0 0.0 0.0 0.6 0.0
2 0.6 0.0 0.4 0.0 0.0 0.0 0.0 0.0 0.0
3 0.0 0.6 0.0 0.4 0.0 0.0 0.0 0.0 0.0
4 0.0 0.0 0.6 0.0 0.4 0.0 0.0 0.0 0.0
5 0.0 0.0 0.0 0.6 0.0 0.4 0.0 0.0 0.0
6 0.0 0.0 0.0 0.0 0.6 0.0 0.4 0.0 0.0
7 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.0 0.4
0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0
8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 
colnames(B1) <- c("Failure","Success")
rownames(B1) <- c("1","2","3","4","5","6","7")

kable(B1, format="pandoc", align = "l", digits = 4, caption = "Absorption Probabilities", row.names=T)
Absorption Probabilities
Failure Success
1 0.9797 0.0203
2 0.9492 0.0508
3 0.9036 0.0964
4 0.8351 0.1649
5 0.7323 0.2677
6 0.5781 0.4219
7 0.3469 0.6531

(b) Smith bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).

Q <- matrix(0, nrow=3, ncol=3)
Q[1,2] <- p
Q[2,3] <- p

R <- matrix(c(0.6,0,0.6,0,0.6,0.4), nrow=3, byrow=T)
Z <- matrix(0, nrow=2, ncol=3)
I3 <- diag(3)

I2 <- diag(2)

#fundamental matrix
N1 = solve(I3 - Q)

#absorption probabilities
B = N1%*%R

t1 = cbind(Q,R)
t2 = cbind(Z,I2)
ActualMatrix = rbind(t1,t2)

colnames(ActualMatrix) <- c("1","2","4","0","8")
rownames(ActualMatrix) <- c("1","2","4","0","8")

kable(ActualMatrix, format="pandoc", align = "l", digits = 3, caption = "Transition Matrix", row.names=T)
Transition Matrix
1 2 4 0 8
1 0 0.4 0.0 0.6 0.0
2 0 0.0 0.4 0.6 0.0
4 0 0.0 0.0 0.6 0.4
0 0 0.0 0.0 1.0 0.0
8 0 0.0 0.0 0.0 1.0 
colnames(B) <- c("Failure","Success")
rownames(B) <- c("1","2","4")

kable(B, format="pandoc", align = "l", digits = 3, caption = "Absorption Probabilities", row.names=T)
Absorption Probabilities
Failure Success
1 0.936 0.064
2 0.840 0.160
4 0.600 0.400

(c) Which strategy gives Smith the better chance of getting out of jail?

Bold strategy gives Smith better chance to get out to jail, as probability of winning first bet is slightly higer than timid strategy.

References