(a) he bets 1 dollar each time (timid strategy).

The timid strategy can be modeled as an absorbing Markov chain. From any given dollars amount n where n is 1 to 8, the probability of moving to n???1 is 0.6 while the probability of moving to n+1 is 0.4. The absorbing states are 0 (no money left to bet) and 8 (the game has been won).

• probability of loosing A dollars: \(q=0.6\);
• probability of winning A dollars: \(p=0.4\);
• Stake: \(s=1\);
• Absorbing state: \(M=8\);

Therefore,

\(\frac{q}{p} = \frac {0.6}{0.4}=1.5\)

\(P=\frac {1-(\frac{q}{p})^s}{1-(\frac{q}{p})^M}\)

\(P=\frac{1-1.5^1}{1-1.5^8}=0.0203\)

q <- 0.6
p <- 0.4
m <- 8
qp = q/p
for (s in 1:8) {
P = (1 - qp^s)/(1-qp^m)
print(paste0("s: " ,s," P:  ", P))
print(P)
}

## [1] "s: 1 P:  0.0203013481363997"
## [1] 0.02030135
## [1] "s: 2 P:  0.0507533703409993"
## [1] 0.05075337
## [1] "s: 3 P:  0.0964314036478986"
## [1] 0.0964314
## [1] "s: 4 P:  0.164948453608248"
## [1] 0.1649485
## [1] "s: 5 P:  0.267724028548771"
## [1] 0.267724
## [1] "s: 6 P:  0.421887390959556"
## [1] 0.4218874
## [1] "s: 7 P:  0.653132434575734"
## [1] 0.6531324
## [1] "s: 8 P:  1"
## [1] 1

Probability is 0.0203

• Above indicates the probabilities of winning starting at position 1 of approximately 0.0203 or about 2% and a probaiblity of losing of 0.979 or about 98%.

(b) he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).

Answer

Since the prisoner bets A dollars and will lose or gain A dollars, and he bets his entire money each time until $8. he must win each time or lose. If he wins sequence is: $1,$2,$4,$8. He starts with $1 and must win 3 consecutive bets in a row at probability p=0.4

# Transition matrix in canonical form
s <- matrix(c(1,  0,  0,0,0,
              0.6,0,0.4,0,0,
              0.6,0,0,0.4,0,
              0.6,0,0,0,0.4,
              0,  0,  0,0,1), nrow = 5, byrow=TRUE)
s

##      [,1] [,2] [,3] [,4] [,5]
## [1,]  1.0    0  0.0  0.0  0.0
## [2,]  0.6    0  0.4  0.0  0.0
## [3,]  0.6    0  0.0  0.4  0.0
## [4,]  0.6    0  0.0  0.0  0.4
## [5,]  0.0    0  0.0  0.0  1.0

s1 <- matrix(c(0,1,0,0,0),nrow=1)
s1

##      [,1] [,2] [,3] [,4] [,5]
## [1,]    0    1    0    0    0

s2 <- s1 %*% s
s2

##      [,1] [,2] [,3] [,4] [,5]
## [1,]  0.6    0  0.4    0    0

s3 <- s2 %*% s
s3

##      [,1] [,2] [,3] [,4] [,5]
## [1,] 0.84    0    0 0.16    0

s4 <- s3 %*% s
s4

##       [,1] [,2] [,3] [,4]  [,5]
## [1,] 0.936    0    0    0 0.064

Therefore, from the Binomial Distribution, we have that:

(P <- 0.4^3)

## [1] 0.064

Probability is 0.064

• Thus the probability of winning is: 0.4×0.4×0.4=0.64 or 6.4%
• The probability of losing is comprised of 3 events, losing on the 1st, 2nd, or 3rd play: 0.6+0.6×0.4+0.6×0.4×0.4=0.936 or 93.6%

(c) Which strategy gives Smith the better chance of getting out of jail?

Answer

To summarize, this game has a negative expected value. That is, on any given bet, the prisoner is more likely to lose than win. In such cases, the best strategy is to play as few times as possible. The more prisoner plays the closer the result will match the expected outcome (due to the law of large numbers). The expected outcome is that the prisoner will lose all his money.

The bold strategy will play at most 3 times while the timid strategy must play, at a minimum, 8 times to win the game.

Bold strategy gives Smith a better chance to get out of jail because it gives him more than triple his winning chances with this strategy.

---