Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6. Find the probability that he wins 8 dollars before losing all of his money if (a) he bets 1 dollar each time (timid strategy).

Probability of successful bet for Smith: p = 0.4 Probability of unsuccessful bet for Smith: q = 0.6 (where q = 1 - p) Ratio of success probability to failure probability: s = p/q = 2/3 Starting wealth A = $1 Let Rn denote Smith’s wealth after the nth bet. If he wins the first game, his wealth becomes: Rn = A + 1. If Smith loses the first bet, his wealth becomes: Rn = A - 1. The sum of money at stake is $8 i.e. when Smith makes enough to get bail. The game ends when Smith’s wealth either becomes 8 dollars (i.e. Rn = 8) or loses everything (Rn = 0). From a Markov chain perspective, these 2 states are the absorbing states i.e. the chain stops if either of these states is hit. The in-between states from 1 to 7 are the recurrent states.

Based on section 12.2 on Page 496 of the Grinstead textbook, we have: q=0.6; p=0.4; s=1; M=8; q/p=0.6/0.4=1.5;

\(P=(1-(q/p)^s)/(1-(q/p)^M)\)

P=(1-1.5)/(1-1.5)^8=0.0203

Setting up the problem as a simulation in R below:

#   bet(k, n, p)
#   k: Smith's initial state
#   n: Smith bets until either $8 or he goes bust
#   p: Probability of winning $1 at each play
#   Function returns 0 if Smith is eventually ruined
#   Function eturns 1 if Smith eventually wins $n
  
bet <- function(k,n,p) {
        stake <- k
        while (stake > 0 & stake < n) {
                betval <- sample(c(-1,1),1,prob=c(1-p,p))
                stake <- stake + betval
        }
        if (stake == 0) return(0) else return(1)
        }   

k <- 1 
n <- 8  
p <- 0.4  
trials <- 500000
simlist <- replicate(trials, bet(k, n, p))
print(mean(simlist)) # Estimate of probability that Smith makes bail
## [1] 0.02057

The probability that Smith wins $8 following the timid strategy is about 0.0205

  1. he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).

Smith can only bet as much money as he has at any state and his probability of winning p=0.4 remains constant at every state. At state 1, Smith starts with 1, so he can bet a maximum of 1. If we wins, his new stake is k=2. At state 2, Smith starts with 2, so he can bet a maximum of 2. If we wins, his new stake is k=4. At state 3, Smith starts with 4, so he can bet a maximum of 4. If we wins, his new stake is k=8. Since having 8 dollars allows Smith to get out on bail, he will stop at this state. In all other scenarios, Smith loses at either state 1, 2 or 3 ending his game. Smith must win 3 times in a row in order to make bail. Since each state has the probability p=0.4 of winning, we know that the final probability of making bail using this strategy is p^3.

The probability of the scenarios 1, 2, and 3 happening in a row is:0.064

p<-0.4
p^3
## [1] 0.064
  1. Which strategy gives Smith the better chance of getting out of jail?

The timid strategy gives Smith a 2.03% chance of getting out of jail, whereas the bold strategy gives him a 6.4% chance of getting out of jail, so the bold strategy is better.