Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6.

Find the probability that he wins 8 dollars before losing all of his money if

  1. he bets 1 dollar each time (timid strategy).

        0.4  0.4  0.4  0.4  0.4  0.4  0.4

    $0 $1 → $2 → $3 → $4 → $5 → $6 → $7 → $8 ← ← ← ← ← ← ←
    0.6 0.6 0.6 0.6 0.6 0.6 0.6

Let ϕ(i) be the probability that the chain reaches state 8 before reaching state 0, starting from state i. In other words, if Sj is the first n ≥ 0 such that Xn = j,

          ϕ(i) = Pi(S8 < S0) = P(S8 < S0|X0 = i).

Using first-step analysis (viz. the Markov property at time n = 1), we have

          ϕ(i) = 0.4ϕ(i + 1) + 0.6ϕ(i − 1), i = 1, 2, 3, 4, 5, 6, 7
          ϕ(0) = 0
          ϕ(8) = 1.

We solve this system of linear equations and find

  ϕ = (ϕ(1), ϕ(2), ϕ(3), ϕ(4), ϕ(5), ϕ(6), ϕ(7))
    = (0.0203, 0.0508, 0.0964, 0.1649, 0.2677, 0.4219, 0.6531, 1).
w <- 0.4
l <- 0.6
p <- l/w

for (i in 1:8){
  p_bail <- (1-p^i)/(1-p^8)
  print(p_bail)
}
## [1] 0.02030135
## [1] 0.05075337
## [1] 0.0964314
## [1] 0.1649485
## [1] 0.267724
## [1] 0.4218874
## [1] 0.6531324
## [1] 1

This means that the probability that the chain reaches state 8 before reaching state 0 starting from state 1 is the first component of this vector and equal to .0203. The probability increases as Smith wins.

  1. he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).

The probability that the chain reaches state 8 before reaching state 0, starting from state 3 is the third component of this vector and is equal to 0.0964. Note that ϕ(i) is increasing in i, which was expected.

Now the chain is: see attached handwritten drawing in microsoft word.

and the equations are:

                ϕ(3) = 0.4ϕ(6)
                ϕ(6) = 0.4ϕ(8) + 0.6ϕ(4)
                ϕ(4) = 0.4ϕ(8)
                ϕ(0) = 0
                ϕ(8) = 1.

We solve and find ϕ(3) = 0.256,ϕ(4) = 0.4,ϕ(6) = 0.64

The sequence in the drawing is 1,2,4 and 8 dollar states. P(1)=0.4P(2) P(2)=0.4P(4) P(4)=0.4P(8)

#he needs to win each time
p_bail_b <- 0.4*0.4*0.4
p_bail_b
## [1] 0.064
  1. Which strategy gives Smith the better chance of getting out of jail?

By comparing the third components of the vector ϕ we find that the bold strategy gives Smith a better chance to get out jail.