Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6.

Find the probability that he wins 8 dollars before losing all of his money if:

  1. he bets 1 dollar each time (timid strategy).
  2. he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).
  3. Which strategy gives Smith the better chance of getting out of jail?

Solution:

(a) he bets 1 dollar each time (timid strategy).

Given:

\[ p = 0.4, \quad q = 0.6 \]

\[ P(0) = 0, \quad P(8) = 1 \]

For \(1 \leq i \leq 7\):

\[ P(i) = p \cdot P(i+1) + q \cdot P(i-1) \]

We solve for \(P(1)\) through \(P(7)\) using backward induction:

\[ P(7) = 0.4 \times 1 + 0.6 \times P(6) \] \[ P(6) = 0.4 \times P(7) + 0.6 \times P(5) \] \[ P(5) = 0.4 \times P(6) + 0.6 \times P(4) \] \[ P(4) = 0.4 \times P(5) + 0.6 \times P(3) \] \[ P(3) = 0.4 \times P(4) + 0.6 \times P(2) \] \[ P(2) = 0.4 \times P(3) + 0.6 \times P(1) \] \[ P(1) = 0.4 \times P(2) \] (since \(P(0) = 0\))

Starting with \(P(7)\), we can iteratively substitute back into each previous equation until we reach \(P(1)\). This gives us a value for \(P(1)\), which is the probability of reaching $8 starting with $1.

\[ p(7) \approx 0.653 \times p(8) = 0.653 \] \[ p(6) \approx 0.422 \times p(8) = 0.422 \] \[ p(5) \approx 0.268 \times p(8) = 0.268 \] \[ p(4) \approx 0.165 \times p(8) = 0.165 \] \[ p(3) \approx 0.0964 \times p(8) = 0.0964 \] \[ p(2) \approx 0.0508 \times p(8) = 0.0508 \] \[ p(1) \approx 0.0203 \times p(8) = 0.0203 \]

I also wanted to do the same thing using Markov’s chain transition matrix:

library(markovchain)
## Package:  markovchain
## Version:  0.9.5
## Date:     2023-09-24 09:20:02 UTC
## BugReport: https://github.com/spedygiorgio/markovchain/issues
states=c("0", "1", "2", "3", "4", "5", "6", "7", "8")
transitionMatrix = matrix(c(
  1, 0, 0, 0, 0, 0, 0, 0, 0,
  0.6, 0, 0.4, 0, 0, 0, 0, 0, 0,
  0, 0.6, 0, 0.4, 0, 0, 0, 0, 0,
  0, 0, 0.6, 0, 0.4, 0, 0, 0, 0,
  0, 0, 0, 0.6, 0, 0.4, 0, 0, 0,
  0, 0, 0, 0, 0.6, 0, 0.4, 0, 0,
  0, 0, 0, 0, 0, 0.6, 0, 0.4, 0,
  0, 0, 0, 0, 0, 0, 0.6, 0, 0.4,
  0, 0, 0, 0, 0, 0, 0, 0, 1
  ), nrow = 9, byrow = TRUE, dimnames=list(states, states))

transitionMatrix
##     0   1   2   3   4   5   6   7   8
## 0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
## 1 0.6 0.0 0.4 0.0 0.0 0.0 0.0 0.0 0.0
## 2 0.0 0.6 0.0 0.4 0.0 0.0 0.0 0.0 0.0
## 3 0.0 0.0 0.6 0.0 0.4 0.0 0.0 0.0 0.0
## 4 0.0 0.0 0.0 0.6 0.0 0.4 0.0 0.0 0.0
## 5 0.0 0.0 0.0 0.0 0.6 0.0 0.4 0.0 0.0
## 6 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.4 0.0
## 7 0.0 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.4
## 8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0
mc <- new("markovchain", states = states, transitionMatrix = transitionMatrix, name = "markov chain object")

absorptionProbabilities(mc)
##           0          8
## 1 0.9796987 0.02030135
## 2 0.9492466 0.05075337
## 3 0.9035686 0.09643140
## 4 0.8350515 0.16494845
## 5 0.7322760 0.26772403
## 6 0.5781126 0.42188739
## 7 0.3468676 0.65313243

As we see in both approaches we get the same answer for \[ p(1) = 0.0203 \]

(b) he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).

Bold startegy is simpler. We can get the results without the transiotion matrix using just simple logic.

So the probability of Smith getting up to 8 dollars with the bold startegy is:

\[ 0.4 \times \times 0.4 \times 0.4 = 0.064 \]

(c) Which strategy gives Smith the better chance of getting out of jail?

Given that \(P(1) \approx 0.0203\) for the timid strategy and \(P(1) = 0.064\) for the bold strategy, Smith has a better chance of reaching $8 before losing all his money using the bold strategy.