Chapter 11.2 Exercise 13

Smith is in jail and has \(1\) dollar; he can get out on bail if he has \(8\) dollars. A guard agrees to make a series of bets with him. If Smith bets \(A\) dollars, he wins A dollars with probability \(.4\) and loses A dollars with probability \(.6\). Find the probability that he wins \(8\) dollars before losing all of his money if

  1. he bets \(1\) dollar each time (timid strategy).
  2. he bets, each time, as much as possible but not more than necessary to bring his fortune up to \(8\) dollars (bold strategy).
  3. Which strategy gives Smith the better chance of getting out of jail?

ANSWERS

  1. \(P_{timid} \approx 0.0203\)
  2. \(P_{bold} = 0.064\)
  3. Bold strategy gives Smith significantly better chance of getting out of jail than the timid strategy. It is still a very small chance, so I wouldn’t count on it.

SOLUTION

Using the bold strategy Smith needs to bet 3 times to reach $8 - to get from $1 to $2, from $2 to $4 and from $4 to $8. The probability of success for each bet is \(0.4\), so the probability of 3 successes in a row is \(P_{bold}=p^3 = 0.4^3 = 0.064\).

For the timid strategy, I have decided to run a simulation first to see what kind of values to expect.

trials <- 10000
out_of_jail <- 0
for (i in 1:trials) {
  bank <- 1
  while (bank>0 & bank<8) {
    bank<-bank+sample(c(-1,1),1,replace=TRUE,prob=c(0.6,0.4))
  }
  out_of_jail<-out_of_jail+(bank>=8)
}
paste0("Smith got out of jail ",out_of_jail,
      " times in ",trials,
      " simulations with probability of success ",out_of_jail/trials,".")
## [1] "Smith got out of jail 218 times in 10000 simulations with probability of success 0.0218."

After simulating the strategy 10,000 times we can see that expected the probability of success is about 0.02. This will not be a better strategy than the bold one.

In order to find the theoretical solution, I will use the transition matrix. It is presented below in canonical form. Each state is the amount of money Smith holds. States 0 and 8 are absorbing ones. Other states can increase by one with probability of 0.4 and decrease by one with probability of 0.6.

# Transition matrix in canonical form
P <- matrix(c(0,0.4,0,0,0,0,0,0.6,0,
              0.6,0,0.4,0,0,0,0,0,0,
              0,0.6,0,0.4,0,0,0,0,0,
              0,0,0.6,0,0.4,0,0,0,0,
              0,0,0,0.6,0,0.4,0,0,0,
              0,0,0,0,0.6,0,0.4,0,0,
              0,0,0,0,0,0.6,0,0,0.4,
              0,0,0,0,0,0,0,1,0,
              0,0,0,0,0,0,0,0,1), nrow=9, byrow=TRUE)
rownames(P) <- c("1","2","3","4","5","6","7","0","8")
colnames(P) <- c("1","2","3","4","5","6","7","0","8")
P
##     1   2   3   4   5   6   7   0   8
## 1 0.0 0.4 0.0 0.0 0.0 0.0 0.0 0.6 0.0
## 2 0.6 0.0 0.4 0.0 0.0 0.0 0.0 0.0 0.0
## 3 0.0 0.6 0.0 0.4 0.0 0.0 0.0 0.0 0.0
## 4 0.0 0.0 0.6 0.0 0.4 0.0 0.0 0.0 0.0
## 5 0.0 0.0 0.0 0.6 0.0 0.4 0.0 0.0 0.0
## 6 0.0 0.0 0.0 0.0 0.6 0.0 0.4 0.0 0.0
## 7 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.0 0.4
## 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0
## 8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0

Now let us find the fundametal matrix.

# Transient to Transient matrix
Q <- P[1:7,1:7]
# Transient to Absorbing matrix
R <- P[1:7,8:9]

# Fundamental matrix
I <- diag(7)
N <- solve(I-Q)
N
##           1        2         3         4         5         6          7
## 1 1.6328311 1.054718 0.6693101 0.4123711 0.2410785 0.1268834 0.05075337
## 2 1.5820777 2.636796 1.6732752 1.0309278 0.6026963 0.3172086 0.12688343
## 3 1.5059477 2.509913 3.1792228 1.9587629 1.1451229 0.6026963 0.24107851
## 4 1.3917526 2.319588 2.9381443 3.3505155 1.9587629 1.0309278 0.41237113
## 5 1.2204600 2.034100 2.5765266 2.9381443 3.1792228 1.6732752 0.66931007
## 6 0.9635210 1.605868 2.0340999 2.3195876 2.5099128 2.6367962 1.05471848
## 7 0.5781126 0.963521 1.2204600 1.3917526 1.5059477 1.5820777 1.63283109

Now let us find the matrix of absorbing probabilities.

# Absorption probablities
B <- N %*% R
B
##           0          8
## 1 0.9796987 0.02030135
## 2 0.9492466 0.05075337
## 3 0.9035686 0.09643140
## 4 0.8350515 0.16494845
## 5 0.7322760 0.26772403
## 6 0.5781126 0.42188739
## 7 0.3468676 0.65313243

Per stated problem, Smith starts with $1 and needs to end up with $8. The probability of this happening is

B["1","8"]
## [1] 0.02030135

Interestingly even starting with $6, the probability of getting out of jail is not on Smith’s side.

Neither strategy gives you particulary good chance of getting out of jail. However, bold strategy is noticeably better than timid strategy (1 in 15 chance vs. 1 in 50 chance). Timid strategy may feel like a “safe” one since losses are minimized during each bet, but in fact, being bold pays off here.