Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6.

Find the probability that he wins 8 dollars before losing all of his money if

(a) he bets 1 dollar each time (timid strategy).

Smith’s outcome is a Markov Chain

Let’s create the transition matrix in canonical form.

P = \[\begin{array}{c | c} Q & R \\ \hline 0 & I\\ \end{array}\]

\

P = \[\begin{array}{c c c c c c c c | c c } & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 0 & 8 \\ 1 & 0 & 0.4 & 0 & 0 & 0 & 0 & 0 & 0.6 & 0\\ 2 & 0.6 & 0 & 0.4 & 0 & 0 & 0 & 0 & 0 & 0 \\ 3 & 0 & 0.6 & 0 & 0.4 & 0 & 0 & 0 & 0 & 0\\ 4 & 0 & 0 & 0.6 & 0 & 0.4 & 0 & 0 & 0 & 0\\ 5 & 0 & 0 & 0 & 0.6 & 0 & 0.4 & 0 & 0 & 0\\ 6 & 0 & 0 & 0 & 0 & 0.6 & 0 & 0.4 & 0 & 0\\ 7 & 0 & 0 & 0 & 0 & 0 & 0.6 & 0 & 0 & 0.4\\ \hline 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0\\ 8 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 \\ \end{array}\]

From this we see that the matrix Q is

Q = \[\begin{pmatrix} 0 & 0.4 & 0 & 0 & 0 & 0 & 0\\ 0.6 & 0 & 0.4 & 0 & 0 & 0 & 0 \\ 0 & 0.6 & 0 & 0.4 & 0 & 0 & 0\\ 0 & 0 & 0.6 & 0 & 0.4 & 0 & 0\\ 0 & 0 & 0 & 0.6 & 0 & 0.4 & 0\\ 0 & 0 & 0 & 0 & 0.6 & 0 & 0.4\\ 0 & 0 & 0 & 0 & 0 & 0.6 & 0 \end{pmatrix}\] I = \[\begin{pmatrix} 1 & 0 & 0 & 0 & 0 & 0 & 0\\ 0 & 1 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 1 & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & 1 & 0 & 0\\ 0 & 0 & 0 & 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 0 & 0 & 0 & 1 \end{pmatrix}\] R = \[\begin{array}{ccc} & 0 & 8 \\ 1 & 0.6 & 0 \\ 2 & 0 & 0 \\ 3 & 0 & 0\\ 4 & 0 & 0\\ 5 & 0 & 0\\ 6 & 0 & 0\\ 7 & 0 & 0.4\\ \end{array}\]

Calculation:

I <- diag(7)
I
##      [,1] [,2] [,3] [,4] [,5] [,6] [,7]
## [1,]    1    0    0    0    0    0    0
## [2,]    0    1    0    0    0    0    0
## [3,]    0    0    1    0    0    0    0
## [4,]    0    0    0    1    0    0    0
## [5,]    0    0    0    0    1    0    0
## [6,]    0    0    0    0    0    1    0
## [7,]    0    0    0    0    0    0    1
Q <- matrix(c(rep(c(0, 0.4, 0, 0, 0, 0, 0, 0.6), 6), 0), 7, byrow = T)

Q
##      [,1] [,2] [,3] [,4] [,5] [,6] [,7]
## [1,]  0.0  0.4  0.0  0.0  0.0  0.0  0.0
## [2,]  0.6  0.0  0.4  0.0  0.0  0.0  0.0
## [3,]  0.0  0.6  0.0  0.4  0.0  0.0  0.0
## [4,]  0.0  0.0  0.6  0.0  0.4  0.0  0.0
## [5,]  0.0  0.0  0.0  0.6  0.0  0.4  0.0
## [6,]  0.0  0.0  0.0  0.0  0.6  0.0  0.4
## [7,]  0.0  0.0  0.0  0.0  0.0  0.6  0.0
I - Q
##      [,1] [,2] [,3] [,4] [,5] [,6] [,7]
## [1,]  1.0 -0.4  0.0  0.0  0.0  0.0  0.0
## [2,] -0.6  1.0 -0.4  0.0  0.0  0.0  0.0
## [3,]  0.0 -0.6  1.0 -0.4  0.0  0.0  0.0
## [4,]  0.0  0.0 -0.6  1.0 -0.4  0.0  0.0
## [5,]  0.0  0.0  0.0 -0.6  1.0 -0.4  0.0
## [6,]  0.0  0.0  0.0  0.0 -0.6  1.0 -0.4
## [7,]  0.0  0.0  0.0  0.0  0.0 -0.6  1.0
# Fundamental Matrix
N <- solve(I - Q)
round(N, 4)
##        [,1]   [,2]   [,3]   [,4]   [,5]   [,6]   [,7]
## [1,] 1.6328 1.0547 0.6693 0.4124 0.2411 0.1269 0.0508
## [2,] 1.5821 2.6368 1.6733 1.0309 0.6027 0.3172 0.1269
## [3,] 1.5059 2.5099 3.1792 1.9588 1.1451 0.6027 0.2411
## [4,] 1.3918 2.3196 2.9381 3.3505 1.9588 1.0309 0.4124
## [5,] 1.2205 2.0341 2.5765 2.9381 3.1792 1.6733 0.6693
## [6,] 0.9635 1.6059 2.0341 2.3196 2.5099 2.6368 1.0547
## [7,] 0.5781 0.9635 1.2205 1.3918 1.5059 1.5821 1.6328
R <- matrix(rep(c(0, 0), 7), nrow = 7)
R[1,1] <- 0.6
R[7,2] <- 0.4
R
##      [,1] [,2]
## [1,]  0.6  0.0
## [2,]  0.0  0.0
## [3,]  0.0  0.0
## [4,]  0.0  0.0
## [5,]  0.0  0.0
## [6,]  0.0  0.0
## [7,]  0.0  0.4
B <- N %*% R
colnames(B) <- c(0, 8)
round(B, 4)
##           0      8
## [1,] 0.9797 0.0203
## [2,] 0.9492 0.0508
## [3,] 0.9036 0.0964
## [4,] 0.8351 0.1649
## [5,] 0.7323 0.2677
## [6,] 0.5781 0.4219
## [7,] 0.3469 0.6531

(b) he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).

In other words with each bet, his winning doubles. The chances of this happening is like so:

Smith starts with 1

Risk 1: He wins 2

Risk 2: He wins 4

Risk 3: He wins 8

Just 3 chances (in a row) he may need to take using the bold strategy. The chances of luck being on Smith’s side is:

0.4^3
## [1] 0.064

(c) Which strategy gives Smith the better chance of getting out of jail?

The bold strategy with 6.4% compared to the timid strategy with 2.03%