Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6. Find the probability that he wins 8 dollars before losing all of his money if

(a) he bets 1 dollar each time (timid strategy).

The transition matrix is going to look like this:

\[P=\begin{bmatrix} 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0.6 & 0 & 0.4 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0.6 & 0 & 0.4 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0.6 & 0 & 0.4 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0.6 & 0 & 0.4 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0.6 & 0 & 0.4 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0.6 & 0 & 0.4 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0.6 & 0 & 0.4 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1\\ \end{bmatrix}\]

According to timid strategy, the probability that Smith wins 8 dollars before losing all is

\[P = \frac{1-(\frac{q}{p})^S}{1-(\frac{q}{p})^M} \\\] where p is probability of success (winning one dollar), q is probability of failure (loosing one dollar), S is initial state, M is final state

p <- 0.4
q <- 0.6
S <- 1
M <- 8
P <- (1-(q/p)^S)/(1-(q/p)^M)
P
## [1] 0.02030135
(b) he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).

Bet 1:\(0 \xleftarrow[]{\text{}}1\xrightarrow[]{\text{}} 2\) Bet 2:\(0 \xleftarrow[]{\text{}}2\xrightarrow[]{\text{}} 4\) Bet 3:\(0 \xleftarrow[]{\text{}}4\xrightarrow[]{\text{}} 8\)

The transition matrix is going to look like this:

\[P=\begin{bmatrix} 0 & 0 & 0 & 0 & 0 \\ 0.6 & 0 & 0.4 & 0 & 0\\ 0.6 & 0 & 0 & 0.4 & 0\\ 0.6 & 0 & 0 & 0 & 0.4\\ 0 & 0 & 0 & 0 & 1 \end{bmatrix}\]

Rows and columns represent five states: $0, $1, $2, $4, $8

#transition matrix
P <- matrix(c(1,0,0,0,0,0.6,0,0.4,0,0,0.6,0,0,0.4,0,0.6,0,0,0,0.4,0,0,0,0,1),ncol=5,nrow=5, byrow = TRUE)

#vector that represents the initial state of $1
V <- matrix(c(0,1,0,0,0),ncol=5,nrow=1, byrow = TRUE)

V%*%(P%^%4)
##       [,1] [,2] [,3] [,4]  [,5]
## [1,] 0.936    0    0    0 0.064
(c) Which strategy gives Smith the better chance of getting out of jail?

The probability that Smith wins 8 dollars before losing all of his money is about three times larger if he will use bold strategy(6.4%) rather than timid strategy (2.03%).