Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability \(.4\) and loses A dollars with probability \(.6\). Find the probability that he wins 8 dollars before losing all of his money if

  1. he bets 1 dollar each time (timid strategy).

The timid strategy can be modeled as an absorbing Markov chain. From any given dollars amount \(n\) where \(n\) is 1 to 7, the probability of moving to \(n-1\) is 0.6 while the probability of moving to \(n+1\) is 0.4. The absorbing states are 0 (no money left to bet) and 8 (the game has been won).

\[\mathbf{Q} = \left(\begin{array} {rrr} 0 & .4 & 0 & 0 & 0 & 0 & 0 \\ .6 & 0 & .4 & 0 & 0 & 0 & 0 \\ 0 & .6 & 0 & .4 & 0 & 0 & 0 \\ 0 & 0 & .6 & 0 & .4 & 0 & 0 \\ 0 & 0 & 0 & .6 & 0 & .4 & 0 \\ 0 & 0 & 0 & 0 & .6 & 0 & .4 \\ 0 & 0 & 0 & 0 & 0 & .6 & 0 \\ \end{array}\right)\]

\[\mathbf{R} = \left(\begin{array} {rrr} 0.6 & 0 \\ 0 & 0 \\ 0 & 0 \\ 0 & 0 \\ 0 & 0 \\ 0 & 0 \\ 0 & 0.4 \\ \end{array}\right)\]

Using these two matrices we can find the probability of winning or losing from any starting position by solving \((\mathbf{I}-\mathbf{Q})^{-1}\mathbf{R}\)

q <- matrix(c(0, .4, 0, 0, 0, 0, 0, .6, 0, 0.4, 0, 0, 0, 0, 0, 0.6, 0, 0.4, 0, 0, 0, 0, 0, 0.6, 0, 0.4, 0, 0, 0, 0, 0, 0.6, 0, 0.4, 0, 0, 0, 0, 0, 0.6, 0, 0.4, 0, 0, 0, 0, 0, 0.6, 0), byrow=TRUE, nrow=7, ncol=7)
r <- matrix(c(.6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.4), byrow=TRUE, nrow=7, ncol=2)
solve(diag(7)-q) %*% r
##           [,1]       [,2]
## [1,] 0.9796987 0.02030135
## [2,] 0.9492466 0.05075337
## [3,] 0.9035686 0.09643140
## [4,] 0.8350515 0.16494845
## [5,] 0.7322760 0.26772403
## [6,] 0.5781126 0.42188739
## [7,] 0.3468676 0.65313243

The top row indicates the probabilities of winning starting at position 1 of approximately \(0.02\) or about \(2\%\) and a probaiblity of losing of \(0.979\) or about \(98\%\).

  1. he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).

This can be modeled with a Markov chain, but given the limited options available, it is easier to model this with a branching tree. This version of the game is played at most 3 times. If the player wins 3 times, they have the 8 dollars needed to win and if they lose any game then they bust. Thus the probability of winning is:

\(0.4 \times 0.4 \times 0.4=0.64\) or \(6.4\%\)

The probability of losing is comprised of 3 events, losing on the 1st, 2nd, or 3rd play:

\(0.6 + 0.6\times 0.4 + 0.6\times 0.4 \times 0.4=0.936\) or \(93.6\%\)

As expected these two probabilities sum to 1, encompassing the complete possible outcomes of the game.

  1. Which strategy gives Smith the better chance of getting out of jail?

The bold strategy is the better strategy. Generally speaking, this game has a negative expected value. That is, on any given bet, you are more likely to lose than win. In such cases, the best strategy is to play as little as possible. The more you play the closer the result will match the expected outcome (due to the law of large numbers). The expected outcome is that you will lose all your money. The bold strategy will play at most 3 times while the timid strategy must play, at a minimum, 8 times to win the game.