Problem

Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6. Find the probability that he wins 8 dollars before losing all of his money if

  1. he bets 1 dollar each time (timid strategy).

  2. he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).

  3. Which strategy gives Smith the better chance of getting out of jail?


Timid Strategy Markov with Matrix Approach

Markov Equation

For the timid strategy, we use the Markov equation: \[ p_{i} = 0.4 \cdot p_{i+1} + 0.6 \cdot p_{i-1} \quad \text{for } 1 \leq i \leq 7, \quad \text{with } p_0 = 0 \text{ and } p_8 = 1. \]

Given:

  • \(p_i\) is probability of reaching 8 dollars starting from the current state \(i\), where \(i\) is the amount of money currently held.
  • \(p_{i+1}\) is the probability of reaching 8 dollars starting from the next state, i.e., having one more dollar than in state \(i\). This is the next winning bet.
  • \(p_{i-1}\) represents the probability of reaching the goal starting from the previous state, i.e., having one less dollar than in state \(i\). This is the next losing bet.
  • \(0.4\) is the probability of winning any given bet, moving from state \(i\) to state \(i+1\), increasing the current money by 1 dollar.
  • \(0.6\) is the probability of losing a bet, moving from state \(i\) to state \(i-1\), losing 1 dollar from the current state.


Absorbing States | Base Cases

These two states provide the boundaries of the problem. At \(P_0\), Smith is out of money and will remain in jail. At \(P_8\), Smith has 8 dollars and will go free.


\[\begin{align*} p_0 &= 0 \quad &\text{When there are 0 dollars, Smtih cannot reach 8 dollars} \\ p_8 &= 1 \quad &\text{When 8 dollars are reached, the goal has been met} \end{align*}\]


System of Probaility Equations

The probabilities can be calculated using a system of equations for states \(\text{for } 1 \leq i \leq 7\)

\[\begin{align*} p_1 &= 0.4 \cdot p_2 + 0.6 \cdot 0 && \text{(since $p_0 = 0$)} \\ p_2 &= 0.4 \cdot p_3 + 0.6 \cdot p_1 \\ p_3 &= 0.4 \cdot p_4 + 0.6 \cdot p_2 \\ p_4 &= 0.4 \cdot p_5 + 0.6 \cdot p_3 \\ p_5 &= 0.4 \cdot p_6 + 0.6 \cdot p_4 \\ p_6 &= 0.4 \cdot p_7 + 0.6 \cdot p_5 \\ p_7 &= 0.4 \cdot 1 + 0.6 \cdot p_6 && \text{(since $p_8 = 1$)} \end{align*}\]


Matrix Approach to solve for \(P_1\)

For the matrix approach, we start with \[ Ax = b \]

and solve for x. We rewrite the matrix equations fo to be

\[ x = A^{-1}b \]


Define the Matrices

Matrix A is derived from the system of equations.

\[ \text{} A = \begin{pmatrix} -1 & 0.4 & 0 & 0 & 0 & 0 & 0 \\ 0.6 & -1 & 0.4 & 0 & 0 & 0 & 0 \\ 0 & 0.6 & -1 & 0.4 & 0 & 0 & 0 \\ 0 & 0 & 0.6 & -1 & 0.4 & 0 & 0 \\ 0 & 0 & 0 & 0.6 & -1 & 0.4 & 0 \\ 0 & 0 & 0 & 0 & 0.6 & -1 & 0.4 \\ 0 & 0 & 0 & 0 & 0 & 0.6 & -1 \\ \end{pmatrix} \]


Matrix \(A^{-1}\) is the inverse of A (we multiple both side the inverse to solve for x)

\[ \text{} A^{-1} = \begin{pmatrix} -1.633 & -1.055 & -0.669 & -0.412 & -0.241 & -0.127 & -0.051 \\ -1.582 & -2.637 & -1.673 & -1.031 & -0.603 & -0.317 & -0.127 \\ -1.506 & -2.510 & -3.179 & -1.959 & -1.145 & -0.603 & -0.241 \\ -1.392 & -2.320 & -2.938 & -3.351 & -1.959 & -1.031 & -0.412 \\ -1.220 & -2.034 & -2.577 & -2.938 & -3.179 & -1.673 & -0.669 \\ -0.964 & -1.606 & -2.034 & -2.320 & -2.510 & -2.637 & -1.055 \\ -0.578 & -0.964 & -1.220 & -1.392 & -1.506 & -1.582 & -1.633 \\ \end{pmatrix} \]


The b vector is necessary to account for the absorbing \(P_8\) state = -0.4 as \(P_7\) transition to \(P_8\)

\[ \text{Vector } b = \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 0 \\ 0 \\ -0.4 \\ \end{pmatrix} \]


Timid Result Probabilities \(\text{for } 1 \leq i \leq 7\)

\[ \text{} x = A^{-1}b = \begin{pmatrix} 0.02030135 \\ 0.05075337 \\ 0.0964314 \\ 0.16494845 \\ 0.26772403 \\ 0.42188739 \\ 0.65313243 \\ \end{pmatrix} \]


From \(P_1\), there is a 0.0203 or 2.03% probability to reach the $8 dollar goal.

# Load the packages
library(Matrix)
library(pracma)

# Define the A matrix
A <- matrix(c(
  -1, 0.4, 0, 0, 0, 0, 0,
  0.6, -1, 0.4, 0, 0, 0, 0,
  0, 0.6, -1, 0.4, 0, 0, 0,
  0, 0, 0.6, -1, 0.4, 0, 0,
  0, 0, 0, 0.6, -1, 0.4, 0,
  0, 0, 0, 0, 0.6, -1, 0.4,
  0, 0, 0, 0, 0, 0.6, -1
), nrow = 7, byrow = TRUE)

# Define the b vector
b <- matrix(c(0, 0, 0, 0, 0, 0, -0.4), ncol = 1)

# Calculate the inverse of A
A_inv <- inv(A)

# Solve for x using the equation x = A_inv %*% b
x <- A_inv %*% b


# Display the results
cat("Matrix A:\n")
## Matrix A:
print(A)
##      [,1] [,2] [,3] [,4] [,5] [,6] [,7]
## [1,] -1.0  0.4  0.0  0.0  0.0  0.0  0.0
## [2,]  0.6 -1.0  0.4  0.0  0.0  0.0  0.0
## [3,]  0.0  0.6 -1.0  0.4  0.0  0.0  0.0
## [4,]  0.0  0.0  0.6 -1.0  0.4  0.0  0.0
## [5,]  0.0  0.0  0.0  0.6 -1.0  0.4  0.0
## [6,]  0.0  0.0  0.0  0.0  0.6 -1.0  0.4
## [7,]  0.0  0.0  0.0  0.0  0.0  0.6 -1.0
cat("\nInverse of Matrix A (A^-1):\n")
## 
## Inverse of Matrix A (A^-1):
print(A_inv)
##            [,1]      [,2]       [,3]       [,4]       [,5]       [,6]
## [1,] -1.6328311 -1.054718 -0.6693101 -0.4123711 -0.2410785 -0.1268834
## [2,] -1.5820777 -2.636796 -1.6732752 -1.0309278 -0.6026963 -0.3172086
## [3,] -1.5059477 -2.509913 -3.1792228 -1.9587629 -1.1451229 -0.6026963
## [4,] -1.3917526 -2.319588 -2.9381443 -3.3505155 -1.9587629 -1.0309278
## [5,] -1.2204600 -2.034100 -2.5765266 -2.9381443 -3.1792228 -1.6732752
## [6,] -0.9635210 -1.605868 -2.0340999 -2.3195876 -2.5099128 -2.6367962
## [7,] -0.5781126 -0.963521 -1.2204600 -1.3917526 -1.5059477 -1.5820777
##             [,7]
## [1,] -0.05075337
## [2,] -0.12688343
## [3,] -0.24107851
## [4,] -0.41237113
## [5,] -0.66931007
## [6,] -1.05471848
## [7,] -1.63283109
cat("\nb Vector:\n")
## 
## b Vector:
print(b)
##      [,1]
## [1,]  0.0
## [2,]  0.0
## [3,]  0.0
## [4,]  0.0
## [5,]  0.0
## [6,]  0.0
## [7,] -0.4
cat("\nx Vector (Result):\n")
## 
## x Vector (Result):
print(x)
##            [,1]
## [1,] 0.02030135
## [2,] 0.05075337
## [3,] 0.09643140
## [4,] 0.16494845
## [5,] 0.26772403
## [6,] 0.42188739
## [7,] 0.65313243


Bold Strategy Random Walk

Given that the bold strategy leads to a sequence of winning bets until reaching $8 or going bust, the solution is the probability of winning transitions in the random walk. These are the steps with probability of \(0.4\). Each winning step bets the total amount in the current state to reach 8 dollars (effectively doubling the amount in each step)

\[ P(\text{Success}) = 0.4 \times 0.4 \times 0.4 = 0.064 \] From \(P_1\), there is a 0.064 or 6.4% probability to reach the $8 dollar goal.

# Calculate the probability of success
success <- 0.4 * 0.4 * 0.4
success
## [1] 0.064


Final Answer

Given the timid approach has a probability of 0.0203 or 2.03% to reach 8 dollars from \(P_1\) and the bold strategy has a .064 or 6.4% to reach 8 dollars from \(P_1\), the BOLD approach gives Smith the better chance of getting out of jail from the initial \(P_1\) state.