Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets \(A\) dollars, he wins \(A\) dollars with probability .4 and loses \(A\) dollars with probability .6.
Find the probability that he wins 8 dollars before losing all of his money if:
Setup
win = \(p\) = .4
lose = \(q\) = .6
\(S_i\) = 1
\(S_N\) = 8
Absorbing states are 0 and 8.
Transition Matrix
$1 Bets
\[\begin{array}{c|ccccccccc} & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 \\ \hline 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\ 1 & .6 & 0 & .4 & 0 & 0 & 0 & 0 & 0 & 0\\ 2 & 0 & .6 & 0 & .4 & 0 & 0 & 0 & 0 & 0\\ 3 & 0 & 0 & .6 & 0 & .4 & 0 & 0 & 0 & 0\\ 4 & 0 & 0 & 0 & .6 & 0 & .4 & 0 & 0 & 0\\ 5 & 0 & 0 & 0 & 0 & .6 & 0 & .4 & 0 & 0\\ 6 & 0 & 0 & 0 & 0 & 0 & .6 & 0 & .4 & 0\\ 7 & 0 & 0 & 0 & 0 & 0 & 0 & .6 & 0 & .4\\ 8 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1\\ \end{array} \]
Initial State Vector \[v = \left [ \begin{array}{ccccccccc} 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{array} \right] \] The 9th column (state 8) will have the probability of achieving that state when starting at state 1
trans_matA <- matrix(c(1,0,0,0,0,0,0,0,0,
.6,0,.4,0,0,0,0,0,0,
0,.6,0,.4,0,0,0,0,0,
0,0,.6,0,.4,0,0,0,0,
0,0,0,.6,0,.4,0,0,0,
0,0,0,0,.6,0,.4,0,0,
0,0,0,0,0,.6,0,.4,0,
0,0,0,0,0,0,.6,0,.4,
0,0,0,0,0,0,0,0,1), ncol = 9, byrow = T)
#initial state vector
v1 <- c(0,1,0,0,0,0,0,0,0)
#since we want to do row-wise operations and vectors are stored as columns, we transpose the matrix
trans_matA <- t(trans_matA)
m1 <- trans_matA
#Run 100 'steps'
for(i in 1:100){m1 <- m1 %*% trans_matA}
m1 %*% v1## [,1]
## [1,] 9.796892e-01
## [2,] 0.000000e+00
## [3,] 4.732561e-06
## [4,] 0.000000e+00
## [5,] 4.461902e-06
## [6,] 0.000000e+00
## [7,] 2.103361e-06
## [8,] 0.000000e+00
## [9,] 2.029948e-02\(S_i\) is the starting state
\(S_N\) is the ending state
\(P = \frac{1 - (\frac{q}{p})^{S_i}}{1 - (\frac{q}{p})^{S_N}}\)
Answer \(P = .02\)
p <- (1-(1.5)^1)/(1-(1.5)^8)
p## [1] 0.02030135I interpreted this question as, if he has $4 or less, he bets it all. If he has $5, he only bets $3, etc…
The 9th column (state 8) will have the probability of achieving that state when starting at state 1
Transition Matrix
Max $ Bets
\[\begin{array}{c|ccccccccc} & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 \\ \hline 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\ 1 & .6 & 0 & .4 & 0 & 0 & 0 & 0 & 0 & 0\\ 2 & .6 & 0 & 0 & 0 & .4 & 0 & 0 & 0 & 0\\ 3 & .6 & 0 & 0 & 0 & 0 & 0 & .4 & 0 & 0\\ 4 & .6 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & .4\\ 5 & 0 & 0 & .6 & 0 & 0 & 0 & 0 & 0 & .4\\ 6 & 0 & 0 & 0 & 0 & .6 & 0 & 0 & 0 & .4\\ 7 & 0 & 0 & 0 & 0 & 0 & 0 & .6 & 0 & .4\\ 8 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1\\ \end{array} \]
Initial State Vector \[v = \left [ \begin{array}{ccccccccc} 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{array} \right] \]
trans_matB <- matrix(c(1,0,0,0,0,0,0,0,0,
.6,0,.4,0,0,0,0,0,0,
.6,0,0,0,.4,0,0,0,0,
.6,0,0,0,0,0,.4,0,0,
.6,0,0,0,0,0,0,0,.4,
0,0,0,0,.6,0,0,0,.4,
0,0,0,0,0,.6,0,0,.4,
0,0,0,0,0,0,.6,0,.4,
0,0,0,0,0,0,0,0,1), ncol = 9, byrow = T)
#initial state vector
v2 <- c(0,1,0,0,0,0,0,0,0)
#since we want to do row-wise operations and vectors are stored as columns, we transpose the matrix
trans_matB <- t(trans_matB)
m2 <- trans_matB
#Run 100 'steps'
for(i in 1:100){m2 <- m2 %*% trans_matB}
m2 %*% v2## [,1]
## [1,] 0.936
## [2,] 0.000
## [3,] 0.000
## [4,] 0.000
## [5,] 0.000
## [6,] 0.000
## [7,] 0.000
## [8,] 0.000
## [9,] 0.064Answer
\(P = .064\)
Even though the odds are still very low, he has almost a 4% better chance when maxing his bets.