Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6. Find the probability that he wins 8 dollars before losing all of his money if
he bets 1 dollar each time (timid strategy).
he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).
Which strategy gives Smith the better chance of getting out of jail?
For this strategy he bets only $1 each time never increasing or decreasing his bet.
To find the probabilities we will setup the transition matrix for this strategy it is shown below.
You can think about labeling each row and column starting from 0 -> 8 these represent how much money he has in each state and the probaility of going to another state.
\[ \begin{bmatrix} 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ .6 & 0 & .4 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & .6 & 0 & .4 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & .6 & 0 & .4 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & .6 & 0 & .4 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & .6 & 0 & .4 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 &.6 & 0 & .4 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & .6 & 0 & .4 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 \end{bmatrix} \]
We can use this transition matrix to create a markov chain for this problem and we can compute the probability of him making bail with this strategy.
library(markovchain)## Package: markovchain
## Version: 0.9.5
## Date: 2023-09-24 09:20:02 UTC
## BugReport: https://github.com/spedygiorgio/markovchain/issuesstrat_timid_transition_matrix <- matrix(c(
1,0,0,0,0,0,0,0,0,
.6,0,.4,0,0,0,0,0,0,
0,.6,0,.4,0,0,0,0,0,
0,0,.6,0,.4,0,0,0,0,
0,0,0,.6,0,.4,0,0,0,
0,0,0,0,.6,0,.4,0,0,
0,0,0,0,0,.6,0,.4,0,
0,0,0,0,0,0,.6,0,.4,
0,0,0,0,0,0,0,0,1
), nrow = 9, byrow = TRUE)
statesNames = c("0","1","2","3","4","5","6","7","8")
strat_timid <- new("markovchain", states = statesNames, byrow = TRUE, transitionMatrix = strat_timid_transition_matrix)
strat_timid_probs <- absorptionProbabilities(strat_timid)
print(strat_timid_probs)## 0 8
## 1 0.9796987 0.02030135
## 2 0.9492466 0.05075337
## 3 0.9035686 0.09643140
## 4 0.8350515 0.16494845
## 5 0.7322760 0.26772403
## 6 0.5781126 0.42188739
## 7 0.3468676 0.65313243Computing the absorption probabilities we can see that with the timid strategey only betting $1 each time we will be able to bail ourselves out about 2% of the time and we will lose all our money about 98% of the time
he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).
To find the probabilities we will setup the transition matrix for this strategy it is shown below.
You can think about labeling each row and column starting from 0 -> 8 these represent how much money he has in each state and the probaility of going to another state.
\[ \begin{bmatrix} 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ .6 & 0 & .4 & 0 & 0 & 0 & 0 & 0 & 0 \\ .6 & 0 & 0 & 0 & .4 & 0 & 0 & 0 & 0 \\ .6 & 0 & 0 & 0 & 0 & 0 & .4 & 0 & 0 \\ .6 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & .4 \\ 0 & 0 & .6 & 0 & 0 & 0 & 0 & 0 & .4 \\ 0 & 0 & 0 & 0 & .6 &0 & 0 & 0 & .4 \\ 0 & 0 & 0 & 0 & 0 & 0 & .6 & 0 & .4 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 \end{bmatrix} \]
We can use this transition matrix to create a markov chain for this problem and we can compute the probability of him making bail with this strategy.
strat_bold_transition_matrix <- matrix(c(
1,0,0,0,0,0,0,0,0,
.6,0,.4,0,0,0,0,0,0,
.6,0,0,0,.4,0,0,0,0,
.6,0,0,0,0,.4,0,0,0,
.6,0,0,0,0,0,0,0,.4,
0,0,.6,0,0,0,0,0,.4,
0,0,0,0,0,.6,0,0,.4,
0,0,0,0,0,0,.6,0,.4,
0,0,0,0,0,0,0,0,1
), nrow = 9, byrow = TRUE)
statesNames = c("0","1","2","3","4","5","6","7","8")
strat_bold <- new("markovchain", states = statesNames, byrow = TRUE, transitionMatrix = strat_bold_transition_matrix)
strat_bold_probs <- absorptionProbabilities(strat_bold)
print(strat_bold_probs)## 0 8
## 1 0.93600 0.06400
## 2 0.84000 0.16000
## 3 0.80160 0.19840
## 4 0.60000 0.40000
## 5 0.50400 0.49600
## 6 0.30240 0.69760
## 7 0.18144 0.81856Looking at the computed probabilities we can see that with this bold strategy he has a 6% chance of making bail using the bold strategy and about a 94% chance of losing all his money.
Which strategy gives Smith the better chance of getting out of jail?
Based off of the computed probabilities from the markov chain Smith has a slightly better chance to get out of jail using the bold strategy with a 6% chance of making bail vs a 2% chance of making bail with the timid strategy.