library(markovchain)
## Warning: package 'markovchain' was built under R version 4.3.3
## Package:  markovchain
## Version:  0.9.5
## Date:     2023-09-24 09:20:02 UTC
## BugReport: https://github.com/spedygiorgio/markovchain/issues

Question

Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .

Find the probability that he wins 8 dollars before losing all of his money if

A

Timid strategy: He bets $1 each time

We will solve this using a transition matrix. The possible outcomes of a bet are the values whole values from $0 to $8. To build the table we will look at the possibility of moving to the next possible outcome.
Therefore;
If we have 0: 100% chance of staying at 0 because we have no money
If we have 1: 60% chance of moving to 0 and a 40% chance of moving to 2
If we have 2: 60% chance of moving to 1 and a 40% chance of moving to 3
If we have 3: 60% chance of moving to 2 and a 40% chance of moving to 4
If we have 4: 60% chance of moving to 3 and a 40% chance of moving to 5
If we have 5: 60% chance of moving to 4 and a 40% chance of moving to 6
If we have 6: 60% chance of moving to 5 and a 40% chance of moving to 7
If we have 7: 60% chance of moving to 6 and a 40% chance of moving to 8
If we have 8: 100% chance of staying at 8 because we win

The matrix is thus

bet_outcome <- as.character(seq(0, 8, 1))
transition_matrix <- matrix(
  c(
    1, 0, 0, 0, 0, 0, 0, 0, 0,
    0.6, 0, 0.4, 0, 0, 0, 0, 0, 0,
    0, 0.6, 0, 0.4, 0, 0, 0, 0, 0, 
    0, 0, 0.6, 0, 0.4, 0, 0, 0, 0, 
    0, 0, 0, 0.6, 0, 0.4, 0, 0, 0,
    0, 0, 0, 0, 0.6, 0, 0.4, 0, 0,
    0, 0, 0, 0, 0, 0.6, 0, 0.4, 0, 
    0, 0, 0, 0, 0, 0, 0.6, 0, 0.4, 
    0, 0, 0, 0, 0, 0, 0, 0, 1
  ), 
  byrow = TRUE,
  nrow = 9, 
  dimnames = list(bet_outcome, bet_outcome)
)

probabilityA <- new('markovchain', states = bet_outcome, transitionMatrix = transition_matrix)
absorptionProbabilities(probabilityA)
##           0          8
## 1 0.9796987 0.02030135
## 2 0.9492466 0.05075337
## 3 0.9035686 0.09643140
## 4 0.8350515 0.16494845
## 5 0.7322760 0.26772403
## 6 0.5781126 0.42188739
## 7 0.3468676 0.65313243

From this we see that starting with $1, the probability that Smith would win $8 and his freedom is 0.0203 or 2.03%.

B

Bold strategy: He bets as much as possible but not more than necessary

The transition matrix for this scenario is;
If we have 0: 100% chance of staying at 0 because we have no money
If we have 1: 60% chance of moving to 0 and a 40% chance of moving to 2
If we have 2: 60% chance of moving to 0 and a 40% chance of moving to 4
If we have 3: 60% chance of moving to 0 and a 40% chance of moving to 6
If we have 4: 60% chance of moving to 0 and a 40% chance of moving to 8
If we have 5: 60% chance of moving to 2 and a 40% chance of moving to 8 (most he can bet is $3)
If we have 6: 60% chance of moving to 4 and a 40% chance of moving to 8 (most he can bet is $2)
If we have 7: 60% chance of moving to 6 and a 40% chance of moving to 8 (most he can bet is $1)
If we have 8: 100% chance of staying at 8 because we win

The matrix is thus

bet_outcome <- as.character(seq(0, 8, 1))
transition_matrix <- matrix(
  c(
    1, 0, 0, 0, 0, 0, 0, 0, 0,
    0.6, 0, 0.4, 0, 0, 0, 0, 0, 0,
    0.6, 0, 0, 0, 0.4, 0, 0, 0, 0,  
    0.6, 0, 0, 0, 0, 0, 0.4, 0, 0, 
    0.6, 0, 0, 0, 0, 0, 0, 0, 0.4,
    0, 0, 0.6, 0, 0, 0, 0, 0, 0.4, 
    0, 0, 0, 0.6, 0, 0, 0, 0, 0.4,
    0, 0, 0, 0, 0, 0, 0.6, 0, 0.4, 
    0, 0, 0, 0, 0, 0, 0, 0, 1
  ), 
  byrow = TRUE,
  nrow = 9, 
  dimnames = list(bet_outcome, bet_outcome)
)

probabilityB <- new('markovchain', states = bet_outcome, transitionMatrix = transition_matrix)
absorptionProbabilities(probabilityB)
##           0         8
## 1 0.9360000 0.0640000
## 2 0.8400000 0.1600000
## 3 0.7894737 0.2105263
## 4 0.6000000 0.4000000
## 5 0.5040000 0.4960000
## 6 0.4736842 0.5263158
## 7 0.2842105 0.7157895

From this we see that starting with $1, the probability that Smith would win $8 and his freedom is 0.064 or 6.4%.

C

The strategy that will Smith the best chance of getting out of jail is the bold strategy.