Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6.

Find the probability that he wins 8 dollars before losing all of his money if:

  1. He bets 1 dollar each time (timid strategy).
Gambler’s Ruin Formula

\[ P = (1-(q/p)^{s})/(1-(q/p)^{M})\]

p <- 0.4 # prob of winning
q <- 0.6 # prob of losing
M <- 8 # desired value to reach

s <- 1 # starting stake or bet

P = (1 - (q/p)^s)/(1-(q/p)^M)
print(paste0("Probability of reaching 8 dollars with a stake/Bet of 1: ", P))
## [1] "Probability of reaching 8 dollars with a stake/Bet of 1: 0.0203013481363997"
  1. He bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).
With a bold strategy, the gambler bets the maximum bet every time in order to reach the target of 8 as fast as possible. Desired progressive bets (states) to reach the target 8 are: 1 (starting bet), then 2 and finally 4. Total of 3 states
Using Markov Chains
The probability that the chain is in state Si after n steps is the ith entry in the vector

\[ u^{n} = uP^{n}\] \[ Probability \space vector \space u; \] \[ Transition \space Matrix \space P; \]

transition_matrix <- matrix(c(1,0,0,0,0,0.6,0,0.4,0,0,0.6,0,0,0.4,0,0.6,0,0,0,0.4,0,0,0,0,1), ncol=5,nrow=5, byrow = TRUE)
transition_matrix
##      [,1] [,2] [,3] [,4] [,5]
## [1,]  1.0    0  0.0  0.0  0.0
## [2,]  0.6    0  0.4  0.0  0.0
## [3,]  0.6    0  0.0  0.4  0.0
## [4,]  0.6    0  0.0  0.0  0.4
## [5,]  0.0    0  0.0  0.0  1.0
u0 <- matrix(c(0,1,0,0,0), ncol=5,nrow = 1,byrow = TRUE) # initial state
u0
##      [,1] [,2] [,3] [,4] [,5]
## [1,]    0    1    0    0    0
u_s1 <- u0%*%transition_matrix # bet of 1
u_s2 <- u_s1%*%transition_matrix # bet of 2
u_s3 <- u_s2%*%transition_matrix# bet of 4

u_s3
##       [,1] [,2] [,3] [,4]  [,5]
## [1,] 0.936    0    0    0 0.064
u_s3[5]
## [1] 0.064
  1. Which strategy gives Smith the better chance of getting out of jail?
P(Timid_Strategy) = 0.0203
P(Bold_Strategy) = 0.064
The Bold strategy presents a better chance (3x) for the gambler of getting out of jail faster