Question

Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6.

Find the probability that he wins 8 dollars before losing all of his money if

a. he bets 1 dollar each time (timid strategy).

library(markovchain)
# markov matrix - rows represent money he has, columns represent transitioning state
mat_a <- matrix(c(1,0,0,0,0,0,0,0,0,
               0.6,0,0.4,0,0,0,0,0,0,
               0,0.6,0,0.4,0,0,0,0,0,
               0,0,0.6,0,0.4,0,0,0,0,
               0,0,0,0.6,0,0.4,0,0,0,
               0,0,0,0,0.6,0,0.4,0,0,
               0,0,0,0,0,0.6,0,0.4,0,
               0,0,0,0,0,0,0.6,0,0.4,
               0,0,0,0,0,0,0,0,1), byrow = T, nrow = 9)

print(mat_a)
##       [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
##  [1,]  1.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
##  [2,]  0.6  0.0  0.4  0.0  0.0  0.0  0.0  0.0  0.0
##  [3,]  0.0  0.6  0.0  0.4  0.0  0.0  0.0  0.0  0.0
##  [4,]  0.0  0.0  0.6  0.0  0.4  0.0  0.0  0.0  0.0
##  [5,]  0.0  0.0  0.0  0.6  0.0  0.4  0.0  0.0  0.0
##  [6,]  0.0  0.0  0.0  0.0  0.6  0.0  0.4  0.0  0.0
##  [7,]  0.0  0.0  0.0  0.0  0.0  0.6  0.0  0.4  0.0
##  [8,]  0.0  0.0  0.0  0.0  0.0  0.0  0.6  0.0  0.4
##  [9,]  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  1.0
statesNames <- c("0","1","2","3","4","5","6","7","8")
timid <- new("markovchain", states = statesNames, byrow=TRUE, transitionMatrix = mat_a)

timid_probabilities <- absorptionProbabilities(timid)
print(timid_probabilities)
##           0          8
## 1 0.9796987 0.02030135
## 2 0.9492466 0.05075337
## 3 0.9035686 0.09643140
## 4 0.8350515 0.16494845
## 5 0.7322760 0.26772403
## 6 0.5781126 0.42188739
## 7 0.3468676 0.65313243
cat("The probability that he wins 8 dollars before losing all of his money betting 1 dollar at a time is", timid_probabilities[1,"8"], ".")
## The probability that he wins 8 dollars before losing all of his money betting 1 dollar at a time is 0.02030135 .

b. he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).

# markov matrix - rows represent money he has, columns represent transitioning state
mat_b <- matrix(c(1,0,0,0,0,0,0,0,0,
               0.6,0,0.4,0,0,0,0,0,0,
               0.6,0,0,0,0.4,0,0,0,0,
               0.6,0,0,0,0,0,0.4,0,0,
               0.6,0,0,0,0,0,0,0,0.4,
               0,0,0,0.6,0,0,0,0,0.4,
               0,0,0.6,0,0,0,0,0,0.4,
               0,0.6,0,0,0,0,0,0,0.4,
               0,0,0,0,0,0,0,0,1), byrow = T, nrow = 9)

print(mat_b)
##       [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
##  [1,]  1.0  0.0  0.0  0.0  0.0    0  0.0    0  0.0
##  [2,]  0.6  0.0  0.4  0.0  0.0    0  0.0    0  0.0
##  [3,]  0.6  0.0  0.0  0.0  0.4    0  0.0    0  0.0
##  [4,]  0.6  0.0  0.0  0.0  0.0    0  0.4    0  0.0
##  [5,]  0.6  0.0  0.0  0.0  0.0    0  0.0    0  0.4
##  [6,]  0.0  0.0  0.0  0.6  0.0    0  0.0    0  0.4
##  [7,]  0.0  0.0  0.6  0.0  0.0    0  0.0    0  0.4
##  [8,]  0.0  0.6  0.0  0.0  0.0    0  0.0    0  0.4
##  [9,]  0.0  0.0  0.0  0.0  0.0    0  0.0    0  1.0
statesNames <- c("0","1","2","3","4","5","6","7","8")
bold <- new("markovchain", states = statesNames, byrow=TRUE, transitionMatrix = mat_b)

bold_probabilities <- absorptionProbabilities(bold)
print(bold_probabilities)
##         0       8
## 1 0.93600 0.06400
## 2 0.84000 0.16000
## 3 0.80160 0.19840
## 4 0.60000 0.40000
## 5 0.48096 0.51904
## 6 0.50400 0.49600
## 7 0.56160 0.43840
cat("The probability that he wins 8 dollars before losing all of his money betting as much as possible at a time is", bold_probabilities[1,"8"], ".")
## The probability that he wins 8 dollars before losing all of his money betting as much as possible at a time is 0.064 .

c. Which strategy gives Smith the better chance of getting out of jail?

The bold strategy gives Smith a better chance of getting out of jail.