Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6.

Find the probability that he wins 8 dollars before losing all of his money if (a) he bets 1 dollar each time (timid strategy).

# we can solve using the Markov chain (X_n, n = 0,1,…) which represents the  Smith’s money. Let φ(i) represent the probaility that the chain reaches state 8 before reaching state 0 starting from state i.φ(i)=Pi(S8<S0)=P(S8<S0|X0=i).
p = 0.4
q = 0.6
r = q/p

for (i in seq(1, 7 , 1)){
  print ((1-r^i)/(1-r^8))
}
## [1] 0.02030135
## [1] 0.05075337
## [1] 0.0964314
## [1] 0.1649485
## [1] 0.267724
## [1] 0.4218874
## [1] 0.6531324

he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).

we are only concerned with the outputs of having 0, 1, 2, 4, and 8 dollars.

earnings <- matrix(c(1, .6, .6, .6, 0, 0, 0, 0, 0, 0, 0, .4, 0, 0, 0, 0, 0, .4, 0, 0, 0, 0, 0, .4, 1),nrow = 5, byrow=FALSE)
earnings
##      [,1] [,2] [,3] [,4] [,5]
## [1,]  1.0    0  0.0  0.0  0.0
## [2,]  0.6    0  0.4  0.0  0.0
## [3,]  0.6    0  0.0  0.4  0.0
## [4,]  0.6    0  0.0  0.0  0.4
## [5,]  0.0    0  0.0  0.0  1.0
i = matrix(c(0,1,0,0,0),nrow=1)
i
##      [,1] [,2] [,3] [,4] [,5]
## [1,]    0    1    0    0    0
p1 = i %*% earnings
p2 = p1 %*% earnings
p3 = p2 %*% earnings
p3
##       [,1] [,2] [,3] [,4]  [,5]
## [1,] 0.936    0    0    0 0.064
# resulting probability is .064 and this is the probability that Smith gets the $8.

C. Which strategy gives Smith the better chance of getting out of jail?

from the calculation of our probabilites in parts a and b, it would be wiser for Smith to take the bolder approach in order to have a better chance of getting out on bail and obtaining the 8 dollars.