Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6.

Find the probability that he wins 8 dollars before losing all of his money if…

  1. he bets 1 dollar each time (timid strategy). (b) he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy). (c) Which strategy gives Smith the better chance of getting out of jail?
Use Markov Chain & create a transition matrix

State 0: 0 dollar- lost all his money, State 1- has 1 dollar…state 8- has $8 and can get out of jail ( 9 by 9 matrix), where the rows = his current money and columns = his money in the next round

Transitional Prob- From state 1, he can win 1 dollar with probability 0.4 and go to state 2, or he can lose 1 dollar with probability 0.6 and stay in state 1. From state 2, 3, 4, 5, and 6, he can win one dollar and go to the next state or lose 1 dollar and go to the previous state. We need to calculate the probability of getting out on bail or reaching stat 8

(a) timid

# create transition probabilities with 9 by 9 matrix (fill it with zeros)
transition_matrix <- matrix(0, nrow = 9, ncol = 9)

# add in the transition probabilities
for (i in 0:8) {
  if (i < 8) {
    transition_matrix[i + 1, i + 2] <- 0.4  # P winning $1 
    transition_matrix[i + 1, max(0, i - 1)] <- 0.6  #p losing $1
  } else {
    transition_matrix[i + 1, i] <- 1  # game ends if/when he has $8 dollars
  }
}
transition_matrix #this can be done manually but takes time
##       [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
##  [1,]  0.0  0.4  0.0  0.0  0.0  0.0  0.0  0.0  0.0
##  [2,]  0.0  0.0  0.4  0.0  0.0  0.0  0.0  0.0  0.0
##  [3,]  0.6  0.0  0.0  0.4  0.0  0.0  0.0  0.0  0.0
##  [4,]  0.0  0.6  0.0  0.0  0.4  0.0  0.0  0.0  0.0
##  [5,]  0.0  0.0  0.6  0.0  0.0  0.4  0.0  0.0  0.0
##  [6,]  0.0  0.0  0.0  0.6  0.0  0.0  0.4  0.0  0.0
##  [7,]  0.0  0.0  0.0  0.0  0.6  0.0  0.0  0.4  0.0
##  [8,]  0.0  0.0  0.0  0.0  0.0  0.6  0.0  0.0  0.4
##  [9,]  0.0  0.0  0.0  0.0  0.0  0.0  0.0  1.0  0.0
#Next calculate p of winning $8
initial_state <- c(0, 1, 0, 0, 0, 0, 0, 0, 0) #initial=$1

# we want probability of reaching state 9 (8 dollars) from the initial state
final_state <- initial_state %*% (transition_matrix ^ 7)

# Probability of winning 8 dollars
p_winning_8 <- final_state[9]
p_winning_8
## [1] 0

\[ P = \frac{{1 - \left(\frac{q}{p}\right)^x}}{{1 - \left(\frac{q}{p}\right)^y}} \]

# Using Gambler's Ruin - I  did this because I saw other folks did not get 0 for this answer but maybe I did something wrong in the code chunk above 
stake <- 1
bail <- 8
p_win <- .4
p_lose<- .6

timid <- (1-(p_lose/p_win)^stake) / (1-(p_lose/p_win)^bail)
print (timid)
## [1] 0.02030135

(b) bold

The states on our chain are 0, 1, 2, 4, and 8 dollars.

# matrix is 5 by 5 states
state_matrix <- matrix(c(1,0,0,0,0,0.6,0,0.4,0,0,0.6,0,0,0.4,0,0.6,0,0,0,0.4,0,0,0,0,1), ncol=5,nrow=5, byrow = TRUE)
#initial state vector
initial <- matrix(c(0,1,0,0,0), ncol=5,nrow = 1,byrow = TRUE)
#solve for four bets
p1 <- initial%*%state_matrix
p2<-p1%*%state_matrix
p3<-p2%*%state_matrix
p4<-p3%*%state_matrix
p4
##       [,1] [,2] [,3] [,4]  [,5]
## [1,] 0.936    0    0    0 0.064

(c) better

The probability of the bold strategy is better at 6.4%.