Problem 1

Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6. Find the probability that he wins 8 dollars before losing all of his money if

  1. he bets 1 dollar each time (timid strategy).

The probability that he wins $8 before losing all his money if he starts with $1 is 2.03%

transition_matrix <- matrix(c(1,0,0,0,0,0,0,0,0,
                              0.6,0,0.4,0,0,0,0,0,0,
                              0,0.6,0,0.4,0,0,0,0,0,
                              0,0,0.6,0,0.4,0,0,0,0,
                              0,0,0,0.6,0,0.4,0,0,0,
                              0,0,0,0,0.6,0,0.4,0,0,
                              0,0,0,0,0,0.6,0,0.4,0,
                              0,0,0,0,0,0,0.6,0,0.4,
                              0,0,0,0,0,0,0,0,1),nrow=9,byrow=TRUE)

colnames(transition_matrix) <- c(0,1,2,3,4,5,6,7,8)
rownames(transition_matrix) <- c(0,1,2,3,4,5,6,7,8)

Q <- matrix(c(0,0.4,0,0,0,0,0,
              0.6,0,0.4,0,0,0,0,
              0,0.6,0,0.4,0,0,0,
              0,0,0.6,0,0.4,0,0,
              0,0,0,0.6,0,0.4,0,
              0,0,0,0,0.6,0,0.4,
              0,0,0,0,0,0.6,0),nrow=7,byrow=TRUE)
I <- diag(7)
N <- solve(I-Q)
r <- matrix(c(0.6,0,0,0,0,0,0,0,0,0,0,0,0,0.4),nrow=7,byrow=TRUE)
B = N %*% r
B[1,2]
## [1] 0.02030135

We could also solve this using the Gambler’s ruin probability function:

q = 0.6
p = 0.4
n = 1
N = 8

((1-(q/p)^n)/(1-(q/p)^N))
## [1] 0.02030135
  1. he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).

In this case, Smith has to always win the bet otherwise, he will lose all of this money. This route to winning only includes: * Bet #1: bet $1, win and then have $2
* Bet #2: bet $2, win and then have $4
* Bet #3: bet $4, win and then have $8

The probability of this is 6.4%

p = 0.4
win_prob = p^3
win_prob
## [1] 0.064
  1. Which strategy gives Smith the better chance of getting out of jail?
    Strategy b, the bolder strategy.