Markov Chain

Load Packages

library(knitr)
library(matlib)

Problem 1

Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6.
Find the probability that he wins 8 dollars before losing all of his money if
(a) he bets 1 dollar each time (timid strategy).
(b) he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).
(c) Which strategy gives Smith the better chance of getting out of jail?

Answer

1. Timid way of gambling:

number_of_gambles <- 100000

w = 0
l = 0
bail_amt = 8
for(gambles in 1:number_of_gambles) {
  smith = 1
  while(smith < bail_amt & smith > 0) {
    roll = runif(1) # randomize
    if(roll < 0.4) { smith = smith + 1 } 
    else { smith = smith - 1 }
    if(smith == 8) { w = w +1 }
    if(smith == 0) { l = l +1 }
  }
}
w/l

## [1] 0.02121054

2. Bold gambling:

w = 0
l = 0
bail_amt = 8
for(gambles in 1:number_of_gambles){
  smith = 1
  while(smith < bail_amt & smith > 0){
    roll = runif(1)
    if(roll < 0.4){ smith = smith + smith }
    else{ smith = smith - smith }
    if(smith == 8){ w = w +1 }
    if(smith == 0){ l = l +1 }
  }
}
w/l

## [1] 0.06778286

Please note that it’s gambling (achieved by runif(1) function), so the ratio w/l varies in every trial, but in each algorithm, the ratio revolves around (a) 0.02111669 and (b) 0.06777145.

3. As we see, if he gambles with a bolder strategy, then the probability of being bailed is higher:

Marker: 605-10