Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6. Find the probability that he wins 8 dollars before losing all of his money if

  1. he bets 1 dollar each time (timid strategy).

I form a matrix with columns as a list of all possible money on hold for begin for each game, col<-($0, $1, $2, $3, $4, $5, $6, $7, $8), and in similar way, rows represent each stage probability of loss and win.

p=matrix(c(1,0,0,0,0,0,0,0,0,0.6,0,0.4,0,0,0,0,0,0,0,0.6,0,0.4,0,0,0,0,0,0,0,0.6,0,0.4,0,0,0,0,0,0,0,0.6,0,0.4,0,0,0,0,0,0,0,0.6,0,0.4,0,0,0,0,0,0,0,0.6,0,0.4,0,0,0,0,0,0,0,0.6,0,0.4,0,0,0,0,0,0,0,0,1),nrow=9,ncol=9,byrow=TRUE)
colnames(p)<-c("$0", "$1", "$2", "$3", "$4", "$5", "$6", "$7", "$8")
rownames(p) <- c("$0", "$1", "$2", "$3", "$4", "$5", "$6", "$7", "$8")
print(p)
##     $0  $1  $2  $3  $4  $5  $6  $7  $8
## $0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
## $1 0.6 0.0 0.4 0.0 0.0 0.0 0.0 0.0 0.0
## $2 0.0 0.6 0.0 0.4 0.0 0.0 0.0 0.0 0.0
## $3 0.0 0.0 0.6 0.0 0.4 0.0 0.0 0.0 0.0
## $4 0.0 0.0 0.0 0.6 0.0 0.4 0.0 0.0 0.0
## $5 0.0 0.0 0.0 0.0 0.6 0.0 0.4 0.0 0.0
## $6 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.4 0.0
## $7 0.0 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.4
## $8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0
P<-function(n){
  s=p
  for (i in as.numeric(1:(n-1))){s=s%*%p}
  print(format(s,scientific =FALSE))
  }
P(2)
##    $0     $1     $2     $3     $4     $5     $6     $7     $8    
## $0 "1.00" "0.00" "0.00" "0.00" "0.00" "0.00" "0.00" "0.00" "0.00"
## $1 "0.60" "0.24" "0.00" "0.16" "0.00" "0.00" "0.00" "0.00" "0.00"
## $2 "0.36" "0.00" "0.48" "0.00" "0.16" "0.00" "0.00" "0.00" "0.00"
## $3 "0.00" "0.36" "0.00" "0.48" "0.00" "0.16" "0.00" "0.00" "0.00"
## $4 "0.00" "0.00" "0.36" "0.00" "0.48" "0.00" "0.16" "0.00" "0.00"
## $5 "0.00" "0.00" "0.00" "0.36" "0.00" "0.48" "0.00" "0.16" "0.00"
## $6 "0.00" "0.00" "0.00" "0.00" "0.36" "0.00" "0.48" "0.00" "0.16"
## $7 "0.00" "0.00" "0.00" "0.00" "0.00" "0.36" "0.00" "0.24" "0.40"
## $8 "0.00" "0.00" "0.00" "0.00" "0.00" "0.00" "0.00" "0.00" "1.00"
P(20)
##    $0           $1           $2           $3           $4          
## $0 "1.00000000" "0.00000000" "0.00000000" "0.00000000" "0.00000000"
## $1 "0.94631205" "0.01015469" "0.00000000" "0.01619069" "0.00000000"
## $2 "0.88101215" "0.00000000" "0.03444072" "0.00000000" "0.03216496"
## $3 "0.78306231" "0.03642905" "0.00000000" "0.05840212" "0.00000000"
## $4 "0.69078111" "0.00000000" "0.07237116" "0.00000000" "0.06823218"
## $5 "0.55235932" "0.05391316" "0.00000000" "0.08711625" "0.00000000"
## $6 "0.42559637" "0.00000000" "0.07603080" "0.00000000" "0.07237116"
## $7 "0.23545195" "0.03317645" "0.00000000" "0.05391316" "0.00000000"
## $8 "0.00000000" "0.00000000" "0.00000000" "0.00000000" "0.00000000"
##    $5           $6           $7           $8          
## $0 "0.00000000" "0.00000000" "0.00000000" "0.00000000"
## $1 "0.01064951" "0.00000000" "0.00291261" "0.01378045"
## $2 "0.00000000" "0.01501843" "0.00000000" "0.03736374"
## $3 "0.03871833" "0.00000000" "0.01064951" "0.07273868"
## $4 "0.00000000" "0.03216496" "0.00000000" "0.13645059"
## $5 "0.05840212" "0.00000000" "0.01619069" "0.23201846"
## $6 "0.00000000" "0.03444072" "0.00000000" "0.39156096"
## $7 "0.03642905" "0.00000000" "0.01015469" "0.63087470"
## $8 "0.00000000" "0.00000000" "0.00000000" "1.00000000"
P(200)
##    $0                    $1                    $2                   
## $0 "1.00000000000000000" "0.00000000000000000" "0.00000000000000000"
## $1 "0.97969865131871836" "0.00000000016399923" "0.00000000000000000"
## $2 "0.94924662854279496" "0.00000000000000000" "0.00000000055992840"
## $3 "0.90356859437890957" "0.00000000059389375" "0.00000000000000000"
## $4 "0.83505154402392168" "0.00000000000000000" "0.00000000118778751"
## $5 "0.73227596849144205" "0.00000000089084063" "0.00000000000000000"
## $6 "0.57811260652898022" "0.00000000000000000" "0.00000000125983890"
## $7 "0.34686756358528997" "0.00000000055349741" "0.00000000000000000"
## $8 "0.00000000000000000" "0.00000000000000000" "0.00000000000000000"
##    $3                    $4                    $5                   
## $0 "0.00000000000000000" "0.00000000000000000" "0.00000000000000000"
## $1 "0.00000000026395278" "0.00000000000000000" "0.00000000017596852"
## $2 "0.00000000000000000" "0.00000000052790556" "0.00000000000000000"
## $3 "0.00000000095585757" "0.00000000000000000" "0.00000000063723838"
## $4 "0.00000000000000000" "0.00000000111985680" "0.00000000000000000"
## $5 "0.00000000143378636" "0.00000000000000000" "0.00000000095585757"
## $6 "0.00000000000000000" "0.00000000118778751" "0.00000000000000000"
## $7 "0.00000000089084063" "0.00000000000000000" "0.00000000059389375"
## $8 "0.00000000000000000" "0.00000000000000000" "0.00000000000000000"
##    $6                    $7                    $8                   
## $0 "0.00000000000000000" "0.00000000000000000" "0.00000000000000000"
## $1 "0.00000000000000000" "0.00000000004859236" "0.02030134802876870"
## $2 "0.00000000024885707" "0.00000000000000000" "0.05075337012051413"
## $3 "0.00000000000000000" "0.00000000017596852" "0.09643140325813221"
## $4 "0.00000000052790556" "0.00000000000000000" "0.16494845314052803"
## $5 "0.00000000000000000" "0.00000000026395278" "0.26772402796412165"
## $6 "0.00000000055992840" "0.00000000000000000" "0.42188739046346424"
## $7 "0.00000000000000000" "0.00000000016399923" "0.65313243421247869"
## $8 "0.00000000000000000" "0.00000000000000000" "1.00000000000000000"
  1. he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).

I form a matrix with columns as a list of all possible money on hold for begin for each game, col<-($0, $1, $2, $4, $8), and in similar way, rows represent each stage probability of loss and win.

p=matrix(c(1,0,0,0,0,0.6,0,0.4,0,0,0.6,0,0,0.4,0,0.6,0,0,0,0.4,0,0,0,0,1),nrow=5,ncol=5,byrow=TRUE)
colnames(p)<-c("$0", "$1", "$2", "$4", "$8") 
rownames(p) <-c("$0", "$1", "$2", "$4", "$8")
print(p)
##     $0 $1  $2  $4  $8
## $0 1.0  0 0.0 0.0 0.0
## $1 0.6  0 0.4 0.0 0.0
## $2 0.6  0 0.0 0.4 0.0
## $4 0.6  0 0.0 0.0 0.4
## $8 0.0  0 0.0 0.0 1.0
P(2)
##    $0     $1     $2     $4     $8    
## $0 "1.00" "0.00" "0.00" "0.00" "0.00"
## $1 "0.84" "0.00" "0.00" "0.16" "0.00"
## $2 "0.84" "0.00" "0.00" "0.00" "0.16"
## $4 "0.60" "0.00" "0.00" "0.00" "0.40"
## $8 "0.00" "0.00" "0.00" "0.00" "1.00"
P(3)
##    $0      $1      $2      $4      $8     
## $0 "1.000" "0.000" "0.000" "0.000" "0.000"
## $1 "0.936" "0.000" "0.000" "0.000" "0.064"
## $2 "0.840" "0.000" "0.000" "0.000" "0.160"
## $4 "0.600" "0.000" "0.000" "0.000" "0.400"
## $8 "0.000" "0.000" "0.000" "0.000" "1.000"
  1. Which strategy gives Smith the better chance of getting out of jail?

Comparing two stragies by observing each $0 and $8 columns, which are the probability of loss and win the game at different stages. If smith has 7 dollars on hand, he has 0.6531 chance to get out of jail. However, with bold strategy, even smith has 4 dollars on hand, he has 0.6 chance to loss his game.

With timid strategy, it has higher probability to loss the game when smith less than 4 dollars, but it has higher probability to win the game if smith has 4 dollars. Therefore, if smith has an option to choose to switch between two stratigies, I will say to begin with bold strategy until smith wins 4 dollars, then switch to timid strategy.