Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6. Find the probability that he wins 8 dollars before losing all of his money if (a) he bets 1 dollar each time (timid strategy). (b) he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy). (c) Which strategy gives Smith the better chance of getting out of jail?
Lets represent Smith fortunes: State 0: Smith has lost all his money. State 1: Smith has 1 dollar. State 2: Smith has 2 dollars. State 8: Smith has 8 dollars and can get out on bail.
The transition probabilities between states depend on whether Smith wins or loses the bet. Let p be the probability of winning a bet (0.4), and q be the probability of losing a bet (0.6).
library(markovchain)## Package: markovchain
## Version: 0.9.5
## Date: 2023-09-24 09:20:02 UTC
## BugReport: https://github.com/spedygiorgio/markovchain/issuesoutcomes <- c('0','1','2','3','4','5','6','7','8')
# Define transition matrix for the timid strategy
trans_matrix_1 = matrix(c(1,0,0,0,0,0,0,0,0,
0.6,0,0.4,0,0,0,0,0,0,
0,0.6,0,0.4,0,0,0,0,0,
0,0,0.6,0,0.4,0,0,0,0,
0,0,0,0.6,0,0.4,0,0,0,
0,0,0,0,0.6,0,0.4,0,0,
0,0,0,0,0,0.6,0,0.4,0,
0,0,0,0,0,0,0.6,0,0.4,
0,0,0,0,0,0,0,0,1),
byrow = T, nrow = 9,
dimnames = list(outcomes,outcomes))
trans_matrix_1## 0 1 2 3 4 5 6 7 8
## 0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
## 1 0.6 0.0 0.4 0.0 0.0 0.0 0.0 0.0 0.0
## 2 0.0 0.6 0.0 0.4 0.0 0.0 0.0 0.0 0.0
## 3 0.0 0.0 0.6 0.0 0.4 0.0 0.0 0.0 0.0
## 4 0.0 0.0 0.0 0.6 0.0 0.4 0.0 0.0 0.0
## 5 0.0 0.0 0.0 0.0 0.6 0.0 0.4 0.0 0.0
## 6 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.4 0.0
## 7 0.0 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.4
## 8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0# Create markovchain objects
mc_timid <- new("markovchain", transitionMatrix = trans_matrix_1)
absorptionProbabilities(mc_timid)## 0 8
## 1 0.9796987 0.02030135
## 2 0.9492466 0.05075337
## 3 0.9035686 0.09643140
## 4 0.8350515 0.16494845
## 5 0.7322760 0.26772403
## 6 0.5781126 0.42188739
## 7 0.3468676 0.65313243The proability when he bet each is time is 0.02030135
trans_matrix_2 = matrix(c(1,0,0,0,0,0,0,0,0,
0.6,0,0.4,0,0,0,0,0,0,
0.6,0,0,0,0.4,0,0,0,0,
0.6,0,0,0,0,0,0.4,0,0,
0.6,0,0,0,0,0,0,0,0.4,
0,0,0.6,0,0,0,0,0,0.4,
0,0,0,0,0.6,0,0,0,0.4,
0,0,0,0,0,0,0.6,0,0.4,
0,0,0,0,0,0,0,0,1),
byrow = T, nrow = 9,
dimnames = list(outcomes,outcomes))
mc_bold <- new("markovchain", transitionMatrix = trans_matrix_2)
absorptionProbabilities(mc_bold)## 0 8
## 1 0.936 0.064
## 2 0.840 0.160
## 3 0.744 0.256
## 4 0.600 0.400
## 5 0.504 0.496
## 6 0.360 0.640
## 7 0.216 0.784The probability that he brings his fortune up to 8 dollars is 0.064 on his first try.
The bold tratejy give Smith a better chance of getting out of jail. bold is 6.04% compare to 2.03%.