Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6. Find the probability that he wins 8 dollars before losing all of his money if
he bets 1 dollar each time (timid strategy).
he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).
Which strategy gives Smith the better chance of getting out of jail?
Solution
The gambler aim to reach a total fortune of N without first getting ruined (running out of money). when the gambler starts with i where 0 < i < N,
Pr(win) = p = 0.4, Pr(lose) = q = 1 ??? p = 0.6
When he win, his stock becomes i+A, the probability to win is \(P_{i+A}\). When he lose, his stock becomes i+A, the probability to win is \(P_{i-A}\). \(P_0 = 0\), \(P_N = 1\)
So the probability that he win is: \[\begin{equation} \begin{split} P_i &= pP_{i+A} + qP_{i-A}\\ pP_i + qP_i &= pP_{i+A} + qP_{i-A}\\ P_{i+A} - P_i &= \frac{p}{q}(Pi - P_{i-A}) \end{split} \end{equation}\]If the gambler bet A dollars,
\[\begin{equation} \begin{split} P_{2A} - P_A &= (q/p)(P_A - P_0) \\ &= (q/p)P_A \\ \end{split} \end{equation}\] \[\begin{equation} \begin{split} P_{(i+1)A} - P_{iA} = (\frac{p}{q})^i P_A \end{split} \end{equation}\] \[\begin{equation} \begin{split} P_{(i+1)A} - P_A &= \displaystyle\sum_{k=1}^{i}(P_{(k+1)A} - P_{kA})\\ &=\displaystyle\sum_{k=1}^{i}(\frac{q}{p})^k P_A \end{split} \end{equation}\] \[\begin{equation} \begin{split} P_{(i+1)A} = \displaystyle\sum_{k=0}^{i}(\frac{q}{p})^k P_A \end{split} \end{equation}\]because \(p \neq q\)
\[\begin{equation} P_{(i+1)A} = P_A\frac{1-(\frac{q}{p})^{i+1}}{1-(\frac{q}{p})} \end{equation}\]Because \(P_N = 1\), when (i+1)A=N, then (i+1)=N/A
\[\begin{equation} 1 = P_A\frac{1-(\frac{q}{p})^{N/A}}{1-(\frac{q}{p})} \end{equation}\]So we know
\[\begin{equation} P_A = \frac{1-(\frac{q}{p})}{1-(\frac{q}{p})^{N/A}} \end{equation}\]So
\[\begin{equation} \begin{split} P_{iA} &= P_A\frac{1-(\frac{q}{p})^i}{1-(\frac{q}{p})}\\ &=\frac{1-(\frac{q}{p})}{1-(\frac{q}{p})^{N/A}} \times \frac{1-(\frac{q}{p})^i}{1-(\frac{q}{p})}\\ &=\frac{1-(\frac{q}{p})^i}{1-(\frac{q}{p})^{N/A}} \end{split} \end{equation}\](a) he bets 1 dollar each time (timid strategy).
When iA = 1, N = 8, A = 1, then i = 1
\[\begin{equation} P_1 = \frac{1-\frac{0.6}{0.4}}{1-(\frac{0.6}{0.4})^8}\\ \end{equation}\]p_a <- (1-(0.6/0.4))/(1-(0.6/0.4)^8)
cat("The probability that Smith wins 8 dollars before losing all of his money if he bets 1 dollar each time is: ", p_a)## The probability that Smith wins 8 dollars before losing all of his money if he bets 1 dollar each time is: 0.02030135(b) he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).
The Markav Chain will become: i=1 to i=2 to i=4 to i=8.
start from iA=1,N=2, A=1, then i = 1
\(P_1=\frac{1-\frac{0.6}{0.4}}{1-(\frac{0.6}{0.4})^2}\)
if win, then iA=2,N=4,A=2, then i = 1
\(P_2=\frac{1-(\frac{0.6}{0.4})^1}{1-(\frac{0.6}{0.4})^{4/2}}\)
if win, then iA=4,N=8,A=4, then i = 1 \(P_4=\frac{1-(\frac{0.6}{0.4})^1}{1-(\frac{0.6}{0.4})^{8/4}}\)
p_1 <- (1-(0.6/0.4))/(1-(0.6/0.4)^2)
p_2 <- (1-(0.6/0.4))/(1-(0.6/0.4)^2)
p_4 <- (1-(0.6/0.4))/(1-(0.6/0.4)^2)
cat("The probability that Smith wins 8 dollars before losing all of his money if he bets 1 dollar each time is: ", p_1*p_2*p_4)## The probability that Smith wins 8 dollars before losing all of his money if he bets 1 dollar each time is: 0.064simulation
win <- 0
size <- 10000
for(i in 1:size){
s <- 1
while (s<8 & s>0){
result <- runif(1)
if (result <0.4){
s <-s+s
}
else{
s <-s-s
}
if (s>=8){
win <- win + 1
}
}
}
win/size## [1] 0.072(c) Which strategy gives Smith the better chance of getting out of jail?
Because the probability of winning of bold strategy is larger than that of timid strategy, the bold strategy gives Smith the better chance of getting out of jail.