The purpose of this assignment is to explore the application and properties of the Markov Chains and Random Walks.
Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6. Find the probability that he wins 8 dollars before losing all of his money if
Apply what we know regarding the Gambler’s Ruin problem (p 488 - 489 of course text) and substituting our variables in, we can compute the probability of losing as:
\(P_L\) = \({({q \over p}^z - {q \over p}^M) \over (1 - {q \over p}^M)}\)
and the probability of winning as:
\(P_W\) = 1 - \(P_L\) = 1 - \({({q \over p}^z - {q \over p}^M) \over (1 - {q \over p}^M)}\)
Utilizing this equation and entering values based on the problem description, we get 0.0203 as calculated below in R:
#Initialize variables
p <- 0.4
q <- 0.6
M <- 8
z <- 1
#Calculate the probability of losing to calculate the probability of winning
P_L1 <- (((q/p)^z - (q/p)^M) / (1 - (q/p)^M))
P_W1 <- 1 - P_L1
#Display the probability of winning with a "timid" strategy
P_W1
## [1] 0.02030135
Utilizing the same equation (noted above), we adapt our M value to be 3 since placing 3 consecutive winning bets would follow the following flow: $1 –> $2 –> $4 –> $8 and thus 3 consecutive winning bets is all Smith needs to place bail. Given this adaptation we get 0.211 as calculated below in R:
#Initialize variables
p <- 0.4
q <- 0.6
M <- 3
z <- 1
#Calculate the probability of losing to calculate the probability of winning
P_L1 <- (((q/p)^z - (q/p)^M) / (1 - (q/p)^M))
P_W1 <- 1 - P_L1
#Display the probability of winning with a "timid" strategy
P_W1
## [1] 0.2105263
Note: I wasn’t sure whether to apply the same equation or just p^3 (P = 0.064) … either way both calculations provide a greater probability value than the timid strategy.
As shown above in our calculations, the bold strategy gives Smith a better chance of getting out of jail.