Background

The purpose of this assignment is to explore the application and properties of the Markov Chains and Random Walks.


Gambler’s Ruin

Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6. Find the probability that he wins 8 dollars before losing all of his money if

  1. he bets 1 dollar each time (timid strategy).

Apply what we know regarding the Gambler’s Ruin problem (p 488 - 489 of course text) and substituting our variables in, we can compute the probability of losing as:

\(P_L\) = \({({q \over p}^z - {q \over p}^M) \over (1 - {q \over p}^M)}\)

and the probability of winning as:

\(P_W\) = 1 - \(P_L\) = 1 - \({({q \over p}^z - {q \over p}^M) \over (1 - {q \over p}^M)}\)

Utilizing this equation and entering values based on the problem description, we get 0.0203 as calculated below in R:

#Initialize variables
p <- 0.4
q <- 0.6
M <- 8
z <- 1

#Calculate the probability of losing to calculate the probability of winning
P_L1 <- (((q/p)^z - (q/p)^M) / (1 - (q/p)^M))
P_W1 <- 1 - P_L1

#Display the probability of winning with a "timid" strategy
P_W1
## [1] 0.02030135
  1. he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).

Utilizing the same equation (noted above), we adapt our M value to be 3 since placing 3 consecutive winning bets would follow the following flow: $1 –> $2 –> $4 –> $8 and thus 3 consecutive winning bets is all Smith needs to place bail. Given this adaptation we get 0.211 as calculated below in R:

#Initialize variables
p <- 0.4
q <- 0.6
M <- 3
z <- 1

#Calculate the probability of losing to calculate the probability of winning
P_L1 <- (((q/p)^z - (q/p)^M) / (1 - (q/p)^M))
P_W1 <- 1 - P_L1

#Display the probability of winning with a "timid" strategy
P_W1
## [1] 0.2105263

Note: I wasn’t sure whether to apply the same equation or just p^3 (P = 0.064) … either way both calculations provide a greater probability value than the timid strategy.

  1. Which strategy gives Smith the better chance of getting out of jail?

As shown above in our calculations, the bold strategy gives Smith a better chance of getting out of jail.