Assignment #10
Sie Siong Wong
10/26/2020
Problem Set
Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6. Find the probability that he wins 8 dollars before losing all of his money if
- he bets 1 dollar each time (timid strategy).
- he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).
- Which strategy gives Smith the better chance of getting out of jail?
Answer:
Part (a)
Gambler’s ruin probabilities method:
\({ p }_{ z }=\frac { 1-{ (q/p) }^{ z } }{ 1-{ (q/p) }^{ M } }\) where \(q\neq p\), \(0\le z\le M\) and the quantity \({p}_{z}\) to be the probability that the gambler’s stake reaches M without ever having reached 0.
From the below calculation, the probability that he wins 8 dollars before losing all of his money if he bets 1 dollar each time is 0.0203, with this timid strategy.
z = 1
M = 8
p = 0.4
q = 0.6
p = (1-(q/p)^z) / (1-(q/p)^M)
p## [1] 0.02030135Absorbing Markov chain method:
Below is the evolution of Smith’s money diagram in 9 sates where state 0 is no money left to bet and state 8 wins the game.
Below is the Canonical form where the transition matrix can be turned into if there are absorbing states and transient states.
We can see from the absorption probabilities result in the below calculation that the chain reaches state 8 before reaching state 0 is 0.0203.
# Transition matrix in canonical form
p = matrix(c(0, 0.4, 0, 0, 0, 0, 0, 0.6, 0, 0.6, 0, 0.4, 0, 0, 0, 0, 0, 0, 0, 0.6, 0, 0.4, 0, 0, 0, 0, 0, 0, 0, 0.6, 0, 0.4, 0, 0, 0, 0, 0, 0, 0, 0.6, 0, 0.4, 0, 0, 0, 0, 0, 0, 0, 0.6, 0, 0.4, 0, 0, 0, 0, 0, 0, 0, 0.6, 0, 0, 0.4, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0.4, 0, 1), byrow=T, nrow=9, ncol=9, dimnames = list(c(1,2,3,4,5,6,7,0,8), c(1,2,3,4,5,6,7,0,8)))
p## 1 2 3 4 5 6 7 0 8
## 1 0.0 0.4 0.0 0.0 0.0 0.0 0.0 0.6 0.0
## 2 0.6 0.0 0.4 0.0 0.0 0.0 0.0 0.0 0.0
## 3 0.0 0.6 0.0 0.4 0.0 0.0 0.0 0.0 0.0
## 4 0.0 0.0 0.6 0.0 0.4 0.0 0.0 0.0 0.0
## 5 0.0 0.0 0.0 0.6 0.0 0.4 0.0 0.0 0.0
## 6 0.0 0.0 0.0 0.0 0.6 0.0 0.4 0.0 0.0
## 7 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.0 0.4
## 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0
## 8 0.0 0.0 0.0 0.0 0.0 0.0 0.4 0.0 1.0# Absorbing states
R = matrix(c(0.6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.4), byrow=T, nrow=7, ncol=2, dimnames = list(c(1,2,3,4,5,6,7), c(0,8)))
R## 0 8
## 1 0.6 0.0
## 2 0.0 0.0
## 3 0.0 0.0
## 4 0.0 0.0
## 5 0.0 0.0
## 6 0.0 0.0
## 7 0.0 0.4# Transient states
Q = matrix(c(0, 0.4, 0, 0, 0, 0, 0, 0.6, 0, 0.4, 0, 0, 0, 0, 0, 0.6, 0, 0.4, 0, 0, 0, 0, 0, 0.6, 0, 0.4, 0, 0, 0, 0, 0, 0.6, 0, 0.4, 0, 0, 0, 0, 0, 0.6, 0, 0.4, 0, 0, 0, 0, 0, 0.6, 0), byrow=T, nrow=7, ncol=7, dimnames = list(c(1,2,3,4,5,6,7), c(1,2,3,4,5,6,7)))
Q## 1 2 3 4 5 6 7
## 1 0.0 0.4 0.0 0.0 0.0 0.0 0.0
## 2 0.6 0.0 0.4 0.0 0.0 0.0 0.0
## 3 0.0 0.6 0.0 0.4 0.0 0.0 0.0
## 4 0.0 0.0 0.6 0.0 0.4 0.0 0.0
## 5 0.0 0.0 0.0 0.6 0.0 0.4 0.0
## 6 0.0 0.0 0.0 0.0 0.6 0.0 0.4
## 7 0.0 0.0 0.0 0.0 0.0 0.6 0.0# Identity matrix
I = diag(7)
I## [,1] [,2] [,3] [,4] [,5] [,6] [,7]
## [1,] 1 0 0 0 0 0 0
## [2,] 0 1 0 0 0 0 0
## [3,] 0 0 1 0 0 0 0
## [4,] 0 0 0 1 0 0 0
## [5,] 0 0 0 0 1 0 0
## [6,] 0 0 0 0 0 1 0
## [7,] 0 0 0 0 0 0 1# Compute the fundamental matrix of P, inverse of (I - Q)
N = solve(I-Q)
N## 1 2 3 4 5 6 7
## 1 1.6328311 1.054718 0.6693101 0.4123711 0.2410785 0.1268834 0.05075337
## 2 1.5820777 2.636796 1.6732752 1.0309278 0.6026963 0.3172086 0.12688343
## 3 1.5059477 2.509913 3.1792228 1.9587629 1.1451229 0.6026963 0.24107851
## 4 1.3917526 2.319588 2.9381443 3.3505155 1.9587629 1.0309278 0.41237113
## 5 1.2204600 2.034100 2.5765266 2.9381443 3.1792228 1.6732752 0.66931007
## 6 0.9635210 1.605868 2.0340999 2.3195876 2.5099128 2.6367962 1.05471848
## 7 0.5781126 0.963521 1.2204600 1.3917526 1.5059477 1.5820777 1.63283109# Absorption probabilities
B = N%*%R
B## 0 8
## 1 0.9796987 0.02030135
## 2 0.9492466 0.05075337
## 3 0.9035686 0.09643140
## 4 0.8350515 0.16494845
## 5 0.7322760 0.26772403
## 6 0.5781126 0.42188739
## 7 0.3468676 0.65313243Part (b)
If he uses the bold strategy, it means that he would bet all the money each time he wins. So, in this case there are only 4 steps ($1 -> $2 -> $4_>$8) to reach $8. Because this is bold strategy, we can also solve this problem by Binomial distribution besides the Absorbing Markov chain.
Binomial method:
Number of trials, n = 3
Number of successes, x = 3
\({ _{ n }{ C }_{ x } }{ p }^{ x }{ (1-p) }^{ n-x }={ _{ 3 }{ C }_{ 3 } }{ (0.4) }^{ 3 }{ (1-0.4) }^{ 3-3 }={ (0.4) }^{ 3 }=0.064\)
Absorbing Markov chain method:
With the bold strategy, he has to win in every step else he loses the game if he loses in any step. So, the transition matrix in canonical form can be written as below and from there we can again know the absorbing states (R) and transient states (Q) to compute the absorption probabilities. From the calculation result in below, we can see that the probability he wins 8 dollars before losing all of his money if he bets all the money each time is 0.064
# Transition matrix in canonical form
p = matrix(c(0, 0.4, 0, 0.6, 0, 0, 0, 0.4,0.6, 0, 0, 0, 0, 0.6, 0.4, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1), byrow=T, nrow=5, ncol=5, dimnames = list(c(1,2,4,0,8), c(1,2,4,0,8)))
p## 1 2 4 0 8
## 1 0 0.4 0.0 0.6 0.0
## 2 0 0.0 0.4 0.6 0.0
## 4 0 0.0 0.0 0.6 0.4
## 0 0 0.0 0.0 1.0 0.0
## 8 0 0.0 0.0 0.0 1.0# Absorbing states
R = matrix(c(.6, 0, .6, 0, 0.6, 0.4), byrow=T, nrow=3, ncol=2, dimnames = list(c(1,2,4), c(0,8)))
R## 0 8
## 1 0.6 0.0
## 2 0.6 0.0
## 4 0.6 0.4# Transient states
Q = matrix(c(0, 0.4, 0, 0, 0, 0.4, 0, 0, 0), byrow=T, nrow=3, ncol=3, dimnames = list(c(1,2,4), c(1,2,4)))
Q## 1 2 4
## 1 0 0.4 0.0
## 2 0 0.0 0.4
## 4 0 0.0 0.0# Identity matrix
I = diag(3)
I## [,1] [,2] [,3]
## [1,] 1 0 0
## [2,] 0 1 0
## [3,] 0 0 1# Compute the fundamental matrix of P, inverse of (I - Q)
N = solve(I-Q)
N## 1 2 4
## 1 1 0.4 0.16
## 2 0 1.0 0.40
## 4 0 0.0 1.00# Absorption probabilities
B = N%*%R
B## 0 8
## 1 0.936 0.064
## 2 0.840 0.160
## 4 0.600 0.400Part (c)
The bold strategy gives Smith approximately 3 times better chance of getting out of jail with the probability 0.064 than the smaller probability 0.0203 given by the timid strategy.