Assignment 10

Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6. Find the probability that he wins 8 dollars before losing all of his money if (a) he bets 1 dollar each time (timid strategy).

Reordering into the Canonical Form

# Get q and r
q <- timid[2:8, c(2:8)]
q

##     1   2   3   4   5   6   7
## 1 0.0 0.4 0.0 0.0 0.0 0.0 0.0
## 2 0.6 0.0 0.4 0.0 0.0 0.0 0.0
## 3 0.0 0.6 0.0 0.4 0.0 0.0 0.0
## 4 0.0 0.0 0.6 0.0 0.4 0.0 0.0
## 5 0.0 0.0 0.0 0.6 0.0 0.4 0.0
## 6 0.0 0.0 0.0 0.0 0.6 0.0 0.4
## 7 0.0 0.0 0.0 0.0 0.0 0.6 0.0

r <- timid[2:8, c(1,9)]
i = diag(7)
n = inv(i -q)
n

##                                                                           
## [1,] 1.6328311 1.054718 0.6693101 0.4123711 0.2410785 0.1268834 0.05075337
## [2,] 1.5820777 2.636796 1.6732752 1.0309278 0.6026963 0.3172086 0.12688343
## [3,] 1.5059477 2.509913 3.1792228 1.9587629 1.1451229 0.6026963 0.24107851
## [4,] 1.3917526 2.319588 2.9381443 3.3505155 1.9587629 1.0309278 0.41237113
## [5,] 1.2204599 2.034100 2.5765266 2.9381443 3.1792228 1.6732752 0.66931007
## [6,] 0.9635210 1.605868 2.0340999 2.3195876 2.5099128 2.6367962 1.05471848
## [7,] 0.5781126 0.963521 1.2204599 1.3917526 1.5059477 1.5820777 1.63283109

t = n %*% r
t

##              0          8
## [1,] 0.9796987 0.02030135
## [2,] 0.9492466 0.05075337
## [3,] 0.9035686 0.09643140
## [4,] 0.8350515 0.16494845
## [5,] 0.7322760 0.26772403
## [6,] 0.5781126 0.42188739
## [7,] 0.3468676 0.65313244

We are interested in the path of t[1,8] so the probability is 0.02.

he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).

In this case, he can go faster to eight dollars: 1 -> 2 -> 4 -> 8. But at each round he can end up at zero dollars (with a 0.6 probability).

dollars <- c('0', '1', '2', '4', '8')
rows <- c('1', '2', '3', '4', '5')
faster <- matrix(c(1,   0, 0,     0,  0,
                   0.6, 0, 0.4,   0,  0,
                   0.6, 0, 0,   0.4,  0,
                   0.6, 0, 0,     0,  0.4,
                   0,   0, 0,     0,   1), nrow=5, byrow=TRUE, dimnames = list(rows,dollars))

faster

##     0 1   2   4   8
## 1 1.0 0 0.0 0.0 0.0
## 2 0.6 0 0.4 0.0 0.0
## 3 0.6 0 0.0 0.4 0.0
## 4 0.6 0 0.0 0.0 0.4
## 5 0.0 0 0.0 0.0 1.0

Repeating the same process as before.

# Get q and r
q <- faster[2:4, c(2:4)]
q

##   1   2   4
## 2 0 0.4 0.0
## 3 0 0.0 0.4
## 4 0 0.0 0.0

r <- faster[2:4, c(1,5)]
i = diag(3)
n = inv(i -q)
n

##                
## [1,] 1 0.4 0.16
## [2,] 0 1.0 0.40
## [3,] 0 0.0 1.00

f = n %*% r
f

##          0     8
## [1,] 0.936 0.064
## [2,] 0.840 0.160
## [3,] 0.600 0.400

We see that [1,8] gives a probability of 0.064.

Which strategy gives Smith the better chance of getting out of jail?

Since the faster strategy has a higher probability (0.064 vs 0.203), he should bet with the second strategy.

Assignment 10

Cliff Lee

Reordering into the Canonical Form