Assignment 10

Tom Detzel

4/14/2018


Gambler’s Ruin


Q. Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability 0.4 and loses A dollars with probability 0.6.

Find the probability that he wins 8 dollars before losing all of his money if

  1. he bets 1 dollar each time (timid strategy).
  2. he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).
  3. Which strategy gives Smith the better chance of getting out of jail?

A. In Markov parlance, zeroing out and winning $8 for freedom are ‘absorbing states’. We’re interested in which probability is greater.

It’s time for some matrix mania: Q, R, I, N and B!!

We can do it in R. Smith comes later.


***

# make our transition matrix and canonical matrix
# import transition matrix from data hand coded into excel

transientMat <- as.matrix(fread("markov.csv"))
row.names(transientMat) <- 0:8
colnames(transientMat) <- 0:8

# check row sums
# apply(poorbill, 1, sum)

# make a canonical matrix
canonicalMat <- as.matrix(fread("canonical.csv"))
row.names(canonicalMat) <- as.character(c(1:7,0,8))
colnames(canonicalMat) <- as.character(c(1:7,0,8))


Our transient matrix:

pander(transientMat)
  0 1 2 3 4 5 6 7 8
0 1 0 0 0 0 0 0 0 0
1 0.6 0 0.4 0 0 0 0 0 0
2 0 0.6 0 0.4 0 0 0 0 0
3 0 0 0.6 0 0.4 0 0 0 0
4 0 0 0 0.6 0 0.4 0 0 0
5 0 0 0 0 0.6 0 0.4 0 0
6 0 0 0 0 0 0.6 0 0.4 0
7 0 0 0 0 0 0 0.6 0 0.4
8 0 0 0 0 0 0 0 0 1


Our canonical matrix:

pander(canonicalMat)
  1 2 3 4 5 6 7 0 8
1 0 0.4 0 0 0 0 0 0.6 0
2 0.6 0 0.4 0 0 0 0 0 0
3 0 0.6 0 0.4 0 0 0 0 0
4 0 0 0.6 0 0.4 0 0 0 0
5 0 0 0 0.6 0 0.4 0 0 0
6 0 0 0 0 0.6 0 0.4 0 0
7 0 0 0 0 0 0.6 0 0 0.4
0 0 0 0 0 0 0 0 1 0
8 0 0 0 0 0 0 0 0 1


Our Q matrix:

QMat <- canonicalMat[1:7,1:7]
pander(QMat)
1 2 3 4 5 6 7
0 0.4 0 0 0 0 0
0.6 0 0.4 0 0 0 0
0 0.6 0 0.4 0 0 0
0 0 0.6 0 0.4 0 0
0 0 0 0.6 0 0.4 0
0 0 0 0 0.6 0 0.4
0 0 0 0 0 0.6 0



The R matrix:

RMat <- canonicalMat[1:7,8:9]
pander(RMat)
0 8
0.6 0
0 0
0 0
0 0
0 0
0 0
0 0.4


The N matrix:

IminusQ <- as.matrix(fread("IminusQ.csv"))
colnames(IminusQ) <- as.character(1:7)

# fundamental matrix
N <- round(solve(IminusQ),2)
colnames(N) <- as.character(1:7)
pander(N)
1 2 3 4 5 6 7
1.63 1.05 0.67 0.41 0.24 0.13 0.05
1.58 2.64 1.67 1.03 0.6 0.32 0.13
1.51 2.51 3.18 1.96 1.15 0.6 0.24
1.39 2.32 2.94 3.35 1.96 1.03 0.41
1.22 2.03 2.58 2.94 3.18 1.67 0.67
0.96 1.61 2.03 2.32 2.51 2.64 1.05
0.58 0.96 1.22 1.39 1.51 1.58 1.63


The B matrix:

B <- N%*%RMat
pander(B)
0 8
0.978 0.02
0.948 0.052
0.906 0.096
0.834 0.164
0.732 0.268
0.576 0.42
0.348 0.652


The t matrix:

c <- matrix(rep(1, 7), byrow=T, nrow=7)
t <- N%*%c 
pander(t)
4.18
7.97
11.15
13.4
14.29
13.12
8.87


Our critical matrix here is B. It contains the probabilities that Smith will zero out or chill out at every starting point.

The probability that Smith will get out of jail from his/her first bet is hopeless – 2 percent. But it’s not zero.

So if she/he bets $1 and wins, the probabilty of freedom more than doubles.

Score.

If Smith bets all $2 in his/her purse and wins, the odds of getting out on a $4 bet improve to 1 in 6. Shawshank Redemption territory.

Smith can choose the meager route, betting at $1, but the probability of escaping is low for each independent bet. The probability of zeroing out is higher.

Best to go for broke and hope for a winning random walk.

Speaking of which …

Bonus: 20 Random walks of 1,000 fair coin flips

# 20 Random Walks of a coin flip
plot.new()
par(mar=c(1,1,1,1))
par(mfrow=c(5,4))
for (i in 1:20){
    X <- sample(c('H','T'), 1000, replace=T, prob=c(0.5, 0.5))
    n <- 1:1000
    Sn <- numeric()
    for (i in 1:length(X)){
      y <- length(which(X[1:i]=='H'))-length(which(X[1:i]=='T'))
      Sn <- append(Sn, y)
    }
    plot(n, Sn, type='b', col='blue')
    abline(h=0, col='red', lwd=3)
}