HW 10

Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6. Find the probability that he wins 8 dollars before losing all of his money if

1. he bets 1 dollar each time (timid strategy).

2. he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).

3. Which strategy gives Smith the better chance of getting out of jail?

library(markovchain)

1. Betting $1 at a time.

statesNames=c("0","1","2","3","4","5","6","7","8")
mymat=matrix(c(1,0,0,0,0,0,0,0,0,
               0.6,0,0.4,0,0,0,0,0,0,
               0,0.6,0,0.4,0,0,0,0,0,
               0,0,0.6,0,0.4,0,0,0,0,
               0,0,0,0.6,0,0.4,0,0,0,
               0,0,0,0,0.6,0,0.4,0,0,
               0,0,0,0,0,0.6,0,0.4,0,
               0,0,0,0,0,0,0.6,0,0.4,
               0,0,0,0,0,0,0,0,1), byrow=T, nrow=9,
             dimnames=list(statesNames,statesNames))

mymat

##     0   1   2   3   4   5   6   7   8
## 0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
## 1 0.6 0.0 0.4 0.0 0.0 0.0 0.0 0.0 0.0
## 2 0.0 0.6 0.0 0.4 0.0 0.0 0.0 0.0 0.0
## 3 0.0 0.0 0.6 0.0 0.4 0.0 0.0 0.0 0.0
## 4 0.0 0.0 0.0 0.6 0.0 0.4 0.0 0.0 0.0
## 5 0.0 0.0 0.0 0.0 0.6 0.0 0.4 0.0 0.0
## 6 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.4 0.0
## 7 0.0 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.4
## 8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0

markovB <- new("markovchain", states = statesNames, transitionMatrix = mymat, name = "A markovchain Object")

absorptionProbabilities(markovB)

##           0          8
## 1 0.9796987 0.02030135
## 2 0.9492466 0.05075337
## 3 0.9035686 0.09643140
## 4 0.8350515 0.16494845
## 5 0.7322760 0.26772403
## 6 0.5781126 0.42188739
## 7 0.3468676 0.65313243

#meanAbsorptionTime(markovB)

He has a 2% chance that he will get out of jail.

2. Betting all he has each time.

statesNames2=c("0","2","4","8")
mymat2=matrix(c(1, 0, 0, 0,
               0.6, 0, 0.4, 0,
               0, 0.6, 0, 0.4,
               0, 0, 0, 1), byrow=T, nrow=4,
             dimnames=list(statesNames2,statesNames2))

mymat2

##     0   2   4   8
## 0 1.0 0.0 0.0 0.0
## 2 0.6 0.0 0.4 0.0
## 4 0.0 0.6 0.0 0.4
## 8 0.0 0.0 0.0 1.0

markovB2 <- new("markovchain", states = statesNames2, transitionMatrix = mymat2, name = "A markovchain Object2")

absorptionProbabilities(markovB2)

##           0         8
## 2 0.7894737 0.2105263
## 4 0.4736842 0.5263158

#meanAbsorptionTime(markovB2)

He has a 21% chance that will get out of jail.

3. Betting all he has but not more than necessary gives him the better chance.