R Markdown

library(markovchain)
library(diagram)

Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6. Find the probability that he wins 8 dollars before losing all of his money if

(a)

he bets 1 dollar each time (timid strategy). p1 = chance of a win ps = chance of a loss S = Dollar value Smith starts on E = Dollar value Smith needs to get to be free

p1 <- 0.4
p2 <- 1-p1
S <- 1
E <- 8

timid <- (1 - (p2/p1)^S) / (1 - (p2/p1)^E)
timid
## [1] 0.02030135

With this timid strategy Smith has about a 2% chance winning the money needed to free himself.

(b)

Smith bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).

As discussed in class, Smith simply needs to win three times in a row with the more aggressive strategy

bold <- p1^3
bold
## [1] 0.064
# This can also be done using dbinom
binom <-dbinom(3,3,0.4)

bold == binom
## [1] TRUE

Transition matrix attempts

t_mat <- matrix(data = c(1, 0, 0, 0, 0, 0.6, 0, 0.4, 0, 0, 0.6, 0, 0, 0.4, 0, 0.6, 0, 0, 0, 0.4, 0, 0, 0, 0, 1), nrow = 5, byrow = TRUE)
t_mat
##      [,1] [,2] [,3] [,4] [,5]
## [1,]  1.0    0  0.0  0.0  0.0
## [2,]  0.6    0  0.4  0.0  0.0
## [3,]  0.6    0  0.0  0.4  0.0
## [4,]  0.6    0  0.0  0.0  0.4
## [5,]  0.0    0  0.0  0.0  1.0
statesSmith <- c("s1","s2","s3","s4","s5")

t_mat <- matrix(data = c(1, 0, 0, 0, 0, 0.6, 0, 0.4, 0, 0, 0.6, 0, 0, 0.4, 0, 0.6, 0, 0, 0, 0.4, 0, 0, 0, 0, 1), nrow = 5, ncol = 5, byrow = TRUE, dimnames = list(statesSmith))
t_mat
##    [,1] [,2] [,3] [,4] [,5]
## s1  1.0    0  0.0  0.0  0.0
## s2  0.6    0  0.4  0.0  0.0
## s3  0.6    0  0.0  0.4  0.0
## s4  0.6    0  0.0  0.0  0.4
## s5  0.0    0  0.0  0.0  1.0

Visualizing the Markov Chains using the markov chain and diagram packages

I think I messed up the steady states for this equation

mc <- new("markovchain",transitionMatrix=t_mat, states=statesSmith, name = "Smith's MC") 
mc
## Smith's MC 
##  A  5 - dimensional discrete Markov Chain defined by the following states: 
##  s1, s2, s3, s4, s5 
##  The transition matrix  (by rows)  is defined as follows: 
##     s1 s2  s3  s4  s5
## s1 1.0  0 0.0 0.0 0.0
## s2 0.6  0 0.4 0.0 0.0
## s3 0.6  0 0.0 0.4 0.0
## s4 0.6  0 0.0 0.0 0.4
## s5 0.0  0 0.0 0.0 1.0
plot(mc)

(c)

Which strategy gives Smith the better chance of getting out of jail?

The bold strategy gives Smith about three times the chance to get out of jail, jumping from around 2% to 6%.

bold > timid
## [1] TRUE