DATA605 Homework 10

1.

Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars.
A guard agrees to make a series of bets with him. If Smith bets A dollars,
he wins A dollars with probability .4 and loses A dollars with probability .6.
Find the probability that he wins 8 dollars before losing all of his money if

library(markovchain)
## Warning: package 'markovchain' was built under R version 3.3.3
## Package:  markovchain
## Version:  0.6.9.8-1
## Date:     2017-08-15
## BugReport: http://github.com/spedygiorgio/markovchain/issues
library(igraph)
## Warning: package 'igraph' was built under R version 3.3.3
## 
## Attaching package: 'igraph'
## The following objects are masked from 'package:stats':
## 
##     decompose, spectrum
## The following object is masked from 'package:base':
## 
##     union
trans.mat <- matrix(c(0.4, 0.6, 0.6, 0.4), 2,2, byrow = TRUE)
dtmcA <- new("markovchain", transitionMatrix = trans.mat, states = c("Wins", "Loses"), name = "Smith's Markov Chain")
dtmcA
## Smith's Markov Chain 
##  A  2 - dimensional discrete Markov Chain defined by the following states: 
##  Wins, Loses 
##  The transition matrix  (by rows)  is defined as follows: 
##       Wins Loses
## Wins   0.4   0.6
## Loses  0.6   0.4
plot(dtmcA)

initialState <- c(1,1)
steps <- 7
finalState <- initialState*dtmcA^steps
finalState
##      Wins Loses
## [1,]    1     1
steadyStates(dtmcA)
##      Wins Loses
## [1,]  0.5   0.5
initialState <- c(1,1)
steps <- 4
finalState <- initialState*dtmcA^steps
finalState
##      Wins Loses
## [1,]    1     1
steadyStates(dtmcA)
##      Wins Loses
## [1,]  0.5   0.5

(a) he bets 1 dollar each time (timid strategy).

Using the timid strategy, the probability that Smith wins 8 dollars before losing all his money is 0.0203.

(b) he bets, each time, as much as possible but not more than necessary to

bring his fortune up to 8 dollars (bold strategy).

Using the bold strategy, the probability that Smith wins 8 dollars before losing all his money is 0.064.

(c) Which strategy gives Smith the better chance of getting out of jail?

The bold strategy gives Smith the better chance of getting out of jail, since 0.064 is greater than 0.0203.