Data 605 HW10

Question 1

Smith is in jail and has 1 dollar; he can get out on bail if he has 8 dollars. A guard agrees to make a series of bets with him. If Smith bets A dollars, he wins A dollars with probability .4 and loses A dollars with probability .6.
Find the probability that he wins 8 dollars before losing all of his money if:

he bets 1 dollar each time (timid strategy).
To find the probability, we first have to make a transition matrix of the possible outcomes of this strategy.
The range amount of money we can possibly have during this is $0 to 8 (0 being if we lose even the 1 dollar we started with and have no money to continue and 8 being if we successfully got enough to pay bail so we don’t need to try to win more money)
To calculate the matrix, we look at each possible outcome value and find the probability of us move (gain/lose) to a different outcome.
So if I have $0, I have a 100% probability of staying with $0 and no probability of move to another outcome.
If I have $1, I have a 60% chance of moving to $0 and a 40% chance of moving to $2.
If I have $2, I have a 60% chance of moving to $1 and a 40% chance of moving to $3.
Since we are only betting a dollar each time, the pattern is obvious with the last row (if we have $8, looking very similar to the $0 row)

possible_outcomes = c('0','1','2','3','4','5','6','7','8') 
#$0 to $8 to get out of jail
transit_matrixA = matrix(c(1,0,0,0,0,0,0,0,0,
                          0.6,0,0.4,0,0,0,0,0,0,
                          0,0.6,0,0.4,0,0,0,0,0,
                          0,0,0.6,0,0.4,0,0,0,0,
                          0,0,0,0.6,0,0.4,0,0,0,
                          0,0,0,0,0.6,0,0.4,0,0,
                          0,0,0,0,0,0.6,0,0.4,0,
                          0,0,0,0,0,0,0.6,0,0.4,
                          0,0,0,0,0,0,0,0,1), 
                        byrow = T, nrow = 9, 
                        dimnames = list(possible_outcomes,possible_outcomes))
print(transit_matrixA)

##     0   1   2   3   4   5   6   7   8
## 0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
## 1 0.6 0.0 0.4 0.0 0.0 0.0 0.0 0.0 0.0
## 2 0.0 0.6 0.0 0.4 0.0 0.0 0.0 0.0 0.0
## 3 0.0 0.0 0.6 0.0 0.4 0.0 0.0 0.0 0.0
## 4 0.0 0.0 0.0 0.6 0.0 0.4 0.0 0.0 0.0
## 5 0.0 0.0 0.0 0.0 0.6 0.0 0.4 0.0 0.0
## 6 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.4 0.0
## 7 0.0 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.4
## 8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0

Now that we have the transition matrix, we can calculate the probability of this strategy.

library(markovchain)

## Package:  markovchain
## Version:  0.9.5
## Date:     2023-09-24 09:20:02 UTC
## BugReport: https://github.com/spedygiorgio/markovchain/issues

markovA <- new("markovchain", states = possible_outcomes, transitionMatrix = transit_matrixA)
absorptionProbabilities(markovA)

##           0          8
## 1 0.9796987 0.02030135
## 2 0.9492466 0.05075337
## 3 0.9035686 0.09643140
## 4 0.8350515 0.16494845
## 5 0.7322760 0.26772403
## 6 0.5781126 0.42188739
## 7 0.3468676 0.65313243

The probability that we reach $8 using this timid strategy is represented by the first row of the absorption probabilities: 0.02030135 or 2.03%

he bets, each time, as much as possible but not more than necessary to bring his fortune up to 8 dollars (bold strategy).
Like the last strategy, we can make a transition matrix If I have $1, I have a 60% chance of moving to $0 and a 40% chance of moving to $2 (betting all $1).
If I have $2, I have a 60% chance of moving to $0 and a 40% chance of moving to $4 (betting all $2).
If I have $3, I have a 60% chance of moving to $0 and a 40% chance of moving to $6 (betting all $3).
If I have $4, I have a 60% chance of moving to $0 and a 40% chance of moving to $8 (betting all $4).
If I have $5, I have a 60% chance of moving to $2 and a 40% chance of moving to $8 (betting just enough for $8: $3).
If I have $6, I have a 60% chance of moving to $4 and a 40% chance of moving to $8 (betting just enough for $8: $2).
If I have $7, I have a 60% chance of moving to $6 and a 40% chance of moving to $8 (betting just enough for $8: $1).
(With $0 and $8, staying in its unchanging probabilities as they are our possible ending outcomes)

possible_outcomes = c('0','1','2','3','4','5','6','7','8') 
#$0 to $8 to get out of jail
transit_matrixB = matrix(c(1,0,0,0,0,0,0,0,0,
                          0.6,0,0.4,0,0,0,0,0,0,
                          0.6,0,0,0,0.4,0,0,0,0,
                          0.6,0,0,0,0,0,0.4,0,0,
                          0.6,0,0,0,0,0,0,0,0.4,
                          0,0,0.6,0,0,0,0,0,0.4,
                          0,0,0,0,0.6,0,0,0,0.4,
                          0,0,0,0,0,0,0.6,0,0.4,
                          0,0,0,0,0,0,0,0,1), 
                        byrow = T, nrow = 9, 
                        dimnames = list(possible_outcomes,possible_outcomes))
print(transit_matrixB)

##     0 1   2 3   4 5   6 7   8
## 0 1.0 0 0.0 0 0.0 0 0.0 0 0.0
## 1 0.6 0 0.4 0 0.0 0 0.0 0 0.0
## 2 0.6 0 0.0 0 0.4 0 0.0 0 0.0
## 3 0.6 0 0.0 0 0.0 0 0.4 0 0.0
## 4 0.6 0 0.0 0 0.0 0 0.0 0 0.4
## 5 0.0 0 0.6 0 0.0 0 0.0 0 0.4
## 6 0.0 0 0.0 0 0.6 0 0.0 0 0.4
## 7 0.0 0 0.0 0 0.0 0 0.6 0 0.4
## 8 0.0 0 0.0 0 0.0 0 0.0 0 1.0

Now that we have the transition matrix, we can calculate the probability of this strategy.

markovB <- new("markovchain", states = possible_outcomes, transitionMatrix = transit_matrixB)
absorptionProbabilities(markovB)

##       0     8
## 1 0.936 0.064
## 2 0.840 0.160
## 3 0.744 0.256
## 4 0.600 0.400
## 5 0.504 0.496
## 6 0.360 0.640
## 7 0.216 0.784

The probability that we reach $8 using this bold strategy is represented by the first row of the absorption probabilities: 0.064 or 6.4%

Which strategy gives Smith the better chance of getting out of jail?
Smith has a better chance with the bold strategy as it has a success rate of 6.4% compared to the timid strategy’s 2.03% success rate.
This may also be attributed to the less bets you would most likely play with the bold strategy. Less bets is better given your winning chances are lower than your losing chances.

Data 605 HW10

Ariann Chai

2024-03-25

Question 1