Homework 10

Week10 Problem Set -1
Solution

This problem is Gambler’s ruin problem. A gambler starts with an initial fortue. With each successive gamble he either wins or looses. These events are independent of the past with probabilities \(p\) and \(q = (1-p)\) respectively. The gambler’s objective is to reach a total fortune of \(N\) dollars, without getting ruined. He plays until his capital reaches the value \(N\) or the value \(0\). In Markov chains, these two values correspond to absorbing states.

Let \(P_i\) denote the probability that the gambler wins for state \(i\). From this we know that:
\(P_0 = 0\)
\(P_8 = 1\)

In general, the probability for each state is defined as:
\[ P_i = \left\{ \begin{array}{ll} \frac{1-(\frac{q}{p})^i}{1-(\frac{q}{p})^N} & \mbox{if $p \neq q$}\\ \frac{i}{N} & \mbox{if $p = q = 0.5$} \end{array} \right. \]

(a)
Question_10.1 
p = 0.4
q = 0.6
N = 8

P = c()
for (i in 0:N) {
  P[i+1] = round((1-(q/p)^i)/(1-(q/p)^N), 4)
}

P
## [1] 0.0000 0.0203 0.0508 0.0964 0.1649 0.2677 0.4219 0.6531 1.0000

\(P_{timid} = 0.0203\)

(b)

Here Smith bets the entire amount (up to 8). He must win each time or go broke and lose. If he wins, he will follow the following sequence:

\(1, 2, 4, 8\).

He starts with 1 dollar and must win 3 times in a row with probability \(0.4\). So, the probability of 3 successes in a row is

\(P_{bold} = (0.4)^3 = 0.064\)

(c)

The bold strategy gives Smith a probability \(0.064\) of getting out of jail, which is better than the timid strategy which gives him a probability \(0.02\).