Let’s revisit Coke and pepsi DTMC. Describe the following.
Suppose \(P(S_0=c)=0.6\) and \(P(S_0=p)=0.4\), then what is \(P(S_1=c)\)?
\[ a_0 = \begin{bmatrix} .6 & .4\end{bmatrix} \\ P = \begin{bmatrix} .7 & .3 \\ .5 & .5 \end{bmatrix} \\ P(S_1) = a_0 * P = \begin{bmatrix} .62 & .38\end{bmatrix} \\ then, P(S_1=c) = 0.62 \]
Suppose \(P(S_0=c)=0.6\) and \(P(S_0=p)=0.4\), then what is \(P(S_2=c)\)?
\[ a_0 = \begin{bmatrix} .6 & .4\end{bmatrix} \\ P = \begin{bmatrix} .7 & .3 \\ .5 & .5 \end{bmatrix} \\ P(S_1) = a_0 * P = \begin{bmatrix} .62 & .38\end{bmatrix} \\ P(S_2) = S_1 * P = \begin{bmatrix} .624 & .376\end{bmatrix} \\ then, P(S_2=c) = 0.624 \]
Suppose \(S_0=c\), then what is \(P(S_2=p|S_0=c)\)?
\[ a_0 = \begin{bmatrix} 1 & 0\end{bmatrix} \\ P = \begin{bmatrix} .7 & .3 \\ .5 & .5 \end{bmatrix} \\ P(S_1) = a_0 * P = \begin{bmatrix} .7 & .3\end{bmatrix} \\ P(S_2) = S_1 * P = \begin{bmatrix} .64 & .36\end{bmatrix} \\ then, P(S_2=p|S_0=c) = 0.36 \]
Suppose \(S_0=p\), then what is \(P(S_2=p)\)?
\[ a_0 = \begin{bmatrix} 0 & 1\end{bmatrix} \\ P = \begin{bmatrix} .7 & .3 \\ .5 & .5 \end{bmatrix} \\ P(S_1) = a_0 * P = \begin{bmatrix} .5 & .5\end{bmatrix} \\ P(S_2) = S_1 * P = \begin{bmatrix} .6 & .4\end{bmatrix} \\ then, P(S_2=p|S_0=c) = 0.4 \]
What is \(P(S_2=p|S_0=p)\)?
\[ a_0 = \begin{bmatrix} 0 & 1\end{bmatrix} \\ P = \begin{bmatrix} .7 & .3 \\ .5 & .5 \end{bmatrix} \\ P(S_1) = a_0 * P = \begin{bmatrix} .5 & .5\end{bmatrix} \\ P(S_2) = S_1 * P = \begin{bmatrix} .6 & .4\end{bmatrix} \\ then, P(S_2=p|S_0=c) = 0.4 \]
Suppose we are considering the soda problem with \[ P = \begin{bmatrix} 0.7 & 0.3 \\ 0.5 & 0.5 \end{bmatrix} \] and there are 20 people who drink coke today and 10 people who drink pepsi today. What will happen tomorrow?
\[ a_0 = \begin{bmatrix} 20 & 10 \end{bmatrix} \\ a_0 * P = \begin{bmatrix} 20 & 10 \end{bmatrix}\begin{bmatrix} 0.7 & 0.3 \\ 0.5 & 0.5 \end{bmatrix} = \begin{bmatrix} 19 & 11 \end{bmatrix} \\ \]
Again with the soda problem with \[ P = \begin{bmatrix} 0.7 & 0.3 \\ 0.5 & 0.5 \end{bmatrix} \]
Suppose that we happen to have a distribution of \(S_k\) on day \(k\) such as \(a_k = \left( \frac{5}{8}, \frac{3}{8} \right)\).
For each state of the following DTMC, tell whether the state is i) recurrent, ii) transient, or iii) absorbing.
\[ P = \begin{bmatrix} \frac{1}{2} & \frac{1}{2} & 0 \\ \frac{1}{2} & \frac{1}{4} & \frac{1}{4} \\ 0 & \frac{1}{3} & \frac{2}{3} \end{bmatrix} \]
\ (1), (2), and (3) are recurrent
For each state of the following DTMC, tell whether the state is i) recurrent, ii) transient, or iii) absorbing.
\[ P = \begin{bmatrix} 1 & 0 & 0 \\ \frac{1}{4} & \frac{1}{2} & \frac{1}{4} \\ 0 & 0 & 1 \end{bmatrix} \]
\ (1) and (3) is absorbing, but (2) is transient.
For each state of the following DTMC, tell whether the state is i) recurrent, ii) transient, or iii) absorbing.
\[ P = \begin{bmatrix} 1 & 0 & 0 \\ \frac{1}{3} & \frac{1}{2} & \frac{1}{6} \\ 0 & 0 & 1 \end{bmatrix} \]
\ (1) and (3) is absorbing, but (2) is transient.
For each state of the following DTMC, tell whether the state is i) recurrent, ii) transient, or iii) absorbing.
\[ P = \begin{bmatrix} \frac{1}{2} & \frac{1}{2} & 0 & 0\\ \frac{1}{2} & \frac{1}{2} & 0 & 0\\ \frac{1}{4} & \frac{1}{4} & \frac{1}{4} & \frac{1}{4} \\ 0 & 0 & 0 & 1 \end{bmatrix} \]
\ (1), (2), and (3) are transient, and (4) is absorbing.
For \(S = \{0, 1, 2, \ldots \}\) and \(p = \frac{1}{3}\) (#5 in the previous slide), find a stationary distribution using the flow balance equation.
Does #2 and #4 in the previous slide have stationary distribution? Why or why not?
Express the relevant transition probability matrix for \(\{Y_t, n \geq 1\}\).
\[ \mathbf{P} = \begin{pmatrix} (C, C) & (NC, C) & (C, NC) & (NC, NC) \\ 0.2 & 0 & 0.8 & 0 \\ 0.4 & & 0.6 & 0 \\ 0 & 0.6 & 0 & 0.4 \\ 0 & 0.8 & 0 & 0.2 \\ \end{pmatrix} \\ \]
\[ P = \begin{pmatrix} (C, C) & (NC, C) & (C, NC) & (NC, NC) \\ 0.2 & 0 & 0.8 & 0 \\ 0.4 & 0 & 0.6 & 0 \\ 0 & 0.6 & 0 & 0.4 \\ 0 & 0.8 & 0 & 0.2 \\ \end{pmatrix} \\ \]
\((\mathbb{P}[Y_{t+2} = (NC, C) | Y_t = (C, C)])\).
\[ \mathbf{P}^2 = \mathbf{P} \cdot \mathbf{P} = \begin{pmatrix} 0.04 & 0.48 & 0.16 & 0.32 \\ 0.08 & 0.56 & 0.24 & 0.12 \\ 0 & 0.68 & 0 & 0.32 \\ 0 & 0.76 & 0 & 0.24 \\ \end{pmatrix} \\ \mathbb{P}[Y_{t+2} = (NC, C) | Y_t = (C, C)] = 0.48 \]