A certain calculating machine uses only the digits 0 and 1. It is supposed to transmit one of these digits through several stages. However, at every stage, there is a probability p that the digit that enters this stage will be changed when it leaves and a probability q = 1 - p that it won’t. Form a Markov chain to represent the process of transmission by taking as states the digits 0 and 1. What is the matrix of transition probabilities?
Given the outline of this problem there are only four possibilities for how the calculating machine can act.
There are also only two states:
So, the matrix of transition probabilities will look like this:
\[\begin{bmatrix} q & p \\ p & q \end{bmatrix}\] Or, when substituting for q: \[\begin{bmatrix} p - 1 & p \\ p & p - 1 \end{bmatrix}\]