For a Markov chain, the hitting probability from state \(i\) to set \(A\) is the probability of ever reaching the set A, starting from initial state \(i\).

The vector of hitting probabilities \(h_A=（h_{iA}:i \in S)\) is the minimal non-negative solution to the following equations

\[ h_{iA}=\left\{\begin{matrix} 1 & \text{ for } i \in A\\ \sum \limits_{j\in S} p_{ij}h_{jA} & \text{ for } i \notin A\\ \end{matrix}\right. \tag{1}\] Suppose we have the following transition diagram, we want to calculate the hitting probability \(h_{i4}\) i.e from all states \(,1,2,3,4\) to the state set \(\{4\}\)

By the equation \((1)\) we can write

\[h_{i4}=\left\{\begin{matrix} 1 & \text{ for } i =4 \\ \sum \limits_{j\in S} p_{ij}h_{j4} & \text{ for } i \neq 4\\ \end{matrix}\right.\]

To calculate the hitting probability, first let us write down the transition probability matrix of the above Markov chain. \[ \begin{pmatrix} 1 & 0 & 0 & 0\\ 0.5& 0 & 0.5 &0 \\ 0& 0.5& 0 & 0.5\\ 0 & 0 & 0 & 1.0 \end{pmatrix} \tag{2}\]

When \(i=1\)

\(h_{14}=p_{11}h_{14}+p_{12}h_{24}+p_{13}h_{34}+p_{14}h_{44}=h_{14} \tag{3}\)

since from \((2)\) we can see \(p_{11}=1\), \(p_{12}=0\), \(p_{13}=0\) and \(p_{14}=0\).

When \(i=2\)

\(h_{24}=p_{21}h_{14}+p_{22}h_{24}+p_{23}h_{34}+p_{24}h_{44}=0.5h_{14} +0.5h_{34}\tag{4}\)

since from \((2)\) we can see \(p_{21}=0.5\), \(p_{22}=0\), \(p_{23}=0.5\) and \(p_{24}=0\).

When \(i=3\)

\(h_{34}=p_{31}h_{14}+p_{32}h_{24}+p_{33}h_{34}+p_{34}h_{44}=0.5h_{14} +0.5h_{34}=0.5h_{24}+0.5h_{44}\tag{5}\)

since from \((2)\) we can see \(p_{31}=0\), \(p_{32}=0.5\), \(p_{33}=0\) and \(p_{34}=0.5\).

When \(i=4\)

\(h_{44}=p_{41}h_{14}+p_{42}h_{24}+p_{43}h_{34}+p_{44}h_{44}=h_{44}=1\tag{6}\)

since from \((2)\) we can see \(p_{41}=0\), \(p_{42}=0.5\), \(p_{43}=0\) and \(p_{44}=1\).

Now let us solve equations \((3),(4),(5),(6)\)

For equation \((3)\), the solution can be any number, sine the solution should be the minimal non-negative value,it will be set \(h_{14}=0\). Although from the transition diagram \((2)\), it is very easy for human brain to know that \(h_{14}=0\), but it will be very difficult for a computer to understand the diagram, therefore, we give the rule “minimal non-negative solution for the equation” then a computer can understand the rule and gives the solution as “0”.

From \((4)\) we can see \(h_{24}=0.5h_{34}\) combine with \((5)\) and \(h_{44}=1\) we get

\[h_{34}=\frac{2}{3}\] \[h_{24}=\frac{1}{2}\]

Therefore the hitting probability of \(h_4\) is

\[h_4=\begin{pmatrix} h_{14} \\ h_{24} \\ h_{34}\\ h_{44} \end{pmatrix}=\begin{pmatrix} 0 \\ \frac{1}{3} \\ \frac{2}{3}\\ 1 \end{pmatrix}\]