Recall the invariant (unconditional) probability \(\mu\) for a Markov matrix \(\mathbf{P}\): \(\mu=\mathbf{P}\mu\). \(\mu\) is an eignvector associated with the unit eigenvalue. The other eigenvectors are in fact in a “decaying mode”.
Recall that the starting matrix \(\mathbf{P}\) is unrecognizable after a few steps \[ \mathbf{P} = \left[\begin{array}{cc} 0.8 & 0.3\\ 0.2 & 0.7 \end{array}\right], \mathbf{P}^2 = \left[\begin{array}{cc} 0.7 & 0.45\\ 0.3 & 0.55 \end{array}\right],\dots \mathbf{P}^{30} = \left[\begin{array}{cc} 0.6 & 0.6\\ 0.4 & 0.4 \end{array}\right].\]
The initial \(\mathbf{P}\) is different from the stationary \(\mathbf{P}^{30}\). \(\mathbf{P}\) is pseudostationary (disguised) at the beginning.
The eigenvector \(\mathbf{x}_1=[0.6, 0.4]^T\) is for \(\lambda_1=1\). The other eigenvector is \(\mathbf{x}_2=[1,-1]^T\) for \(\lambda_2=0.5\).
19/03/2018
Burn-in for a Markov Chain
Burn-in for a Markov Chain
Each column of \(\mathbf{P}\) can be represented by a combination of eigenvectors \[ \left[\begin{array}{c} 0.8 \\ 0.2 \end{array}\right]=\mathbf{x}_1 + 0.2 \mathbf{x}_2,\,\, \left[\begin{array}{c} 0.3 \\ 0.7 \end{array}\right]=\mathbf{x}_1 - 0.3 \mathbf{x}_2.\] The powers of \(\mathbf{P}\) do not affect the eigenvector \(\mathbf{x}_1\), but for \(\mathbf{x}_2\), \((0.5)^{30}\mathbf{x}_2\) becomes a very small vector that will disappear eventually.
The period that a Markov chain approaches its steady state is called a burn-in time, namely, the moment of eliminating all the eignevectors except the one with the unit eigenvalue.
Gambler’s ruin problem
Consider a set \(\{0,1,2\dots,N \}\). The position \(x\) represents the gambler’s capital.
Each time step represents one game where the gambler may either increase his/her capital to \(x+1\) or decrease it to \(x−1\). (In each game, there is either a gain or loss, never a tie.)
If the gambler’s capital reaches zero, he/she is ruined, the opponent has won, and the games stop, whereas if the capital reaches \(N\), he/she has won all of the capital (the opponent is ruined), and the games stop.
Let \(w\) be the probability of moving to the right (winning a game), \(q\) be the probability of moving to the left (losing a game), and \(w + q = 1\).
Gambler’s ruin problem
- The transition matrix is given by \[ p_{ij} = \Pr\{X_{t+1} = i | X_t=j \} =\begin{cases} w, & \mbox{ if }i=j+1\\ q, & \mbox{ if }i=j-1\\ 0, & \mbox{ if }i\neq j+1,\, j-1 \end{cases}\] for \(j=1,2,\dots,N-1\). The absorbing states are \(p_{00}=1\) and \(p_{NN}=1\). Or \[\mathbf{P}=\left(\begin{array}{ccccccc} 1 & q & 0 & \dots & 0 & 0 & 0\\ 0 & 0 & q & \dots & 0 & 0 & 0\\ 0 & w & 0 & \dots & 0 & 0 & 0\\ 0 & 0 & w & \dots & 0 & 0 & 0\\ \vdots & \vdots & \vdots & \dots & \vdots & \vdots & \vdots\\ 0 & 0 & 0 & \dots & 0 & q & 0\\ 0 & 0 & 0 & \dots & w & 0 & 0\\ 0 & 0 & 0 & \dots & 0 & w & 1 \end{array}\right)\]
Birth and Death
- Modify the gambler’s ruin problem. The probability of birth/winning (or death/losing) is not constant but depends on the size of the population. \[ p_{ij} = \Pr\{X_{t+1} = i | X_t=j \} =\begin{cases} b_j, & \mbox{ if }i=j+1\\ d_j, & \mbox{ if }i=j-1\\ 1- (b_j+d_j), & \mbox{ if }i=j\\ 0, & \mbox{ if }i\neq j+1,\,j, \, j-1 \end{cases} \\ \mathbf{P}=\left(\begin{array}{cccccc} 1 & d_{1} & 0 & \dots & 0 & 0\\ 0 & 1-(b_{1}+d_{1}) & d_{2} & \dots & 0 & 0\\ 0 & b_1 & 1-(b_{2}+d_{2}) & \dots & 0 & 0\\ 0 & 0 & b_{2} & \dots & 0 & 0\\ \vdots & \vdots & \vdots & \dots & \vdots & \vdots\\ 0 & 0 & 0 & \dots & 1-(b_{N-1}+d_{N-1}) & d_{N}\\ 0 & 0 & 0 & \dots & b_{N-1} & 1-d_{N} \end{array}\right)\]
Stochastic Logistic Growth
Recall the logistic growth model \[y_{t+1} - y_t = r y_t \left(1 - \frac{y_t}{K} \right)=ry_t - r\frac{y^2_t}{K}.\]
A quadratic function of \(y\) equals the birth rate minus the death rate.
A stochastic logistic growth \[b_j - d_j = r \times j \left( 1 -\frac{j}{K} \right)= r \times j - r \times \frac{j^2}{K}\] for \(j=0,1,\dots, N\). Assume that \(\max \{b_j + d_j \}\leq 1\) for any time interval \(t\) to \(t+1\).
Stochastic Logistic Growth
K=50; N=2*K; r= 0.004; v=0:N+1; P=matrix(0,nrow=N+1,ncol=N+1);
b=r*v; d=r*v*v/K
for(j in 2:N){
P[j,j]=1-b[j]-d[j];
P[j,j+1]=d[j+1];
P[j+1,j]=b[j]
}
P[1,1]=1; P[1,2]=d[2];P[N+1,N+1]=1-d[N+1]
P[1:5,1:5]
## [,1] [,2] [,3] [,4] [,5] ## [1,] 1 0.00032 0.00000 0.00000 0.000 ## [2,] 0 0.99168 0.00072 0.00000 0.000 ## [3,] 0 0.00800 0.98728 0.00128 0.000 ## [4,] 0 0.00000 0.01200 0.98272 0.002 ## [5,] 0 0.00000 0.00000 0.01600 0.978
Stochastic Logistic Growth
library(markovchain); SimTime = 3000; set.seed(2019)
MC= new("markovchain", transitionMatrix=t(P))
is.irreducible(MC) # It has one absobing state
## [1] FALSE
absorbingStates(MC) # steadyStates(MC) # No steady state
## [1] "1"
Stochastic Logistic Growth
y=matrix(0,nrow=3,ncol=SimTime);
for(i in 1:3){
y[i,]=markovchainSequence(n=SimTime-1, markovchain=MC, t0=5, include=TRUE)
}
assessStationarity(y[1,], 1)
## The assessStationarity test statistic is: 2259.56 ## The Chi-Square d.f. are: 0 ## The p-value is: 0
Stochastic Logistic Growth
Probability Evolution (Eigenvector)
v = eigen(P, symmetric=FALSE); head(v$values)
## [1] -1.1813400 -1.0305484 1.0000000 1.0000000 0.9962551 0.9931675
v3= v$vectors[,3]; v4=v$vectors[,4]; c(v3[1:5],v4[1:5])
## [1] 1.000000e+00 0.000000e+00 0.000000e+00 0.000000e+00 0.000000e+00 ## [6] -1.470733e-03 1.364954e-14 -3.325741e-14 -3.956755e-14 -5.810998e-14
vProb=v4/sum(v4); which.max(vProb)
## [1] 48
vProb[46:50]
## [1] 0.05302102 0.05520522 0.05630741 0.05628396 0.05515828
Probability Evolution (Eigenvector)
plot(vProb, type="l")
Probability Evolution (Beginning)
library(Matrix)
P=Matrix(P, sparse=TRUE); ptemp=matrix(0,nrow=N+1,ncol=1); ptemp[6,1]=1 # X_0=5
for(t in 1:200){ p=P%*%ptemp; ptemp=p}; p = t(p);
pCum=cumsum(p); plot(p, type="l",col="grey", ylim=c(0,1)); lines(pCum)
Probability Evolution (Failure of Matrix Operation)
ptemp=matrix(0,nrow=N+1,ncol=1); ptemp[6,1]=1
for(t in 1:1000){ p=P%*%ptemp; ptemp=p}; p = t(p);
pCum=cumsum(p); plot(p, type="l",col="grey", ylim=c(-2,2)); lines(pCum)
Question One for the Exam
\(S_t\) and \(I_t\) are the number of susceptible and infected individuals at time \(t\).
Suppose the total number \(S_t + I_t = K\) is fixed at level \(K\).
The dynamics follows \[ I_{t+1} = I_t + \beta I_t S_t = I_t + r I_t \left(1- \frac{I_t}{K}\right)\] where \(r = \beta /K\). (It reduces a bivariate Markov chain model to a single chain model of \(I_t\).)
Question One for the Exam
The state space of \(I_t\) is \(\{0,1,2,\dots,2K\}\). From time \(t\) to \(t+1\), there is at most one change in the random variable \(I_t\). If \(I_t = i\), then \(I_{t+1}\) may change to either \(i+1\) or \(i-1\) or remain at the state \(i\).
I can tell you \(p_{i+1, i} = \Pr\{I_{t+1} = i +1 | I_t = i \}= r(i - i^2/2K)\). \(p_{0,0}=1\) ( \((0,0)\) state is an absorbing state). Think about the full transition matrix and simulation of \(I_t\) (and \(S_t\)).
Also, think about the deterministic continuous time version of the problem.
(For large \(K\), the stochastic model follows the deterministic model more closely, the probability distribution is approximately stationary for a long period of time - called quasistationary distribution. But MC sampler may have difficulty for the implementation.)
Estimation
A stationary (time homogeneous) Markov chain can be fit from given data.
The simplest way is MLE (empirical likelihood) \[\hat{p}_{ij} = \frac{\mbox{The number from state j to state i}}{\mbox{The number from any state to state i}} \\ = \sum_{t=0}^{T} \frac{1}{T} \frac{\# \{ X_{t+1} = s_i, X_{t}= s_j \}}{\sum_{k=0}^{N} \# \{ X_{t+1} = s_i, X_{t} = s_k \}}\]
mle=markovchainFit(data=y[1,], method="mle"); # other choices: laplace, map, bootstrap
Estimation
mle$estimate # mle$confidenceInterval
## MLE Fit ## A 61 - dimensional discrete Markov Chain defined by the following states: ## 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 5, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 6, 60, 61, 62, 63, 64, 65, 7, 8, 9 ## The transition matrix (by rows) is defined as follows: ## 10 11 12 13 14 15 16 ## 10 0.8333333 0.16666667 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 11 0.0000000 0.97959184 0.02040816 0.00000000 0.00000000 0.0000 0.0000000 ## 12 0.0000000 0.02222222 0.95555556 0.02222222 0.00000000 0.0000 0.0000000 ## 13 0.0000000 0.00000000 0.00000000 0.91666667 0.08333333 0.0000 0.0000000 ## 14 0.0000000 0.00000000 0.00000000 0.00000000 0.93750000 0.0625 0.0000000 ## 15 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 1.0000000 ## 16 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.9333333 ## 17 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 18 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 19 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 20 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 21 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 22 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 23 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 24 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 25 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 26 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 27 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 28 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 29 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 30 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 31 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 32 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 33 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 34 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 35 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 36 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 37 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 38 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 39 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 40 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 41 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 42 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 43 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 44 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 45 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 46 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 47 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 48 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 49 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 5 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 50 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 51 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 52 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 53 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 54 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 55 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 56 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 57 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 58 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 59 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 6 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 60 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 61 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 62 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 63 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 64 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 65 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 7 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 8 0.0000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 9 0.2500000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000 0.0000000 ## 17 18 19 20 21 22 23 ## 10 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 11 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 12 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 13 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 14 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 15 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 16 0.06666667 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 17 0.97142857 0.02857143 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 18 0.00000000 0.50000000 0.5 0.0000 0.0000000 0.00000000 0.00000000 ## 19 0.00000000 0.00000000 0.9 0.1000 0.0000000 0.00000000 0.00000000 ## 20 0.00000000 0.00000000 0.0 0.9375 0.0625000 0.00000000 0.00000000 ## 21 0.00000000 0.00000000 0.0 0.0000 0.9166667 0.08333333 0.00000000 ## 22 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.92307692 0.07692308 ## 23 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.80000000 ## 24 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 25 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 26 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 27 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 28 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 29 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 30 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 31 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 32 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 33 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 34 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 35 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 36 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 37 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 38 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 39 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 40 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 41 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 42 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 43 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 44 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 45 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 46 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 47 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 48 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 49 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 5 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 50 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 51 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 52 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 53 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 54 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 55 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 56 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 57 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 58 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 59 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 6 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 60 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 61 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 62 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 63 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 64 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 65 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 7 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 8 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 9 0.00000000 0.00000000 0.0 0.0000 0.0000000 0.00000000 0.00000000 ## 24 25 26 27 28 29 ## 10 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 11 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 12 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 13 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 14 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 15 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 16 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 17 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 18 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 19 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 20 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 21 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 22 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 23 0.2000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 24 0.9166667 0.08333333 0.0000000 0.0000000 0.0000000 0.00000000 ## 25 0.0000000 0.85714286 0.1428571 0.0000000 0.0000000 0.00000000 ## 26 0.0000000 0.00000000 0.8888889 0.1111111 0.0000000 0.00000000 ## 27 0.0000000 0.00000000 0.0000000 0.7142857 0.2857143 0.00000000 ## 28 0.0000000 0.00000000 0.0000000 0.1000000 0.8000000 0.10000000 ## 29 0.0000000 0.00000000 0.0000000 0.0000000 0.1000000 0.80000000 ## 30 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.08333333 ## 31 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 32 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 33 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 34 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 35 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 36 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 37 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 38 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 39 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 40 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 41 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 42 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 43 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 44 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 45 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 46 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 47 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 48 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 49 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 5 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 50 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 51 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 52 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 53 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 54 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 55 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 56 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 57 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 58 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 59 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 6 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 60 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 61 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 62 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 63 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 64 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 65 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 7 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 8 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 9 0.0000000 0.00000000 0.0000000 0.0000000 0.0000000 0.00000000 ## 30 31 32 33 34 35 ## 10 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 11 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 12 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 13 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 14 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 15 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 16 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 17 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 18 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 19 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 20 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 21 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 22 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 23 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 24 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 25 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 26 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 27 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 28 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 29 0.10000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 30 0.62500000 0.2916667 0.0000000 0.00000000 0.00000000 0.0000000 ## 31 0.06363636 0.7909091 0.1454545 0.00000000 0.00000000 0.0000000 ## 32 0.00000000 0.1600000 0.7600000 0.08000000 0.00000000 0.0000000 ## 33 0.00000000 0.0000000 0.1176471 0.80882353 0.07352941 0.0000000 ## 34 0.00000000 0.0000000 0.0000000 0.08928571 0.73214286 0.1785714 ## 35 0.00000000 0.0000000 0.0000000 0.00000000 0.11363636 0.7613636 ## 36 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.1375000 ## 37 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 38 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 39 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 40 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 41 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 42 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 43 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 44 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 45 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 46 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 47 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 48 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 49 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 5 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 50 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 51 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 52 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 53 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 54 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 55 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 56 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 57 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 58 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 59 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 6 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 60 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 61 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 62 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 63 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 64 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 65 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 7 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 8 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 9 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.0000000 ## 36 37 38 39 40 41 42 ## 10 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 11 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 12 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 13 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 14 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 15 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 16 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 17 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 18 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 19 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 20 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 21 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 22 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 23 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 24 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 25 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 26 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 27 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 28 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 29 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 30 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 31 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 32 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 33 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 34 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 35 0.1250000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 36 0.7750000 0.0875000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 37 0.1372549 0.6666667 0.1960784 0.0000000 0.0000000 0.00000000 0.0000000 ## 38 0.0000000 0.1030928 0.7835052 0.1134021 0.0000000 0.00000000 0.0000000 ## 39 0.0000000 0.0000000 0.2391304 0.5652174 0.1956522 0.00000000 0.0000000 ## 40 0.0000000 0.0000000 0.0000000 0.1666667 0.7037037 0.12962963 0.0000000 ## 41 0.0000000 0.0000000 0.0000000 0.0000000 0.1228070 0.71929825 0.1578947 ## 42 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.09473684 0.7473684 ## 43 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.1200000 ## 44 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 45 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 46 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 47 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 48 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 49 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 5 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 50 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 51 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 52 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 53 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 54 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 55 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 56 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 57 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 58 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 59 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 6 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 60 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 61 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 62 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 63 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 64 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 65 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 7 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 8 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 9 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 ## 43 44 45 46 47 48 49 ## 10 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 11 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 12 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 13 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 14 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 15 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 16 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 17 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 18 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 19 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 20 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 21 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 22 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 23 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 24 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 25 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 26 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 27 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 28 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 29 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 30 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 31 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 32 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 33 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 34 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 35 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 36 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 37 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 38 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 39 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 40 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 41 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 42 0.1578947 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 43 0.7200000 0.1600000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 44 0.1315789 0.7105263 0.1578947 0.0000000 0.0000000 0.0000000 0.0000000 ## 45 0.0000000 0.1188119 0.7079208 0.1732673 0.0000000 0.0000000 0.0000000 ## 46 0.0000000 0.0000000 0.1502146 0.6866953 0.1630901 0.0000000 0.0000000 ## 47 0.0000000 0.0000000 0.0000000 0.1603376 0.6793249 0.1603376 0.0000000 ## 48 0.0000000 0.0000000 0.0000000 0.0000000 0.2405063 0.6139241 0.1455696 ## 49 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.2555556 0.5666667 ## 5 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 50 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.2318841 ## 51 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 52 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 53 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 54 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 55 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 56 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 57 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 58 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 59 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 6 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 60 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 61 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 62 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 63 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 64 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 65 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 7 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 8 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 9 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 5 50 51 52 53 54 55 ## 10 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 11 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 12 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 13 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 14 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 15 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 16 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 17 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 18 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 19 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 20 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 21 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 22 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 23 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 24 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 25 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 26 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 27 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 28 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 29 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 30 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 31 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 32 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 33 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 34 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 35 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 36 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 37 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 38 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 39 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 40 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 41 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 42 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 43 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 44 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 45 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 46 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 47 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 48 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 49 0.0000000 0.1777778 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 5 0.9824561 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 50 0.0000000 0.6231884 0.1449275 0.0000000 0.0000000 0.0000000 0.0000000 ## 51 0.0000000 0.3125000 0.4687500 0.2187500 0.0000000 0.0000000 0.0000000 ## 52 0.0000000 0.0000000 0.1521739 0.5652174 0.2826087 0.0000000 0.0000000 ## 53 0.0000000 0.0000000 0.0000000 0.2765957 0.5744681 0.1489362 0.0000000 ## 54 0.0000000 0.0000000 0.0000000 0.0000000 0.2121212 0.5454545 0.2424242 ## 55 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.2500000 0.4687500 ## 56 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.2812500 ## 57 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 58 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 59 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 6 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 60 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 61 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 62 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 63 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 64 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 65 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 7 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 8 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 9 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 ## 56 57 58 59 6 60 61 ## 10 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 11 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 12 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 13 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 14 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 15 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 16 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 17 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 18 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 19 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 20 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 21 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 22 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 23 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 24 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 25 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 26 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 27 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 28 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 29 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 30 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 31 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 32 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 33 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 34 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 35 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 36 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 37 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 38 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 39 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 40 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 41 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 42 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 43 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 44 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 45 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 46 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 47 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 48 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 49 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 5 0.0000000 0.0000000 0.0000000 0.0000000 0.01754386 0.0000000 0.0000000 ## 50 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 51 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 52 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 53 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 54 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 55 0.2812500 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 56 0.5000000 0.2187500 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 57 0.2592593 0.5555556 0.1851852 0.0000000 0.00000000 0.0000000 0.0000000 ## 58 0.0000000 0.3846154 0.3076923 0.3076923 0.00000000 0.0000000 0.0000000 ## 59 0.0000000 0.0000000 0.4000000 0.3000000 0.00000000 0.3000000 0.0000000 ## 6 0.0000000 0.0000000 0.0000000 0.0000000 0.88888889 0.0000000 0.0000000 ## 60 0.0000000 0.0000000 0.0000000 0.2727273 0.00000000 0.5454545 0.1818182 ## 61 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.1818182 0.3636364 ## 62 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.4545455 ## 63 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 64 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 65 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 7 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 8 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 9 0.0000000 0.0000000 0.0000000 0.0000000 0.00000000 0.0000000 0.0000000 ## 62 63 64 65 7 8 9 ## 10 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 11 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 12 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 13 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 14 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 15 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 16 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 17 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 18 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 19 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 20 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 21 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 22 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 23 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 24 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 25 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 26 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 27 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 28 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 29 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 30 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 31 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 32 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 33 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 34 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 35 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 36 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 37 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 38 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 39 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 40 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 41 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 42 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 43 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 44 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 45 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 46 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 47 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 48 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 49 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 5 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 50 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 51 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 52 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 53 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 54 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 55 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 56 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 57 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 58 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 59 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 6 0.00000000 0.0000000 0.0000000 0.0 0.1111111 0.0000000 0.00000000 ## 60 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 61 0.45454545 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 62 0.09090909 0.4545455 0.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 63 0.38461538 0.3076923 0.3076923 0.0 0.0000000 0.0000000 0.00000000 ## 64 0.00000000 0.4000000 0.4000000 0.2 0.0000000 0.0000000 0.00000000 ## 65 0.00000000 0.0000000 1.0000000 0.0 0.0000000 0.0000000 0.00000000 ## 7 0.00000000 0.0000000 0.0000000 0.0 0.9000000 0.1000000 0.00000000 ## 8 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.9722222 0.02777778 ## 9 0.00000000 0.0000000 0.0000000 0.0 0.0000000 0.0000000 0.75000000
Estimation
pm=as.matrix(mle$estimate@transitionMatrix); image(pm, axes=FALSE)
Simulation
MCSim= new("markovchain", transitionMatrix=pm)
ys= markovchainSequence(n=SimTime-1, markovchain=MCSim, t0=5, include=TRUE);
ts.plot(ys)
Continuous Time Markov Chains
For the moment, let’s allow time to be continuous and assume a finite state space (\(N\) states).
Transitions probability under the continuous time. \[\Pr\left( X_{t+\Delta t}=S_i | X_{t}= S_j \right) =p_{ij}(t,t+\Delta t)\]
The transition probability matrix \[\mathbf{P}(t, t+\Delta t)=\{p_{ij}(t, t+\Delta t)\}_{i,j=1}^{N}.\]
For time homogenous or stationary: \[\Pr\left( X_{t+\Delta t}=S_i | X_{t}= S_j \right) = \Pr(X_{\Delta t} = S_i | X_{0} = S_j)=p_{ij}(\Delta t).\]
CK Equation
The interval from \(k\) to \(t\) can be partitioned (discretized) into \(k =t_0 < t_1 <\dots < t_m = t\). The transition probability matrix can be written as a product: \[ \mathbf{P}(k,t) = \mathbf{P}(t_0,t_m)=\mathbf{P}(t_0,t_1) \mathbf{P}(t_1,t_2) \dots \mathbf{P}(t_{m-1},t_m)\] This follows the CK equation in discrete time setting.
When the chain is stationary and \(t_1 - t_0 =t_2 -t_1=\cdots =\Delta t\), \[\mathbf{P}(k,t) = \mathbf{P}(t_0,t_m)=[\mathbf{P} (0,\Delta t)]^m=[\mathbf{P} (\Delta t)]^m.\]
CK Equation
- Now consider the stationary CK \(\mathbf{P}(t + \Delta t)= \mathbf{P}(t) \mathbf{P}(\Delta t)\): \[ \frac{\mathbf{P}(t + \Delta t) - \mathbf{P}(t)}{t+\Delta t - t} = \frac{\mathbf{P}(t)\left[\mathbf{P}(\Delta t) - \mathbf{I} \right]}{\Delta t}\] and take the limit \(\Delta t \rightarrow 0\) \[ \lim_{\Delta t \rightarrow 0} \frac{\mathbf{P}(t + \Delta t) - \mathbf{P}(t)}{t+\Delta t - t} = \frac{d\mathbf{P} (t)}{dt}= \mathbf{P}(t) \times \lim_{\Delta t \rightarrow 0} \frac{\left[\mathbf{P}(\Delta t) - \mathbf{I} \right]}{\Delta t}.\]
CK Equation
- For the moment, let’s assume the limit of \((\mathbf{P}(\Delta t) - \mathbf{I})/\Delta t\) exists and we denote it as a matrix \(\mathbf{Q}\). Then we will obtain the continuous time CK forward equation \[\frac{d\mathbf{P}(t)}{dt} = \mathbf{P}(t) \mathbf{Q}.\] Similarly, if we use \(\mathbf{P}(t + \Delta t)= \mathbf{P}(\Delta t)\mathbf{P}(t)\), we will have the CK backward equation \[\frac{d\mathbf{P}(t)}{dt} = \mathbf{Q} \mathbf{P}(t).\]
Remarks
Markov chain’s Burn-in
Construct a stochastic difference equation (based on a deterministic skeleton)
Probability evolution
Estimation and simulation
Continuous time CK (Kolmogorov forward equation for probability evolution)