Project#3

Problem

A computer is shared by 2 users who send tasks to a computer remotely and work independently. At any minute, any connected user may disconnect with probability 0.5, and any disconnect user may connect with a new task with probability 0.2. Let X(t) be the number of concurrent users at time t (in minutes). This is a Markov chain with 3 states: 0, 1, and 2. The probability transition matrix can be calculated and it is

##      [,1] [,2] [,3]
## [1,] 0.64 0.32 0.04
## [2,] 0.40 0.50 0.10
## [3,] 0.25 0.50 0.25

Generate 10000 transitions of the Markov chain. What percent of times do you find your generated Markov chain in each of its 3 states?

Code

The ‘states’ vector and the transition matrix will first be recreated from the original problem.

states <- c("0", "1", "2")
trans_matrix <-matrix(c(0.64, 0.40, 0.25, 0.32, 0.50, 0.50, 0.04, 0.10, 0.25),
                      nrow=3, byrow=FALSE, dimnames = list(states, states))

With this, a Markov object can be created with the ‘markovchain’ package.

MCComp <- new("markovchain", states = states, byrow = TRUE,
              transitionMatrix = trans_matrix)

The results of the Markov chain can be seen as follows:

## Unnamed Markov chain 
##  A  3 - dimensional discrete Markov Chain defined by the following states: 
##  0, 1, 2 
##  The transition matrix  (by rows)  is defined as follows: 
##      0    1    2
## 0 0.64 0.32 0.04
## 1 0.40 0.50 0.10
## 2 0.25 0.50 0.25

With matrix multiplication, 10000 transitions can be generated of the Markov chain.

MCComp_10000 <- MCComp^10000

The results of the 10000 transitions are as follows:

## Unnamed Markov chain^10000 
##  A  3 - dimensional discrete Markov Chain defined by the following states: 
##  0, 1, 2 
##  The transition matrix  (by rows)  is defined as follows: 
##           0         1          2
## 0 0.5102041 0.4081633 0.08163265
## 1 0.5102041 0.4081633 0.08163265
## 2 0.5102041 0.4081633 0.08163265

The probabilities of the generated Markov chain in each of its 3 states are as follows:

##              0         1          2
## [1,] 0.5102041 0.4081633 0.08163265

Graphs/Tables

Table 1 below displays the results for the probability of the generated Markov chain being in each of its 3 states after 10000 transitions.

Probability of Markov Chain in each of its 3 states after 10000 transitions

0	1	2
0.5102041	0.4081633	0.0816327

The original Markov object can be plotted and is seen in Figure 1 below.

Plot of the Original Markov Object

The Markov object after 10000 transitions can also be plotted and is seen in Figure 2 below.

Plot of the Markov Object after 10000 Transitions

Discussion

The percent of finding the generated Markov chain in State 0 is 51.02% (0.5102). The percent of finding the generated Markov chain in State 1 is 40.82% (0.4082). The percent of finding the generated Markov chain in State 2 is 8.16% (0.0816). In the code generated earlier, it was seen that after the 10000 transitions, the values in each column reached steady state - that is, the values were the same in each column. This was conclusive in determining that the percents were 51.02%, 40.82%, and 8.16% for states 0, 1, and 2, respectively.

Complete Code

The complete code used to generate the results seen above is as follows:

if(!require(tidyverse)) install.packages("tidyverse", repos = "http://cran.us.r-project.org")
if(!require(data.table)) install.packages("data.table", repos = "http://cran.us.r-project.org")
if(!require(knitr)) install.packages("knitr", repos = "http://cran.us.r-project.org")
if(!require(markovchain)) install.packages("markovchain", repos = "http://cran.us.r-project.org")

library(tidyverse)
library(data.table)
library(knitr)
library(markovchain)

set.seed(1234)

states <- c("0", "1", "2")

trans_matrix <-matrix(c(0.64, 0.40, 0.25, 0.32, 0.50, 0.50, 0.04, 0.10, 0.25),
                      nrow=3, byrow=FALSE, dimnames = list(states, states))

MCComp <- new("markovchain", states = states, byrow = TRUE, transitionMatrix = trans_matrix)
MCComp

plot(MCComp)

MCComp_10000 <- MCComp^10000

MCComp_10000

steadyStates(MCComp_10000)

plot(MCComp_10000)

markov_results <- as.data.frame(steadyStates(MCComp_10000))
markov_results