Description of the Problem

A computer is shared by 2 users who send tasks to a computer remotely and work independently. At any minute, any connected user may disconnect with probability 0.5, and any disconnected user may connect with a new task with probability 0.2. Let X(t) be the number of concurrent users at time t (in minutes). This is a Markov chain with 3 states: 0, 1, and 2 (or 1, 2, and 3). The probability transition matrix can be calculated and it is


transition_matrix
##      [,1] [,2] [,3]
## [1,] 0.64 0.32 0.04
## [2,] 0.40 0.50 0.10
## [3,] 0.25 0.50 0.25


Generate 10000 transitions of the Markov chain. What percent of times do you find your generated Markov chain in each of its 3 states?

Problem Solving

Simulate the Markov Chain

markov_chain_simulation <- function(transition_matrix, num_transitions) {
  states <- c(1, 2, 3)
  current_state <- sample(states, 1)
  chain <- rep(0, num_transitions)
  chain[1] <- current_state
  
  for (i in 2:num_transitions) {
    probabilities <- transition_matrix[current_state, ]
    current_state <- sample(states, 1, prob = probabilities)
    chain[i] <- current_state
  }
  
  return(chain)
}

num_transitions <- 10000
chain <- markov_chain_simulation(transition_matrix, num_transitions)

print(chain[1:50])
##  [1] 2 2 1 1 2 1 1 2 2 2 1 2 1 2 3 2 2 1 1 2 1 1 2 3 1 1 1 3 3 1 1 2 2 1 2 2 2 2
## [39] 1 1 2 2 1 1 1 1 1 1 1 2


Calculate the frequency of each state in the generated chain

state_frequencies
## chain
##      1      2      3 
## 0.5117 0.4059 0.0824


Plot a bargrah

bd
##      [,1]
## [1,]  0.7
## [2,]  1.9
## [3,]  3.1


Conclusion

The Markov chain spends approximately 70% of the time in state 1, 109% in state 2, and 7.8% in state 3.