Project 3

A computer is shared by 2 users who send tasks to a computer remotely and work independently. At any minute, any connected user may disconnect with probability 0.5, and any disconnected user may connect with a new task with probability 0.2. Let X(t) be the number of concurrent users at time t (in minutes). This is a Markov chain with 3 states: 0, 1, and 2 (or 1, 2, and 3). The probability transition matrix can be calculated, and it is

  0.64  0.32  0.04  
  0.40  0.50  0.10
  0.25  0.50  0.25

Generate 10000 transitions of the Markov chain. What percent of times do you find your generated Markov chain in each of its 3 states?

set.seed(1) #sets the random seed so result are replicable

#Global variables 
N = 10000
counter = 2 #starts the while loop counter at 2 since the first state is known.

initialProb = c(0.5, 0.2, 0.3) #initial probabilities. 
states = c(1,2,3) #the different possible states 


#Probability Transition Matrix
PMatrix = matrix(c(0.64,0.4,0.25,
                   0.32,0.50,0.50,
                   0.04,0.10,0.25), nrow = 3)
PMatrix #printing the probability transition matrix
##      [,1] [,2] [,3]
## [1,] 0.64 0.32 0.04
## [2,] 0.40 0.50 0.10
## [3,] 0.25 0.50 0.25
states = numeric(N) #creates and an empty array of zeros. 
states[1] = 1 #starts the Markov chain in state 1

#while loop that generates 10,000 transitions of the Markov chain.
while(counter<=N)
{
  currentState = states[counter-1] #gets the most recent state
  nextState = sample(1:3, size=1, prob = PMatrix[currentState,]) #Chooses the 
                                          #next state based on the current state
  states[counter] = nextState #Stores the result of the sample 
  counter = counter + 1 #increments the counter.
}

cat("First 50 simulated states:\n")
## First 50 simulated states:
print(states[1:50]) #prints the first 50 simulated states 
##  [1] 1 1 1 1 2 2 1 2 1 1 1 1 1 2 2 1 1 2 3 2 1 2 2 1 1 1 1 1 1 2 2 2 1 1 1 2 1 2
## [39] 2 1 1 2 1 2 1 1 2 2 2 1
#creating the x and y axis of the bar graph 
y = table(factor(states[1:N],levels=1:3)) #creates of table to show the 3 states 
                                          #and there corresponding frequencies
cat("Frequency table\n")
## Frequency table
y
## 
##    1    2    3 
## 5048 4100  852
x =c(1,2,3) #The x-axis states

#bar graph
barplot(y, # frequency of each state
        names.arg = c("State 1", "State 2", "State 3"), #Names the x-axis with
                                                        #each state 
        col = "hotpink", 
        main = "Frequency of States in the First 50 Steps",  
        ylab = "Frequency", 
        xlab = "States")

#scatter plot 
plot(states, 
        type= "p",#plots individual points
        col = "hotpink", 
        main = "States over Time",  
        ylab = "State", 
        xlab = "Step Number",
        pch = 16) #type of dot used in the scatter plot.

Discussion

This project simulates a Markov Chain. The project has three different states. The simulation determines the frequency in which each of the states appears. The program displays the frequency and state relationship in bar graph form highlighting that the first state appears the most frequent. This is inline with the idea that the Markov Chain will approach a steady state.