Introduction

Instructions Based on this week’s model, and the Markov simulation website, create a model in Simio to represent 4 possible states of a process. Impersonate the objects used in the model to resemble the process illustrated with the help of the symbols. Bring a small narrative of the process.

I created a simulation of a doctor office visit process. Using simio I built a model with the 4 state of a doctor office visit. These four states are 1. Check-In, 2. Waiting room, 3. Doctor Examination, 4. Checkout/Exit.

In the simio the system for the model were as follow: Source: Arriving patients (Patients enter the doctor’s office) Server1: CheckIn (Patient check in with the front desk) Server2: Waiting Area (Patient sit and wait to be called to see the doctor) Server3:Doctor Exam (The doctor exams the patient) Sink: Check Out (The patient leaves)

In the simulation it seen that 15% of patients end up leaving without seeing the doctor because they were tired of waiting, and 85% go ahead and see the doctor. It is also seen that 15% of the patients seen by the doctor have to go back to the waiting area (occurs when patients are monitor for blood pressure) and 85% of the patients seen by the doctor head to check out.

I believe the model I created accurately integrates Markov principles seen it has assigned transition probabilities at each state using Simio’s link-weight routing. The Waiting Room and Doctor Examination states include probabilistic branching, allowing patients to either proceed forward, exit the system, or return to a previous state. This ensures that all transitions depend only on the current state, satisfying the no memory property of the Markov chain.

###Simio Model

Markovchain Matrix

set.seed(20)

states<-c("CheckIn","WaitingArea", "DocExam", "CheckOut")

P <- matrix(
c(
# CheckIn
0.00, 1.00, 0.00, 0.00,

# Waiting Area
0.00, 0.00, 0.85, 0.15,

# Doctor Exam
0.00, 0.15, 0.00, 0.85,

# Check Out
0.00, 0.00, 0.00, 1.00
),
nrow = 4, byrow = TRUE,
dimnames = list(from = states, to = states)
)
P
##              to
## from          CheckIn WaitingArea DocExam CheckOut
##   CheckIn           0        1.00    0.00     0.00
##   WaitingArea       0        0.00    0.85     0.15
##   DocExam           0        0.15    0.00     0.85
##   CheckOut          0        0.00    0.00     1.00
n_steps<-10
current_state <- sample(states[-4], 1) #exclude checkout
state_history <- current_state

# simulation loop
for (i in 1:n_steps) {
  current_index <- match(current_state, states)
  next_state <- sample(states, 1, prob = P[current_index, ])
  state_history <- c(state_history, next_state)
  current_state <- next_state
  if (current_state == "CheckOut") break
}

#results
df <- data.frame(Step = 0:(length(state_history) - 1), 
                 State = state_history)

cat("Patients in an Office Visit Path:\n\n")
## Patients in an Office Visit Path:
print(df)
##   Step       State
## 1    0 WaitingArea
## 2    1     DocExam
## 3    2    CheckOut

Steady-State

# Solve stationary distribution
A <- t(P) - diag(4)
A[4, ] <- 1

b <- c(0, 0, 0, 1)

steady_state <- solve(A, b)
steady_state
##     CheckIn WaitingArea     DocExam    CheckOut 
##           0           0           0           1

the steady-state probabilities of the system were computed using the transition matrix of the Markov chain. The results show that all probability mass converges to the CheckOut state in the long run. This occurs because CheckOut is an absorbing state, meaning once a patient reaches this state, they do not transition to any other state. Although patients may cycle between WaitingArea and Doctor states temporarily, the system is not ergodic. Instead, it is an absorbing Markov chain where the long-term behavior is dominated by absorption. This result is consistent with the simulation model in Simio, where all entities eventually exit the system after a finite number of steps.

Fundamental matrix

The fundamental matrix provides the expected number of visits to each transient state before absorption. In this case how many times a patient is expected to visit each state before reaching CheckOut

Q <- P[1:3, 1:3]
I <- diag(3)

N <- solve(I - Q)
N
##             CheckIn WaitingArea  DocExam
## CheckIn           1   1.1461318 0.974212
## WaitingArea       0   1.1461318 0.974212
## DocExam           0   0.1719198 1.146132