DATA 604 : Week 9 Assignment

Instructions
Introduction
Simio Model Design
Transition Behavior
Simulation Flow
Simio Model Implementation
Mapping Transition Matrix
Decision Logic and Flow
Markov Transition Matrix
Markov Chain Object
Simulation KPIs
Personnel Flow Description
KPIs
Visualization & Scenario Analysis
Pipeline Efficiency
Probability Matrix
Conclusion

Instructions

Based on this week’s model, and the Markov simulation website, create a model in Simio to represent 4 possible states of a process. Impersonate the objects used in the model to resemble the process illustrated with the help of the symbols. Bring a small narrative of the process.

Load Libraries

Introduction

Markov models are widely used to represent systems where entities move between different states based on probabilities. These models are especially useful in workforce and personnel systems, where individuals transition between roles such as training, active duty, and retirement. Research shows that Markov chains have long been applied in manpower planning, particularly in military and organizational systems to model transitions and predict workforce dynamics .

In this assignment, a 4-state Markov simulation model is developed in Simio to represent a process system. The model captures how entities transition between states over time, using probabilistic routing similar to the Markov simulator provided.

The system modeled in Simio represents a military personnel training and deployment pipeline, simplified into four states:

• Recruit (State 1)

• Training (State 2)

• Active Duty (State 3)

• Exit/Retirement (State 4)

Simio Model Design

To represent the system visually and logically, the following Simio objects are used:

• Source → Generates incoming recruits

• Server (Training Center) → Represents training phase

• Server (Active Duty Unit) → Represents deployed personnel

• Sink → Represents exit or retirement

• Paths/Connectors → Represent transitions between states

• Decide Steps (Routing Logic) → Used to assign probabilities

Mapping of States

Transition Behavior

The movement of entities is governed by probabilities assigned at each decision point:

Transition Probability Matrix

Simulation Flow

The simulation begins when new recruits enter the system. They are first assigned to training, although some may exit early.

After training, successful individuals move into active duty roles. In this stage, personnel may remain deployed for multiple cycles or eventually leave the system.

Some transitions allow limited re-entry to earlier stages, reflecting retraining or reassignment cases.

Simio Model Implementation

The Markov process described above was implemented in Simio using four main objects to represent the system states:

State Simio Object Recruitment Source Training Server Active Duty Server Exit Sink

A queue was added before each server to represent waiting time for training and deployment.

Entities (personnel) enter the system through the Source (Recruitment) and are routed probabilistically to Training or directly to Exit based on the transition matrix.

After processing:

Personnel may return to Recruitment (reassignment or reprocessing cases) Move from Training to Active Duty or Exit Remain in Active Duty for multiple cycles or eventually leave the system

Simio Model Visualization

The Simio model was developed to visually represent the four-state Markov process for the military personnel pipeline. The model includes a Source (Recruit), two Servers (Training and Active Duty) and a Sink (Exit), connected through directed paths that represent state transitions.

The figure below shows the overall structure of the Simio model, including all states and transitions between them.

Each object in the model was configured to represent a specific stage in the personnel lifecycle. Entities flow from recruitment through training and active duty, and eventually exit the system.

Mapping Transition Matrix to Simio

The model represents a four-state military personnel training and deployment pipeline implemented in Simio. Each object corresponds to a specific stage in the personnel lifecycle.

After each stage, personnel move through the system based on predefined transition probabilities derived from the Markov matrix.

Typical transitions include:

• Some recruits successfully complete training and move into active duty

• Some may leave the system during or after training

• Active-duty personnel may remain in service for multiple cycles or eventually exit

• Limited re-entry may occur in cases of retraining or reassignment

Markov Property in the Model

The system follows the Markov property because each transition depends only on the current state of the personnel. For example, once an individual is in the training stage, their probability of moving to active duty or exiting depends only on their current training status, not on how they entered the system.

This ensures the model behaves as a valid Markov process and allows the personnel flow to be accurately represented using a transition probability matrix in Simio.

Decision Logic and Flow

A recruit enters the Recruit stage and is routed to either Training or directly to Exit based on eligibility and initial screening outcomes. During the training phase, individuals are evaluated for readiness, and based on performance and requirements, they may advance to Active Duty or exit the system if they do not meet the necessary standards.

Once in Active Duty, personnel continue serving operational roles and may either remain in service for multiple cycles (representing retention and continued assignment) or eventually transition to Exit due to retirement, contract completion, or other separation reasons. The Exit stage is an absorbing state, meaning once personnel leave the system, they do not re-enter.

KPI from the model

• Probability that a recruit reaches Active Duty without exiting during training

• Expected number of stage transitions (touchpoints) before reaching Exit

• Share of rework or looping behavior during Training and Active Duty stages

• Distribution of total time spent in the system before Exit (time-to-separation)

set.seed(123)

set.seed(42)
states <- c("Recruit", "Training", "ActiveDuty", "Exit")

Markov Transition Matrix

The markov transition matrix represents the probability of movement between the four states in the military personnel training and deployment pipeline: Recruit, Training, Active Duty, and Exit. Each row defines the probability of transitioning from one state to another in the next time step.

P <- matrix(
  c(
    # From Recruit:
    # to Recruit, Training, ActiveDuty, Exit
    0.00, 0.80, 0.00, 0.20,

    # From Training:
    0.00, 0.10, 0.70, 0.20,

    # From Active Duty:
    0.00, 0.00, 0.60, 0.40,

    # From Exit (absorbing state):
    0.00, 0.00, 0.00, 1.00
  ),
  nrow = 4,
  byrow = TRUE,
  dimnames = list(from = states, to = states)
)

P <- as.matrix(P)

This matrix defines the Markov process governing personnel flow through the system. Each row sums to 1, ensuring valid probabilistic transitions between all states.

Construct Markov Chain Object

The transition matrix is converted into a Markov chain object to enable simulation and analysis of state transitions in the military personnel training and deployment pipeline.

mc <- new("markovchain",
          states = states,
          transitionMatrix = P,
          name = "MilitaryPipelineMC")

mc

MilitaryPipelineMC 
 A  4 - dimensional discrete Markov Chain defined by the following states: 
 Recruit, Training, ActiveDuty, Exit 
 The transition matrix  (by rows)  is defined as follows: 
            to
from         Recruit Training ActiveDuty Exit
  Recruit          0      0.8        0.0  0.2
  Training         0      0.1        0.7  0.2
  ActiveDuty       0      0.0        0.6  0.4
  Exit             0      0.0        0.0  1.0

This object formally defines the Markov process, allowing simulation of personnel movement across Recruit, Training, Active Duty, and Exit states using the specified transition probabilities.

Steady-State and Absorbing State Analysis

To understand long-term behavior in the personnel pipeline, the Markov chain is separated into transient states (Recruit, Training, Active Duty) and the absorbing state (Exit). This decomposition helps evaluate how long individuals remain in the system and their likelihood of eventual separation.

absorbingIdx <- which(states == "Exit")
transientIdx <- setdiff(seq_along(states), absorbingIdx)

Q <- P[transientIdx, transientIdx, drop = FALSE]
R <- P[transientIdx, absorbingIdx, drop = FALSE]

I <- diag(length(transientIdx))

# Fundamental matrix
N <- solve(I - Q)

# Expected time spent in system before absorption
time_to_exit <- N %*% rep(1, nrow(Q))
time_to_exit <- as.vector(time_to_exit)
names(time_to_exit) <- states[transientIdx]

# Probability of eventual absorption into Exit
absorption_prob <- N %*% R
rownames(absorption_prob) <- states[transientIdx]
colnames(absorption_prob) <- "P(Exit)"

result_df <- data.frame(
  State = names(time_to_exit),
  Expected_Steps = round(time_to_exit, 2),
  Probability_of_Exit = round(absorption_prob, 4)
)

kable(result_df, caption = "Markov Chain Results")

The results show that all personnel eventually reach the Exit state, confirming a stable absorbing Markov system. The expected steps indicate how long individuals remain in the system, with earlier stages taking more time. This analysis helps understand retention and overall workforce flow in the pipeline.

Simulation for Personnel Flow KPIs

To evaluate system performance, a simulation is performed to track how personnel move through the pipeline from Recruitment to Exit. Each simulation run represents the path of one individual as they transition between states based on the defined probabilities.

simulate_personnel <- function(P, start = "Recruit", states) {
  cur <- start
  path <- cur
  
  while (cur != "Exit") {
    cur <- sample(states, size = 1, prob = P[cur, ])
    path <- c(path, cur)
    
    # Safety check
    if (length(path) > 1000) break
  }
  
  path
}

nPersonnel <- 10000
paths <- vector("list", nPersonnel)

for (i in seq_len(nPersonnel)) {
  paths[[i]] <- simulate_personnel(P = P, start = "Recruit", states = states)
}

#Steps to Exit
steps_to_exit <- sapply(paths, function(p) length(p) - 1)
#Percentage Reaching Active Duty
reached_active <- sapply(paths, function(p) any(p == "ActiveDuty"))
#Retention
stayed_active <- sapply(paths, function(p) {
  sum(p == "ActiveDuty") > 1
})


kpi_df <- data.frame(
  KPI = c("Avg Steps to Exit", "Median Steps", "% Reached Active Duty", "% Retained in Active Duty"),
  Value = c(
    round(mean(steps_to_exit), 2),
    median(steps_to_exit),
    round(mean(reached_active) * 100, 2),
    round(mean(stayed_active) * 100, 2)
  )
)

kable(kpi_df, caption = "Personnel Simulation KPIs")

The simulation shows how personnel move through the system over time. On average, individuals take a certain number of steps before exiting. A large portion reach Active Duty, and some remain there for multiple cycles, indicating retention. These results help evaluate training effectiveness and workforce stability.

Personnel Flow Description

The process represents a military personnel training and deployment pipeline modeled using four states: Recruit, Training, Active Duty and Exit. Personnel enter the system through the Recruit stage where they are initially accepted into the program. Based on predefined probabilities, most recruits move into Training while some may exit early due to screening or eligibility issues.

In the Training stage, individuals develop the necessary skills and are evaluated for readiness. After training, a portion of personnel successfully transition to Active Duty, while others may leave the system due to performance or other factors.

In the Active Duty stage, personnel are assigned to operational roles. Some individuals remain in this stage for multiple cycles, representing retention and continued service while others eventually exit due to retirement, contract completion, or other reasons.

Key Performance Indicators (KPIs)

The KPIs summarize the performance of the military personnel pipeline using simulation results. These metrics evaluate how personnel progress through each stage. How long they remain in the system, and the likelihood of reaching key states such as Training and Active Duty.

The results indicate that a majority of personnel successfully advance through Training and into Active Duty before exiting. The average number of steps suggests a moderate system duration, while a relatively small proportion of individuals experience rework by returning to earlier stages.

steps_to_exit <- sapply(paths, function(p) length(p) - 1)

ever_training <- sapply(paths, function(p) any(p == "Training"))
ever_active <- sapply(paths, function(p) any(p == "ActiveDuty"))

rework_loops <- sapply(paths, function(p) {
  sum(p[-1] == "Recruit") > 0
})

kpi <- list(
  avg_steps_to_exit = mean(steps_to_exit),
  median_steps_to_exit = median(steps_to_exit),
  pct_reached_training = mean(ever_training),
  pct_reached_active_duty = mean(ever_active),
  pct_rework_to_recruit = mean(rework_loops)
)

kpi_df <- data.frame(
  KPI = c(
    "Avg Steps to Exit",
    "Median Steps to Exit",
    "% Reached Training",
    "% Reached Active Duty",
    "% Rework to Recruit"
  ),
  Value = round(c(
    kpi$avg_steps_to_exit,
    kpi$median_steps_to_exit,
    100 * kpi$pct_reached_training,
    100 * kpi$pct_reached_active_duty,
    100 * kpi$pct_rework_to_recruit
  ), 2)
)

knitr::kable(kpi_df, caption = "Simulation KPIs for Personnel Pipeline")

The results show that most personnel successfully progress through Training and Active Duty before exiting the system. The average number of steps indicates a moderate system duration, while a smaller portion of individuals experience rework by returning to earlier stages. Overall, the KPIs suggest a stable pipeline with predictable progression and realistic retention behavior.

KPI Visualization and Scenario Comparison Analysis

This section presents a visual summary of the key performance indicators (KPIs) from the military personnel training and deployment pipeline. The plots help illustrate how personnel flow through Recruitment, Training, Active Duty, and Exit. It also compares different simulation scenarios to understand how changes in transition probabilities affect system behavior.

hist(steps_to_exit,
     main = "Distribution of Time to Exit",
     xlab = "Number of Steps",
     ylab = "Frequency")

barplot(kpi_df$Value,
        names.arg = kpi_df$KPI,
        las = 2,
        main = "Key Performance Indicators",
        ylab = "Value")

# Example scenario adjustment (higher training success)
P_improved <- P
P_improved["Recruit", "Training"] <- 0.9
P_improved["Recruit", "Exit"] <- 0.1

# Re-simulate quick comparison
simulate_summary <- function(P) {
  steps <- replicate(5000, length(simulate_personnel(P, "Recruit", states)) - 1)
  mean(steps)
}

base_mean <- simulate_summary(P)
improved_mean <- simulate_summary(P_improved)

scenario_df <- data.frame(
  Scenario = c("Base Model", "Improved Training"),
  Avg_Steps = c(base_mean, improved_mean)
)

barplot(scenario_df$Avg_Steps,
        names.arg = scenario_df$Scenario,
        main = "Scenario Comparison: Average Time in System",
        ylab = "Average Steps to Exit")

The visual outputs clearly show how personnel are distributed across different stages of the system. The histogram highlights the variation in time spent before exiting, while the KPI bar chart summarizes overall system performance in a single view.

The scenario comparison demonstrates that improving training success rates reduces the overall time personnel spend in the system and increases efficiency. This shows how small changes in transition probabilities can significantly impact workforce flow and retention outcomes.

Military Personnel Pipeline Efficiency

This section compares two scenarios in the military personnel pipeline: the baseline system and an improved training process. The comparison is based on simulation results that measure the average number of steps required for personnel to move from Recruitment to Exit. The goal is to understand how changes in transition probabilities affect overall system efficiency and personnel flow.

P_base <- P
P_fast <- P

P_fast["Training", ] <- c(0.10, 0.10, 0.10, 0.70)
P_fast["Training", ] <- P_fast["Training", ] / sum(P_fast["Training", ])

simulate_personnel <- function(P, start = "Recruit", states) {
  cur <- start
  path <- cur

  while (cur != "Exit") {
    cur <- sample(states, size = 1, prob = P[cur, ])
    path <- c(path, cur)

    if (length(path) > 1000) break
  }

  path
}

nSim <- 5000

run_model <- function(P) {
  steps <- replicate(nSim, {
    p <- simulate_personnel(P, "Recruit", states)
    length(p) - 1
  })
  mean(steps)
}

base_result <- run_model(P_base)
fast_result <- run_model(P_fast)

scenario_results <- data.frame(
  Scenario = c("Baseline", "Improved Training"),
  Avg_Steps = c(base_result, fast_result)
)

scenario_results

           Scenario Avg_Steps
1          Baseline    3.3958
2 Improved Training    2.3492

barplot(scenario_results$Avg_Steps,
        names.arg = scenario_results$Scenario,
        main = "Scenario Comparison: Military Personnel Pipeline Efficiency",
        ylab = "Average Time to Exit (Steps)")

The results show a clear difference between the baseline and improved scenarios. In the improved training scenario, personnel reach the Exit state in fewer steps on average, indicating a more efficient pipeline with faster progression through the system.

Probability Matrix

This section presents the Markov transition matrix in percentage form to make the movement of military personnel across states easier to interpret. Instead of raw probabilities, values are converted into percentages to clearly show how likely personnel are to move between Recruitment, Training, Active Duty, and Exit at each stage.

# Convert probability matrix into percentage format
P_percent <- round(P * 100, 2)
P_percent

            to
from         Recruit Training ActiveDuty Exit
  Recruit          0       80          0   20
  Training         0       10         70   20
  ActiveDuty       0        0         60   40
  Exit             0        0          0  100

kable(P_percent,
      caption = "Transition Matrix in Percentage Form (%)")

Heatmap Visualization

library(ggplot2)
library(reshape2)

P_df <- melt(P_percent)
colnames(P_df) <- c("From_State", "To_State", "Probability")

ggplot(P_df, aes(x = To_State, y = From_State, fill = Probability)) +
  geom_tile() +
  geom_text(aes(label = Probability), color = "black") +
  labs(title = "Transition Matrix Heatmap (%)",
       x = "To State",
       y = "From State")

The percentage view of the transition matrix clearly highlights the dominant movement patterns in the system. Higher percentages indicate the most likely transitions between stages, such as progression from Training to Active Duty or exit from the system

Conclusion

This assignment developed a four-state Markov model to simulate a military personnel training and deployment pipeline, capturing the flow from recruitment to exit. The model followed the Markov property ensuring that transitions depend only on the current state and the analysis confirmed that the system is stable with Exit as an absorbing state.

Simulation results showed that most personnel successfully progress through Training and Active Duty before leaving the system. The KPIs provided useful insights into retention, time spent in the system and overall flow efficiency. Additionally, scenario analysis demonstrated that adjusting transition probabilities such as improving training success can reduce the time to exit and improve system performance.