Data 604 Discussion 03 Farhana Zahir
Verification and Validation in simulation¶
The simulation model is valid only if the model is an accurate representation of the actual system, else it is invalid.
Validation and verification are the two steps in any simulation project to validate a model.
1) Validation is the process of comparing two results. In this process, we need to compare the representation of a conceptual model to the real system. If the comparison is true, then it is valid, else invalid.
2) Verification is the process of comparing two or more results to ensure its accuracy. In this process, we have to compare the model’s implementation and its associated data with the developer's conceptual description and specifications.
from IPython.display import Image
Image(filename = "validation.jpg", width=800, height=800)
Verification and Validation techniques:¶
Verification & Validation Techniques There are various techniques used to perform Verification & Validation of Simulation Model. Following are some of the common techniques −
Techniques to Perform Verification of Simulation Model
Following are the ways to perform verification of simulation model :
1) By using programming skills to write and debug the program in sub-programs.
2) By using “Structured Walk-through” policy in which more than one person is to read the program.
3) By tracing the intermediate results and comparing them with observed outcomes.
4) By checking the simulation model output using various input combinations.
4) By comparing final simulation result with analytic results.
Techniques to Perform Validation of Simulation Model
Step 1 − Design a model with high validity. This can be achieved using the following steps −
-The model must be discussed with the system experts while designing.
-The model must interact with the client throughout the process.
-The output must be supervised by system experts.
Step 2 − Test the model at assumptions data. This can be achieved by applying the assumption data into the model and testing it quantitatively. Sensitive analysis can also be performed to observe the effect of change in the result when significant changes are made in the input data.
Step 3 − Determine the representative output of the Simulation model. This can be achieved using the following steps −
-Determine how close is the simulation output with the real system output.
-Comparison can be performed using the Turing Test. It presents the data in the system format, which can be explained by experts only.
-Statistical method can be used for compare the model output with the real system output.What was done in the assigned simulation journal reading to conduct V&V?
In the journal article, GAMS was used to simulate and optimize a model on relocation of deployable military hospitals. The model validation based on realistic historical data from Operation Iraqi Freedom (real system output).
Quoting the article,
'The network model is built around an intuitive objective function, one that is derived from military doctrine.'This implies there has been discussions with the client while being designed.
Share some code from last week that was particularly rewarding or frustrating. We will comment on it.
def run_multiple_simulations(p1, p2, num_steps, num_runs):#takes as parameter no of runs for simulation results = TimeSeries()
for i in range(num_runs):
state = run_simulation(p1, p2, num_steps) #runs the simulation for num_runs
results[i] = state.olin_empty + state.wellesley_empty #total unhappy customers for both
return results, results.mean()
I found the above code particularly rewarding because it saves the results of running the simualtion a number of times, making it easier to compare how the results are changing.However, I have been trying but could not come up with a way to change the p1 and p2 values for each run of the simulation using linspace.