2. Verification
Concerned with building the model
right
Comparison of conceptual model
and computer representation
Is the model implemented correctly
in the computer?
Are the inputs and logical
parameters represented properly?
3. Validation
Concerned with building the right
model
Accurate representation of the real
system
This is achieved through the
calibration of the model
Iterative process until accuracy is
acceptable
5. Common sense suggestions
for verification
Have someone check the
computerized model
Make a flow diagram (with logical
actions for each possible event)
Examine model output for
reasonableness
Print the input parameters at the
end of the simulation
6. Common sense suggestions
for verification
Make the computerized
representation as self documenting
as possible
If animated, verify what is seen
Use IRC or debuggers
Use graphical interface
7. Three Classes of Techniques
for Verification
Common sense techniques
Thorough documentation
Traces
8. Calibration and Validation
Validation is the overall process of
comparing the model and its
behavior to the real system and its
behavior
Calibration is the iterative process of
comparing the model to the real
system and making adjustments to
the model, and so on.
9. Iterative Process of
Calibration
REAL SYSTEM
Initial Model
Second
Revision of
Model
First Revision of
Model
Compare Model to
Reality
Compare Revised
Model to Reality
Compare second
Revised Model to
Reality
10. 3 Step Approach by Naylor
and Finger (1967)
Build a model with high face validity
Validate model assumptions
Compare the model input-output
transformations to corresponding
input-output transformations of the
real system
11. Possible validation techniques in
order of increasing cost-value
ratio by Van Horn (1971)
High face validity. Use previous research/
studies/observation/experience
Conduct statistical test for data
homogeneity, randomness, and goodness
of fit test
Conduct Turing test. Have a group of
experts compare model output versus
system output and detect the difference
Compare model output to system output
using statistical tests
12. Possible validation techniques in
order of increasing cost-value
ratio by Van Horn (1971)
After model development, collect
new data and apply previous 3 tests
Build a new system or redesign the
old one based on simulation results
and use this data to validate the
model
Do little or no validation. Implement
results without validating