Thanks to all those who responded to my earlier post. Here is another
question I've been discussing with a colleague for a while and I would be
interested in hearing your opinion about it!
Would you agree that there are differences in the way we should validate
(process interactive) ABS models compared to (event driven) DES models?
In my opinion ABS model output has to be treated (understood) as a
qualitative output rather than a quantitative output (even though the
simulation model output itself is of course quantitative).
We want to study how micro level processes affect macro level outcome. The
macro behaviour emerges from the micro decisions of the individual agents
(Pourdehnad et al, 2002). We cannot possibly model all the decisions and the
uncertainty inherent in these decisions using rules. Therefore we cannot
expect the quantitative performance measures of the simulation model to
reflect the quantitative performance of the real system. We can sometimes
get close to reproduce real system performance output values if we model
very simple systems (e.g. individual processes) or systems where the
behaviour of the entities is very much predictable (e.g. production line
models) or if we have the resources to develop very sophisticated simulation
models (e.g. some of the models developed by the MoD). However, in general I
would argue that the more we abstract the less likely we are to match the
performance of the real system in quantitative terms but we will still be
able to observe the system evolution process over time.
Consequently, if we somehow match the performance of the real system for the
cases validated we are very lucky but is there a guarantee that this holds
for all possible cases we want to investigate (NB: Remember, that for
quantitatative validation we only check the inputs against the outputs and
not the process involved in getting from the input to the output – we
usually imply that the process is correct when we get the correct output)?
Especially, when we model people there is a lot of behavioural uncertainty
that we have to consider and that would be (at least in my opinion)
impossible to validate! It would be difficult to consider all possibilities
for irrational and counterintuitive behaviour and the occasions when they
appear, but often these are important to consider when investigating the
evolution of a human centred system.
Here are some questions to discuss, focussing on ABS models of human-centred
systems:
What do we need to do to be able to claim we have a valid ABS model when we
model human-centred systems? To what extent can we validate such an ABS
model? On what level do we need to validate (micro level; macro level; both)?
Should we use different methods depending on the ABS model’s level of
abstraction?
Can we use ABS models as quantitative predictors (like we often do with DES
models)?
Thanks for your comments,
Peer-Olaf Siebers
Reference:
Pourdehnad, J., Maani, K., and Sedehi, H. (2002). “System Dynamics and
Intelligent Agent-Based Simulation: Where is the Synergy?” Proceedings of
the 20th International Conference of the System Dynamics Society, 28 July -
1 August 2002, Palermo, Italy.
|