Dear Catholijn
Rosaria Conte kindly sent me a list of papers she suggested as
counter-examples. Among those suggested was the ICMAS98 DESIRE paper
(Brasier et al, "Compositional Design and Verification...". I was
struck by the similarity of your verification procedures with what I am
proposing as validation. The reason, I think, for your interest in
verification and ours in validation is that we are concerned with
developing models that usefully reflect and accurately capture existing
real-world phenomena while, if I understand these issues correctly,
you are interested in the design of systems that do not yet exist but
are required to meet particular performance standards. Since applied
social scientists are (or at least should be) developing formalisms that
correspond unambiguously to observed social processes, validation is the
appropriate primary concern in specifying the primitives of
computational models (= systems). The requirement in software
engineering to ensure that software systems that are not yet implemented
will perform in specified (typically novel) ways makes verification the
appropriate primary concern in building those systems.
I have been arguing for some time now that the social science MAS
community and the computer science MAS community have more in common
than applications of the same formal logics. The ICMAS98 DESIRE paper
supports this argument in that the methodological issues are virtually
identical except that some issues of verification in software
engineering correspond essentially to issues of validation in applied
social simulation modelling.
This distinction implies a parallel difference in primitives. In
DESIRE, the primitives are verified in relation to a temporal belief
logic whereas I suggested in my proposal that the primitives comprising
the descriptions of agents should be validated against the behaviour of
observed decision making units and processes. The methodologies
associated with both DESIRE and SDML are intended to ensure that the
formal properties of the systems produced are unambiguous by virtue of
being consistent and sound. I was suggesting what I now believe you
have already been doing in relating the characteristics of the
primitives to the properties of the system by a process of successive
aggregation. My colleague Oswaldo Teran has pointed out that we are
seeking homomorphic transformation (Zeigler, Theory of Modelling and
Simulation, 1984) (or its qualitative equivalent) from one level to the
next level up.
Three final remarks:
1. The worry expressed in my initial note on MAS methodology was (and
remains) that the attempt to develop social theory bottom up (i.e.
from formalisms verified in relation to problems designed to enable
the formalisms to be effective) has not led to a validated social
theory or even validated approaches to policy analysis in the
nearly century and a half since Jevons applied the differential
calculus to utility theory.
2. DESIRE is not subject to this worry because of its explicit concern
with the development of usefully scaled systems. In the ICMAS98
paper, you conclude with suggestions for filling in the formal gaps
between primitives and top-level systems. This is very different
from the suggestion that a test problem has been addressed
satisfactorily and the need is to develop the agent specifications
to deal with larger-scale problems. These test problems are
problems designed to make the agent specification effective rather
than problems that test the efficacy of the specification.
3. The models or systems produced for model-based policy analysis can
be validated against experience but should also be verified against
some formalism since their application will be to circumstances not
yet experienced. Understanding the properties of the models in
order to inform the process of policy analysis when forcasting is
not a feasible option is, in my view, essential. I believe this
position to be common ground for both applied social scientists and
computer scientists working in MAS.
Regards,
Scott
--
Scott Moss
Director
Centre for Policy Modelling
Manchester Metropolitan University
Aytoun Building
Manchester M1 3GH
UNITED KINGDOM
telephone: +44 (0)161 247 3886
fax: +44 (0)161 247 6802
http://www.cpm.mmu.ac.uk/~scott
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
