CALL FOR PAPERS/SUBMISSIONS
VERIFICATION, VALIDATION, AND TESTING OF LEARNING SYSTEMS
- a workshop in conjunction with the NIPS 2004 conference - December 17/18, 2004; Whistler, BC
For details see http://www.dmargineantu.net/nips2004
We invite contributions on the topics outlined below and on related topics. The submissions can be anywhere between one page position papers and 8-page full papers.
===========================
Compared to the attention given to the development of new learning methods, our community has devoted only very little effort to developing princlpled approaches for (1) assessing the goodness of complex systems that contain learning components, (2) estimating the quality of the outputs of learned models in the context of the actual problem that needs to be addressed, (3) assessing online learning methods, (4) evaluating learning methods employed in safety-critical tasks, and for (5) understanding the tradeoffs between robustness and risk in making complex decisions.
Learning has the potential to provide several key features to adaptive, autonomous, and large scale systems: adaptability to changing environments, capability of processing different types of sensor data, and addressing multiple objectives in parallel - to name just a few. In the meantime, most of these systems require a reliable deployment and operation. In other words, for most applications, in order to be deployed, learning components need to be proven as trustable to the users (engineers, designers, quality control specialists). Failures of these systems can occur and will occur, regardless of whether they contain learned or learning components. Therefore, questions such as "what are the tradeoffs for improving the quality of the outputs of a learning system in a certain region of the space?", or "what can be inferred (regarding future decisions) from observing the operation of a learning system?" have deep ramifications and, and if answered can result in learning technology having a deeper impact on newly developed systems.
The workshop aims to explore the requirements of practical applications that make use of, or could benefit from learning methods - such as adaptive flight control systems, autonomous navigation, health management systems, neuroadaptive systems, robotics, or security.
The purpose of the workshop is to bring together researchers and users of learning and adaptive systems and to create a forum for discussing recent advances in verification, validation, and testing of learning systems, to understand better the practical requirements for developing and deploying learning systems, and to inspire research on new methods and techniques for verification, validation, and testing.
Topics of interest include, but are not limited to:
- formal specification of learning requirements and verification
- statistical testing and validation of learned models
- metrics for the performance of learning systems
- integration of learning components into adaptive control systems
- statistical and logical inference for validation purposes
- integration of online learning into large scale systems
- learning for safety-critical applications
- new approaches for machine learning software development
- algorithms and tools for monitoring learning and adaptive systems
- analysis of the robustness vs. risk tradeoff
- validation and testing for sequential decision making
- V&V and testing of learning in robotics
- V&V of neuroadaptive systems
- new problems and applications that require principled assessment of learning
The organizers welcome submissions on all topics outlined above and areas related to these topics. The submissions can be anywhere between one page position papers and 8-page full papers.
The following submission formats are suggested: postscript, PDF, MS Word, or ASCII; 10-12 pt. font, minimum 1 inch (2.5cm) margins; the title of the paper, the name, the affiliation, and the e-mail address of the each author should be listed at the beginning of the first page.
The submissions should be sent by e-mail to [log in to unmask] by October 20, 2004.
The submissions accepted for presentation will be posted on the workshop webpage and hard copies will be distributed to participants.
If you have any questions, please do not hesitate to contact Dragos Margineantu at [log in to unmask]
Organizers:
Dragos Margineantu, Boeing, Mathematics and Computing Technology
Johann Schumann, RIACS / NASA Ames Research Center
Pramod Gupta, QSS Inc. / NASA Ames Research Center
Michael Drumheller, Boeing, Mathematics and Computing Technology
|