================
CEC-BBOB: BLACK-BOX OPTIMIZATION BENCHMARKING at IEEE CEC'2015 (CEC-BBOB)
Special session on Unbounded Real-Parameter Blackbox Optimization
organized as part of the
IEEE Congress on Evolutionary Computation, May 25-28, 2015, Sendai, Japan
================
Benchmarking of optimization algorithms is crucial to assess performance
of optimizers quantitatively, understand weaknesses and strengths of each
algorithm and is the compulsory path to test new algorithm designs.
A thorough benchmarking methodology has been defined in [1] and is
implemented within the COCO framework (http://coco.gforge.inria.fr/)
that furnishes most of the tedious tasks of benchmarking
for the participants:
* choice of well-motivated single-objective benchmark functions
and their implementation in Matlab, C, Java, R, and Python,
* design of an experimental set-up,
* generation of data output, and
* post-processing and presentation of the results in graphs and tables
(up to already prepared LaTeX templates for writing papers).
Note that for the CEC-BBOB-2015 special session, we provide essentially
the same test-suite as in the previous editions of BBOB held at GECCO.
We provide 2 testbeds,
* noise-free and
* noisy,
and we distinguish between
* an expensive optimization scenario (where a focus on the first
100D function evaluations is assumed) and
* a general scenario for which we do not limit the maximal number
of function evaluations made.
The participants can freely choose any or all of them. We entirely
ban different parameter settings for different test functions and
encourage analyses that study the impact of parameter setting changes.
[1] Real-Parameter Black-Box Optimization Benchmarking: Experimental Setup,
http://coco.lri.fr/downloads/download13.09/bbobdocexperiment.pdf.
The Special Session Papers
==========================
We encourage any submission that is concerned with black-box optimization
benchmarking of continuous optimizers, for example papers that:
* decribe and benchmark new or not-so-new algorithms on the CEC-BBOB-2015
testbed,
* compare new or existing algorithms from the COCO database, or
* analyze the data obtained in previous editions of BBOB.
Similarly to the BBOB-2013 edition, we also focus on benchmarking
optimization algorithms for expensive optimization (with limited budget)
which especially invites to benchmark surrogate-assisted algorithms
(e.g. based on kriging, support vector machines etc.). Participants are
also encouraged to use the existing database for statistical analyses or
for designing a portfolio of algorithms.
This session is related to the special session and competition
on bound constrained optimization organized by
Ponnuthurai Nagaratnam Suganthan et al.
(http://www.ntu.edu.sg/home/EPNSugan/index_files/CEC2015/CEC2015.htm).
Submissions to both sessions are encouraged while we require here
that papers presenting benchmarking results follow the benchmarking
methodology of the COCO framework. Papers discussing benchmarking
methodologies in general are also welcome.
Organisation of the Special Session during the CEC conference
=============================================================
During the special session, algorithms and results will be presented
by the participants. An overall analysis and comparison will be accomplished
by the organizers and all submitted papers will be critically reviewed
as for any other CEC'2015 paper. A planned plenary discussion on future
improvements will, among others, address the question, of how the testbed
should evolve.
Support Material and Downloads
==============================
All the support material needed to benchmark your algorithm(s), process the
results, and compile the final papers will be available at the CEC-BBOB-2015
page (http://coco.gforge.inria.fr/doku.php?id=cec-bbob-2015).
The final code and LaTeX templates following the CEC'2015 guidelines
will be available in early November. To ensure/check that your optimizer
and the COCO framework work well together, the currently available downloads
(for past BBOB workshops) can be used. The code for the final experiments
and post-processing obey the very same interface. Subscribe for
special session announcements (see below) to be notified on updates.
Important dates
===============
01/11/2014: code released
19/12/2014: submission deadline
20/02/2014: acceptance notification
13/03/2015: final paper version due
25-28/05/2015: workshop at CEC'2015
Contact and Mailing List
========================
Subscribe to our discussion mailing list by following this link
http://lists.lri.fr/cgi-bin/mailman/listinfo/bbob-discuss.
To receive announcement about the workshop, send an email to the BBOB team
at bbob_at_lri.fr with title "register to BBOB announcement list".
Organization Committee
======================
Youhei Akimoto, Shinshu University, Nagano, Japan
Anne Auger, Inria Saclay - Ile-de-France, Orsay, France
Dimo Brockhoff, Inria Lille - Nord Europe, Villeneuve d'Ascq, France
Nikolaus Hansen, Inria Saclay - Ile-de-France, Orsay, France
Olaf Mersmann, TU Dortmund University, Dortmund, Germany
Petr Pošík, Czech Technical University, Prague, Czech Republic
