The Cluster Benchmarking Task Force of the International Federation of Classification Societies (IFCS) is calling for neutral benchmarking studies in cluster analysis. This call is part of a challenge connected with the 2019 IFCS conference in Thessaloniki, Greece, August 26-29.
    To achieve a cumulative building of knowledge on clustering and classification, careful attention to benchmarking (performance comparison of methods) is very important. New methods of data pre-processing, new data-analytic techniques, and new methods of output post-processing, should be extensively and carefully compared with existing alternatives, and existing methods should be part of neutral comparison studies. Benchmarking studies can frequently been found in supervised learning, but are less common in unsupervised learning. 
   The Task Force has written a white paper (for a preprint, see: https://arxiv.org/abs/1809.10496) that addresses the theoretical and conceptual underpinnings of benchmarking in cluster analysis, and some practicalities for performing sound benchmarking studies. The Task Force is now calling for individuals to contribute neutral benchmarking studies on cluster analysis as part of a challenge. 

Rules for submission:
* Contributions for the challenge must be e-mailed to [log in to unmask] .
* A contribution should comprise a report on a benchmarking study in the context of cluster analysis. This report should have a length of between 5 and 15 pages, including text, tables, figures and references; other materials can be submitted as separate files.
* The benchmarking study may compare methods of data pre-processing, clustering (potentially but not necessarily including the estimation of cluster parameters such as the number of clusters), and/or output post-processing, and may use empirical or simulated data.
* Contributors should have a neutral point of view with respect to the methods studied in the contribution. This means that they should not be an author or co-author of one or more of the studied method(s). It is also expected that the contributors do not have a personal interest to promote any of the involved methods in particular.  
* The deadline for submissions is March 15, 2019, 12:00 pm CET. 

Competition guidelines:
* Contributions to the challenge will be evaluated by the IFCS Task Force on Benchmarking.
* Major evaluation criteria will be: 
* the technical correctness and clarity of the report, 
* that the study shows a critical reflection on principles on which a sound benchmarking study should be based (which may be either in line or at odds with the principles outlined in the white paper contributed by the Task Force), 
* a detailed specification of the choices made in the benchmarking study at issue along with a justification of these based on the principles referred to above.
* Up to 8 contributions will be selected for a short presentation during one or two invited sessions at the 2019 IFCS conference in Thessaloniki, Greece (such a presentation will not conflict with giving another presentation at that conference). The authors of the selected contributions will be notified no later than April 25, 2019. Authors of selected contributions who cannot attend the conference will be invited to prepare a few slides on their contribution. 
* During the closing session of the IFCS conference one or two of the selected contributions will be proclaimed winners of the challenge. The winners will receive a CRC/Chapman and Hall book voucher.
* Contact for questions: [log in to unmask] .

All researchers interested in clustering and classification are most welcome to participate in this challenge!

Task force members: Anne-Laure Boulesteix, Rainer Dangl, Nema Dean, Isabelle Guyon, Christian Hennig, Friedrich Leisch, Douglas Steinley, Iven Van Mechelen, and Matthijs Warrens.

You may leave the list at any time by sending the command

SIGNOFF allstat

to [log in to unmask], leaving the subject line blank.