Call for Participation
ACM SIGIR 2004 Workshop on
New Directions For IR Evaluation: Online Conversations

Sheffield, UK, July 29th, 2004
http://www.ict.usc.edu/~leuski/sigir04-oc/

Online conversations encompass a broad range of interactive
experiences, including personal electronic mail, mailing lists,
instant messaging, chat rooms, Usenet newsgroups, threaded Web-based
discussion lists, and massive multi-player on-line role playing games.
These types of interactions raise challenges for information retrieval
research that are under-explored, including very short "documents" that
can be grouped in one or more ways into time-based conversational
"threads," widespread use of sub-languages, characteristics of
informality such as typographical errors, nonstandard abbreviations,
and the use of emoticons. Conversations naturally link participants,
so the design space for retrieval systems can be based on relationships
among those who do the writing, and broader characteristics such as
expertise and trust in addition to what was actually written.

Existing test collections do not yet support these types of inquiry,
and it is not
yet clear how (or even whether) information seeking within online
conversations should be modeled in the design of test collections.
The goal for this workshop will therefore be to articulate the
challenges and to begin to identify the opportunities for productive
IR research in the domain of online conversations.

Goal

Our goal for this workshop is to identify opportunities for important
new research on online conversations and how this research can be
facilitated through the creation of standard community-wide resources
(e.g., development of standard test collections).  To do this, we plan
to bring together researchers from several communities with expertise
in aspects of this question (e.g., information retrieval, online
communities, computer-supported cooperative work, recommender system,
computational linguistics, and text data mining) to explore the
present state of the art and to synthesize multiple perspectives on
evaluation of systems for working with online conversations.  We will
count the workshop as successful if it results in a concrete
understanding of what types of evaluation resources are needed and how
those resources can practically be created.

Participation

This one-day workshop will be structured around paper presentations,
panel discussions, audience interaction, and informal discussions.

Open participation. The workshop is open to all interested
participants. Conference participation or registration is not required
to
attend the workshop. Advance submission of a statement is
describing research interests, and (if appropriate)
proposed discussion points encouraged, but not required.

Important Dates

July 25, 2004  SIGIR conference begins
July 29, 2004  Workshop date

Organizers:

Anton Leuski (co-chair), USC Institute for Creative Technologies
Douglas W. Oard (co-chair), University of Maryland
Abdur Chowdhury, America Online
David Evans, Clairvoyance
Jennifer Preece, University of Maryland, Baltimore County

Contact Information

Questions or comments should be directed to  [log in to unmask] and
[log in to unmask]