* We apologize if you receive multiple copies of this CfP *
* For the online version of this Call, visit: https://recsys.acm.org/recsys22/call/reproducibility *
RecSys strongly encourages the submission of algorithmic papers that repeat and analyze prior work. We distinguish between:
* replicability papers, which repeat prior experiments using the original source code and datasets to show how, why, and when the methods work (or not); and
* reproducibility papers, which repeat prior experiments, either using the original code or using new implementations, in new contexts (e.g., different application domains and datasets, different evaluation methodologies and metrics) to further generalize and validate (or not) previous work.
Submissions regarding replicability or reproducibility papers are welcome in all areas related to recommender systems (see the main track for the list of topics).
In both replicability and reproducibility papers, we expect authors to provide all materials required for repeating the tests performed, including code, data, and clear instructions on how to run the experiments. Submissions from the same authors of the reproduced experiments will not be accepted. Failure to provide all materials will result in desk rejection.
Each accepted paper will be included in the conference proceedings and presented in a plenary session as part of the main conference program. We encourage authors to create a companion website for each paper where details on how the code and data can be reproduced or re-used.
REVIEW CRITERIA
Both replicability and reproducibility papers will be evaluated along the following criteria:
* Novelty
* What is new about the reproduced experiments?
* Was the original work not supported from the theoretical point of view?
* Were the original experiments unclear about important points or lacking confirmation for some of the original claims?
* Do the reproduced experiments bring more solid conclusions with new datasets, metrics, and unbiased evaluation setups?
* Are there new experiments that allow for a better understanding of the impact of previous results?
* Impact
* How important is the reproduction of the experiments to the community?
* How obvious are the conclusions achieved?
* Do the reproduced prior works, if validated, advance a central topic to recommender systems (a topic with a broad applicability or focused on a hot research area)?
* Reliability
* Is the evaluation methodology in line with the research challenges addressed by the reproduced experiment?
* Are the selected baselines representative of the several algorithm types and techniques available?
* Is the hyperparameter tuning strategy properly described?
* Are algorithms and baselines properly tuned?
* Availability
* Are the code and datasets used to reproduce the experiments available to the reviewers at the time of review?
* Is the shared material released in a permanent repository for easy access by researchers?
* Are the reproduced experiments well documented, with all the details required for other researchers to reproduce the experiments?
* Are there discrepancies between what is described in the paper and what is available in the shared material?
* Is the shared material complete with everything you need to exactly replicate the experiments?
This year we will include a short clarification phase (rebuttal), during which authors will be given a chance to submit a brief response to address factual errors in reviews; clarifications may be considered during the discussion phase.
SUBMISSION GUIDELINES
The maximum length is 14 pages (excluding references) in the new single-column format; reviewers will be asked to comment on whether the length is appropriate for the contribution.
It is expected that at the time of submission, code and datasets used to reproduce the experiments will be available under reasonably liberal terms and sufficiently well-documented such that reviewers may consult that documentation as they conduct their reviews.
Formatting
ACM’s archival publication format separates content from presentation in the Digital Library to enhance accessibility and improve the flexibility and resiliency of our publications. Following the ACM publication workflow, all authors should submit manuscripts for review in a single-column format. Instructions for Word and LaTeX authors are given below:
* Microsoft Word: Write your paper using the Submission Template (Review Submission Format). Follow the embedded instructions to apply the paragraph styles to your various text elements. The text is in single-column format at this stage and no additional formatting is required at this point.
* LaTeX: Please use the latest version of the Primary Article Template – LaTeX to create your submission. You must use the “manuscript” option with the \documentclass[manuscript, anonymous]{acmart} command to generate the output in a single-column format which is required for review. Please see the LaTeX documentation and ACM’s LaTeX best practices guide for further instructions. To ensure 100% compatibility with The ACM Publishing System (TAPS), please restrict the use of packages to the whitelist of approved LaTeX packages.
Authors are strongly encouraged to provide “alt text” (alternative text) for floats (images, tables, etc.) in their content so that readers with disabilities can be given descriptive information for these floats that are important to the work. The descriptive text will be displayed in place of a float if the float cannot be loaded. This benefits the author as well as it broadens the reader base for the author’s work. Moreover, the alt text provides in-depth float descriptions to search engine crawlers, which helps to properly index these floats. Additionally, authors should follow the ACM Accessibility Recommendations for Publishing in Color and SIG ACCESS guidelines on describing figures.
Should you have any questions or issues going through the instructions above, please contact support at [log in to unmask] for both LaTeX and Microsoft Word inquiries.
Accepted papers will be later submitted to ACM’s new production platform where authors will be able to review PDF and HTML output formats before publication.
Anonymity
The peer review process is mutually anonymous (double-blind). This means that all submissions must not include information identifying the authors or their organization. Specifically, do not include the authors’ names and affiliations, refer to your previous work in the third person (e.g., “Shapira and Ekstrand (2022) recommended that RecSys submissions be anonymized by referring to the authors’ prior work in the third person.”), and avoid providing any other information that would allow reviewers to identify the authors, such as acknowledgments of individuals and funding sources. However, it is acceptable to explicitly refer in the paper to the companies or organizations that provided datasets, hosted experiments or deployed solutions if there is no implication that the authors are currently affiliated with the mentioned organization. Reviewers are instructed not to search for tech reports, pre-prints, and other information about your research. Your responsibility is focused on making sure that the paper submission itself does not reveal your identity as author.
Ethical Review for Human-Subjects Research
ACM RecSys expects all authors to comply with ethical and regulatory guidelines associated with human subjects research, including research involving human participants and research using personally identifiable data. Papers reporting on such human subjects research must include a statement identifying any regulatory review the research is subject to (and identifying the form of approval provided), or explaining the lack of required review. Reviewers will be asked to consider whether the research was conducted in compliance with applicable ethical and regulatory guidelines.
We encourage authors to consider further ethical implications and broader impacts of their work, and to discuss these in an appropriate section of their papers; “A Guide to Writing the NeurIPS Impact Statement” provides non-binding guidance on some of the kinds of things authors may wish to consider.
Originality
Each paper should not be previously published or accepted to any peer-reviewed journal or conference, nor currently under review elsewhere (including as another paper submission for RecSys 2022). Papers published in workshop proceedings may only be submitted if the RecSys submission includes at least 30% new content; such papers must also reference the original workshop paper on the submission form (but not in the anonymized paper).
Plagiarism
Plagiarized papers will not be accepted for RecSys 2022. Our committees will be checking the plagiarism level of all submitted papers to ensure content originality using an automated tool.
If you reuse non-novel text from a prior publication (e.g., the description of an algorithm or dataset), please be sure to cite the prior publication as the source of that text. If you have questions about reuse of text or simultaneous submission, please contact the program chairs at least one week prior to the submission deadline. Please refer to the ACM Publishing License Agreement and Authorship Policy for further details.
Papers violating any of the above guidelines are subject to rejection without review and cases may be referred to the ACM Publications Ethics and Plagiarism committee for further action where warranted.
Patenting
Please take note that the official publication date is the date the proceedings are made available in the ACM Digital Library. This date may be up to two weeks prior to the first day of the conference. The official publication date affects the deadline for any patent filings related to published work.
SIGCHI SUBMITTER AGREEMENT
RecSys 2022 is a SIGCHI conference and making a submission to a SIGCHI conference is a serious matter. Submissions require time and effort by SIGCHI volunteers to organize and manage the reviewing process, and, if the submission is accepted, the publication and presentation process. Thus, anyone who submits to RecSys 2022 implicitly confirms the following statements:
* I confirm that this submission is the work of myself and my co-authors.
* I confirm that I or my co-authors hold copyright to the content, and have obtained appropriate permissions for any portions of the content that are copyrighted by others.
* I confirm that any research reported in this submission involving human subjects has gone through the appropriate approval process at my institution.
* I confirm that if this paper is accepted, I or one of my co-authors will present the paper at the conference, either in person or through a conference-designated remote presentation option. Papers that are not presented at the conference by an author may be removed from the proceedings at the discretion of the program chairs.
IMPORTANT DATES
Abstract submission deadline: April 26th, 2022
Paper submission deadline: May 3rd, 2022
Author notification: June 28th, 2022
Camera-ready version deadline: July 26th, 2022
Deadlines refer to 23:59 (11:59pm) in the AoE (Anywhere on Earth) time zone.
REPRODUCIBILITY CHAIRS
Paolo Cremonesi, Politecnico di Milano, Italy
Ludovik Çoba, Koa Health, Spain
E-mail: [log in to unmask]
########################################################################
To unsubscribe from the CSCW-ALL list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CSCW-ALL&A=1
This message was issued to members of www.jiscmail.ac.uk/CSCW-ALL, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/
|