Call For System Demonstrations

The system demonstration track at NAACL-HLT 2021 is a venue for papers describing system demonstrations, ranging from early prototypes to mature production-ready systems. Publicly available open-source or open-access systems are of special interest.

All accepted demos are published in a companion volume of the conference proceedings. We expect at least one of the authors to present a live demo during a demo session at NAACL 2021, with an accompanying poster.

Submissions will undergo a single-blind reviewing process. So papers may include author and affiliation information, and freely make references to previously published material, and URLs.

This year we are incorporating ethical considerations in the review process. Authors will be allowed extra space after the main content for a broader impact statement or other discussion of ethics. Please review the ethics policy before submitting.

Areas of Interest

Areas of interest include all topics related to theoretical and applied computational linguistics, such as (but not limited to) the topics listed on the main conference website. Submitted systems may be of the following types:

  • Natural Language Processing systems or system components
  • Application systems using language technology components
  • Software tools for computational linguistics research
  • Software for demonstration or evaluation
  • Software supporting learning or education
  • Tools for data visualization and annotation
  • Development tools

Please note: Commercial sales and marketing activities are not appropriate in the Demonstrations Program.

Important Dates

Paper submission deadline Monday January 11, 2021
Notification of acceptance Wednesday March 17, 2021
Camera ready submission Sunday April 18, 2021
Final notification for papers requiring ethics re-review Friday May 7, 2021
Publication date Monday May 24, 2021

All deadlines are 11.59 pm UTC -12h (“anywhere on Earth”).

Best Demo Award

We will present a Best Demo Paper Award. The winner will be chosen based on the contribution and completeness of the system, as assessed by the reviewers and also based on the live demo at the conference.

Submission Guidelines

All submissions should be made electronically via START. Submissions must include:

  • A paper describing the motivation and technical details of the system, including visual aids (e.g., screenshots, snapshots, or diagrams). See examples of recently accepted demo papers in EMNLP 2020, ACL 2020, NAACL 2019, NAACL 2018, ACL 2018, ACL 2017, EMNLP 2017, NAACL 2016, ACL 2015, NAACL 2015.

    • The papers must be no more than six (6) pages, plus extra space for an optional ethics/broader impact statement and unlimited references. Please use the main NAACL 2021 paper style files. Accepted papers will be given one additional page of content so that reviewers’ comments can be taken into account.

    • Any papers that do not follow the official style guidelines and page limits will be automatically rejected.

  • A short (max. 2 minutes) video demonstrating the system. This video will be used to evaluate the paper, but won’t be published unless requested.

    • A screencast with audio narration is a natural choice for demos which can be presented on a screen. Otherwise, a video of a user interacting with the system can be used (see this video demonstrating the sounding board system as an example).

    • The production quality of the video is not of interest. Hence, we encourage the videos to be simply a screencast of the software that is getting demoed, with zero to minimal editing efforts.

    • We recommend that you publish your video to YouTube or another website and include the link in your paper. If you prefer not to publicly upload a screencast, please submit the video (in MPEG4 format). The video must be included as supplementary material when you submit your paper through SoftConf.

In addition, we strongly recommend all demos are provided via one of the following formats: (1) A live demo website; or (2) A website with a downloadable installation package of the demo; unless it is impossible because special hardware is required or access is otherwise limited.

Submission Policy

The paper has to be original, written specifically for this conference and cannot be submitted elsewhere. The paper must also report on a substantial improvement (>30%), if the system that is being reported has been reported elsewhere already.

Authors submitting more than one paper to NAACL 2021 must ensure that the submissions do not overlap significantly (>25%) with each other. Submissions of identical or closely related work to multiple tracks at NAACL 2021 (Research, SRW, Industry, or Demo) will be rejected by all tracks.

Reviewing Policy

Reviewing will be single-blind, so authors do not need to conceal their identity. The paper should include the authors’ names and affiliations. Self-references are also allowed. Relevant papers that meet formatting requirements will be assessed on the basis of their relevance to the demo track, contribution, clarity, completeness and novelty.

Ethics Policy

Authors are required to honor the ethical code set out in the ACM Code of Ethics. The consideration of the ethical impact of our research, use of data, and potential applications of our work has always been an important consideration, and as artificial intelligence is becoming more mainstream, these issues are increasingly pertinent. We ask that all authors read the code, and ensure that their work is conformant to this code. We reserve the right to reject papers on ethical grounds, where the authors are judged to have operated counter to the code of ethics, or have inadequately addressed legitimate ethical concerns with their work.

Authors will be allowed extra space after the 6th page for a broader impact statement or other discussion of ethics. The NAACL demonstration review form will include a section addressing these issues and papers flagged for ethical concerns by reviewers will be further reviewed by an ethics committee. Note that an ethical considerations section is not required, but papers working with sensitive data or on sensitive tasks that do not discuss these issues will not be accepted. Conversely, the mere inclusion of an ethical considerations section does not guarantee acceptance. In addition to acceptance or rejection, papers may receive a conditional acceptance recommendation. Camera-ready versions of papers designated as conditional accept will be re-reviewed by the ethics committee to determine whether the concerns have been adequately addressed. Please read the ethics FAQ (shared with the main conference) for more guidance on some problems to look out for and key concerns to consider relative to the code of ethics.

Demonstration Co-chairs

  • Avi Sil (IBM Research AI)
  • Victoria Lin (Facebook AI)