Skip to yearly menu bar Skip to main content


Guidance for NeurIPS Workshop Proposals 2024

 

by Bo Han, Manuel Gomez Rodriguez, Adil Salim and Rose Yu

With the rapid growth and interest in NeurIPS and its associated workshops, the competition for workshops has increased alongside logistical constraints. In an attempt to mitigate confusion and anxiety regarding what is expected, the workshop chairs have agreed on the following guidelines for proposals to hold a NeurIPS workshop in 2024. Organizers of workshop proposals should take care to respect all guidance provided here and to present explicit answers to the questions implied throughout, as well as explicitly addressing the selection criteria listed below.

Important Dates

  1. Workshop Application Open: April 22, 2024
  2. Workshop Application Deadline: Jun 7, 2024, AoE
  3. Workshop Acceptance Notification: July 12, 2024, AoE
  4. Suggested Submission Date for Workshop Contributions: August 30, 2024, AoE
  5. Mandatory Accept/Reject Notification Date: October 9, 2024, AoE
  6. Workshop Date: December 14 or December 15, 2024.

Note that the submission date for workshop contributions is suggested, and there is a trade-off between how much time workshops give authors to submit papers versus reviewers to review. 

The mandatory author notification deadline is October 9, 2024. Workshops that do not meet this deadline will have their complementary tickets withheld, or the workshop may even be canceled if workshop organizers are not responsive after this deadline.

 

Workshop Format

 

NeurIPS 2024 plans to hold only in-person workshops. Workshops will follow the main conference program on December 14 and December 15. Workshops will be one-day events that spread 7-9 hours, with a maximum of 1-hour remote presentations.

Workshop Goals

  • Workshops provide an informal, dynamic venue for discussion of work in progress and future directions. Good workshops have helped to crystallize common problems, explicitly contrast competing frameworks, and clarify essential questions for a subfield or application area.
  • Workshops are a structured means of bringing together people with common interests to form communities. We expect the workshops to include some form of community building and stand apart from other parts of the NeurIPS program such as Tutorials or Competitions.
 

Selection Criteria

  • The degree to which the proposed workshop is focused on an important and topical problem and the community will find the workshop interesting, exciting, and useful. 
  • The degree to which the proposed program offers an opportunity for discussion.
  • Diversity and inclusion in all forms (see expectations below.)
  • Quality of proposed invited speakers. Workshop organizers are encouraged to confirm tentative interest from proposed invited speakers and mention this in their proposal. Speakers are expected to be present at the workshop in person to give their talk unless there are exceptional circumstances.
  • Organizational experience, potential, and ability of the team.
  • Other dimensions in the expectations below that are not explicitly listed in these criteria.
  • Points of difference. What makes this workshop enticingly different from the hundreds of NeurIPS workshops held previously?
  • Details of logistics for the workshop.
 

Workshop Proposal Format

 

Submissions for workshop organization should be no more than three pages of proposal, plus no more than two pages of organizer information, and unlimited references. The reviewers are not obligated to read anything beyond those. To simplify the preparation of the proposal, we strongly recommend utilizing this template as a foundation for every proposal.

 

The three pages (or fewer) for the main proposal must include:

  • A title and a brief description of the workshop topic and content.
  • A list of invited speakers, if applicable, with an indication of which ones have already agreed and which ones are tentative.
  • An account of the efforts made to ensure demographic diversity of the organizers and speakers. Also an account of any efforts to include diverse participants (e.g., via mentoring, subsidies, or the wording and topics in the call for proposal).
  • An estimate of the number of attendees.
  • A description of special requirements and technical needs.
  • If the workshop has been held before, a note specifying how many submissions the workshop received, how many papers were accepted (extended abstract/long format), and how many attendees the workshop attracted.
  • Optionally, a URL for the workshop website.
 

The two pages (or fewer) for information about organizers must include:

  • The names, affiliations, and email addresses of the organizers, with one-paragraph statements of their research interests, areas of expertise, and experience in organizing workshops and related events. Please highlight how the organizers’ profiles can make the proposed workshop successful. Please also indicate what other workshops (if any) are concurrently being proposed by an organizer. 
  • A list of Program Committee members, with an indication of which members have already agreed. Organizers should do their best to estimate the number of submissions (especially for recurring workshops) in order to (a) ensure a sufficient number of reviewers so that each paper receives 3 reviews, and (b) anticipate that no one is committed to reviewing more than 3 papers. This practice is likely to ensure on-time and more comprehensive and thoughtful reviews.
 

Assessment Process and Criteria  

The workshop chairs will appoint a number of reviewers who will provide written assessments of the proposals against the criteria listed above. Reviewers’ reports will be considered by the workshop chairs who will jointly decide upon the selected workshops (subject to the notes on COIs listed below). The final decisions will be made by the workshop chairs via consensus and judgment; we will not simply add up scores assigned to all the criteria.

Hard Constraints/Workshop Requirements

 
  1. Mandatory Accept/Reject Notification Deadline Before September 30, 2024: By submitting a workshop proposal, workshop organizers commit to notifying those who submit contributions (including talks and posters) to the workshop of their acceptance status before September 30, 2024. A timeline should be included in the proposal that will allow for this. This deadline of September 30 will be published on the NeurIPS main web page and cannot be extended under any circumstances.
     
  2. Use CMT or OpenReview for Contributed Work
    Workshops that accept contributions must use either CMT or OpenReview to manage their submission process. This ensures that accepted submissions can be efficiently uploaded to the NeurIPS.cc site to announce the workshop schedule to NeurIPS attendants clearly.
 
  1. Managing Chair and Reviewer Conflicts of Interest
  1. Workshop chairs and assistant chairs cannot be organizers or give invited talks at any workshop. However, they can submit papers and give contributed talks.
  2. Workshop reviewers cannot review any proposal on which they are listed as an organizer or invited speaker, or on which they have conflicts of interest as defined by the NeurIPS Conflicts of Interest. Moreover, they cannot accept invitations to speak at any workshop they have reviewed after the workshop is accepted.
  3. Workshop chairs and reviewers cannot review or shape acceptance decisions about workshops with organizers from within their organization. (For large corporations, this means anyone in the corporation worldwide).
 
  1. Managing Organizer Conflicts of Interest
  1. Workshop organizers cannot give talks at the workshops they organize. They can give a brief introduction to the workshop and/or act as a panel moderator.
  2. Workshop organizers should state in their proposals how they will manage conflicts of interest in assessing submitted contributions. At a minimum, an organizer should not be involved in the assessment of a submission from someone within the same organization. 
 

Other Guidance and Expectations for Workshop Proposals 

 
  1. We encourage, and expect, diversity in the organizing team and speakers. This includes the diversity of viewpoints and thinking regarding the topics discussed at the workshop, gender, race, affiliations, seniority, etc. If a workshop is part of a series, the organizer list should include people who have not organized it in the past. Organizers should articulate how they have addressed diversity in their proposal in each of these senses. Invited speakers should not give the same talk or very similar talks at multiple workshops.
  2. Since the goal of the workshop is to generate discussion, sufficient time and structure need to be included in the program for this. Proposals should explicitly articulate how they will encourage broad discussion. 
  3. Workshop proposals should list explicitly the problems they would like to see solved or at least advances made as part of their workshop. They should explain why these are important problems and how holding their proposed workshop will contribute to their solution.
  4. Workshops are not a venue for work that has been previously published in other conferences on machine learning or related fields. Work that is presented at the main NeurIPS conference should not appear in a workshop, including as part of an invited talk. Organizers should make this clear in their calls and explain in their proposal how they will discourage the presentation of already finalized machine learning work.
  5. We encourage workshop submissions of varying lengths and scopes. Organizers should state whether their workshops are meant to be large-attendance talk format or small group presentations. Organizers should articulate what they hope to achieve from the format of the proposal beyond the talks listed.
  6. Workshops should have a clear and well-communicated agenda or schedule that outlines the topics and speakers to be presented, to provide attendees with the ability to choose which talks or sessions they want to attend based on the content being presented. Good workshops will put talk titles up publicly before site publication and note the archival status of their submissions. Organizers should articulate how they will do this.
  7. Organizing a workshop is a complex task, and proposals should outline the organizational experience and skills of the proposed organizers (as a team). We encourage junior researchers to be involved in workshop organization but prefer some collective experience in organizing a complex event.
 

Frequently Asked Questions From Past Workshops

 
  1. Workshop Series
    Although we ask for statistics and information if the workshop has been held before, we neither encourage nor discourage workshops on topics that have appeared before. Membership in an existing sequence of workshops is irrelevant in the assessment of a workshop proposal (it neither helps nor hinders). Workshop proposals will be evaluated solely on their merits for this year’s conference. That said, proposals from the workshop series are encouraged to differentiate from past versions with some freshness (contents, organizers, etc.) to help the reviewers evaluate the merits.
  2. Overlapping Proposals
    We will not forcibly merge proposals. If multiple strong proposals are submitted on similar topics, we might accept 1-2 workshops in overlapping topics to curate the best workshop program.
 

Common Pitfalls From Past Workshops (see 2019 workshop summary for more discussion)

 
  1. Insufficient time for discussion

Too many invited speakers—some proposals listed a dozen or more—do not make for a great audience experience, and a workshop with nothing but long-form talks is unlikely to lead to new breakthroughs. We encourage organizers to allocate a larger amount of time to contributed talks and posters, and open discussion.

  1. Leaning too heavily on past success
    Proposals for workshops that are part of a series sometimes lean too heavily on the declared popularity of previous workshops. In some cases, this led to proposals that were less creative and innovative than what we had hoped to see.
  2. Unconfirmed or irrelevant speakers

The vast majority of proposals included lists of confirmed invited speakers. This made it hard to champion any workshop that didn’t have at least a few speakers confirmed, especially when many unconfirmed big-name speakers were listed (it’s unlikely all would say yes), or when the diversity statement centered on the assumed presence of unconfirmed speakers. There were also several proposals featuring long lists of “celebrity” speakers without clear relevance to the topic of the workshop.

  1. Going too big

We saw only a few proposals that we felt were too narrow, but many we found too broad. There seems to be a tendency to overreach for the sake of going big, while we’d prefer to see more focused workshops.

  1. Too many organizers or organizing too many workshops

Several proposals had remarkably large organizing committees. It’s not clear why more than five or six organizers would be necessary for a workshop, and it raises concerns about name-dropping or organizers added just for an appearance of diversity. Similarly, organizing too many workshops can be a negative factor towards high-quality workshops.

  1. Diversity lip service

While we were pleased overall by the effort that organizers put into diversity, a lack of diversity in a proposal could be fatal. We were particularly wary of proposals that claimed to be big on diversity while having a full lineup of North American white male speakers or a list of organizers who all recently graduated from the same institution.