Dataset and Benchmark Poster Session 2
The Datasets and Benchmarks track serves as a novel venue for high-quality publications, talks, and posters on highly valuable machine learning datasets and benchmarks, as well as a forum for discussions on how to improve dataset development. Datasets and benchmarks are crucial for the development of machine learning methods, but also require their own publishing and reviewing guidelines. For instance, datasets can often not be reviewed in a double-blind fashion, and hence full anonymization will not be required. On the other hand, they do require additional specific checks, such as a proper description of how the data was collected, whether they show intrinsic bias, and whether they will remain accessible.
Competition Track Day 2: Overviews + Breakout Sessions
The program includes a wide variety of exciting competitions in different domains, with some focusing more on applications and others trying to unify fields, focusing on technical challenges or directly tackling important problems in the world. The aim is for the broad program to make it so that anyone who wants to work on or learn from a competition can find something to their liking.
In this session, we have the following competitions:
* Learning By Doing: Controlling a Dynamical System using Control Theory, Reinforcement Learning, or Causality
* Reconnaissance Blind Chess
* Real Robot Challenge II
* The Billion-Scale Approximate Nearest Neighbor Search Challenge
* MetaDL: Few Shot Learning Competition with Novel Datasets from Practical Domains
Shine in Your Technical Presentation
Our presentations are most likely the highest impact activities we have as researchers. They are often quite dense. In those 10 minutes in your conference oral, you have the chance to show your work to a large audience for world-wide recognition. This is both incredibly stressful and difficult to do. The months of research that you've done, with all the ideas and all the results, have to be jam packed in a short time interval, and your audience is tired of the long conference and the information hoses they are drinking from. How do you make the most out of your presentation? How do you make sure that people understand your work, get excited by it, and remember you in the future?
In the first part of the session we will cover:
- How to structure your presentation and storytelling
- Captivating your audience and making them remember you
- Guiding your audience through tough and difficult to parse material
- Dynamic and easy to follow slide creation
- Preparation for the big moment
- Frequent mistakes and the psychology of insecurity.
In the second part you are invited to bring your own presentations, which will be discussed in a small group. Don’t worry about getting it right, we’re all here to learn. Your teacher for the session will be Tijmen Blankevoort, Senior Staff Manager of Engineering at Qualcomm Technologies Netherlands. A public speaker with over 7 years of experience, recurring radio and podcast guest, former founder of a successful AI start-up, and research team-lead in Qualcomm working on model efficiency.
The Role of Benchmarks in the Scientific Progress of Machine Learning
Benchmark datasets have played a crucial role in driving empirical progress in machine learning, leading to an interesting dynamic between those on a quest for state-of-the-art performance and those creating new challenging benchmarks. In this panel, we reflect on how benchmarks can lead to scientific progress, both in terms of new algorithmic innovations and improved scientific understanding. First, what qualities of a machine learning system should a good benchmark dataset seek to measure? How well can benchmarks assess performance in dynamic and novel environments, or in tasks with an open-ended set of acceptable answers? Benchmarks can also raise significant ethical concerns including poor data collection practices, under- and misrepresentation of subjects, as well as misspecification of objectives. Second, even given high-quality, carefully constructed benchmarks, which research questions can we hope to answer from leaderboard-climbing, and which ones are deprioritized or impossible to answer due to the limitations of the benchmark paradigm? In general, we hope to deepen the community’s awareness of the important role of benchmarks for advancing the science of machine learning.
Dataset and Benchmark Track 2
The Datasets and Benchmarks track serves as a novel venue for high-quality publications, talks, and posters on highly valuable machine learning datasets and benchmarks, as well as a forum for discussions on how to improve dataset development. Datasets and benchmarks are crucial for the development of machine learning methods, but also require their own publishing and reviewing guidelines. For instance, datasets can often not be reviewed in a double-blind fashion, and hence full anonymization will not be required. On the other hand, they do require additional specific checks, such as a proper description of how the data was collected, whether they show intrinsic bias, and whether they will remain accessible.
Demonstrations 2
Demonstrations must show novel technology and must run online during the conference. Unlike poster presentations or slide shows, interaction with the audience is a critical element. Therefore, the creativity of demonstrators to propose new ways in which interaction and engagement can fully leverage this year’s virtual conference format will be particularly relevant for selection. This session has the following demonstrations:
- Automated Evaluation of GNN Explanations with Neuro Symbolic Reasoning
- AIMEE: Interactive model maintenance with rule-based surrogates
- AME: Interpretable Almost Exact Matching for Causal Inference
- Exploring Conceptual Soundness with TruLens
- An Interactive Tool for Computation with Assemblies of Neurons
The Town Hall meeting is an opportunity to connect and to learn about how this year's conference was organized. It is also a place where you can ask questions and provide feedback to the NeurIPS organizers and Board. We encourage you to submit questions in advance via townhall@neurips.cc. This event is open to all registered attendees of NeurIPS 2021.
Lapsed Physicists Wine-and-Cheese
Lapsed" (aka. Former) Physicists are plentiful in the machine learning community. Inspired by Wine and Cheese seminars at many institutions, this BYOWC (Bring Your Own Wine and Cheese) event is an informal opportunity to connect with members of the community.
Hear how others made the transition between fields. Discuss how your physics training prepared you to switch fields or what synergies between physics and machine learning excite you the most. Share your favorite physics jokes your computer science colleagues don't get, and just meet other cool people. Open to everyone, not only physicists; you'll just have to tolerate our humor. Wine and Cheese encouraged, but not required.
Space & ML
METI ‘MEET’ WITH FDL
FDL will be hosting an official NeurIPS social: METI ’Meet’ (Messaging Extraterrestrial Intelligence) with Google Cloud, Intel and the SETI Institute. Learn about METI and make and decode messages from competing ‘alien’ planets and (of course) win cool Intel prizes.
“Where is everybody?” This official NeurIPS social will be a chance to connect with other folks in the rarified space / ML / alien-hunting Venn diagram. You’ll be working together with other aliens to communicate key variables about your home planet and then transmit them into the cosmos (um, via Zoom) for other planetary beings to decrypt! The team that decrypts the most alien messages, wins - and the team that is the most decrypted also wins (assuming no galactic domination intent.) Meet us at NeurIPS on Dec 8th at 10 am PT / 1pm ET / 6pm GMT
This will be an excellent opportunity to convene a community around ML application for space exploration, planetary stewardship of Earth and opportunities for ML in science.
We hope you are able to join us for what will be an inspirational and enjoyable event.
BigScience
BigScience is a one-year long research workshop on large multilingual datasets and large language models. It fosters discussions and reflections around the research questions surrounding large language models (capabilities, limitations, potential improvements, bias, ethics, environmental impact) as well as the challenges around creating and sharing such models and datasets for research purposes and among the research community. For the NeurIPS 2021 BigSience Social we are looking forward to:
Technical talks and paper presentations by BigScience working group chairs Invited talks & interdisciplinary discussions on large language models A carbon footprint tutorial A social space for open discussions
Queer in AI Workshop 2
Queer in AI’s demographic survey reveals that most queer scientists in our community do not feel completely welcome in conferences and their work environments, with the main reasons being a lack of queer community and role models. Over the past years, Queer in AI has worked towards these goals, yet we have observed that the voices of marginalized queer communities - especially transgender, non-binary folks and queer BIPOC folks - have been neglected. The purpose of this workshop is to highlight issues that these communities face by featuring talks and panel discussions on the inclusion of neuro-diverse people in our communities, the intersection of queer and animal rights, as well as worker rights issues around the world.
The main topics of the workshop will revolve around:
- the intersection of AI, queer identity and neurodiversity
- queer identity and labor rights and organization
- AI and animal rights
- queer identity and caste-based discrimination
Additionally, at Queer in AI’s socials at NeurIPS 2021, we will focus on creating a safe and inclusive casual networking and socializing space for LGBTQIA+ individuals involved with AI. There will also be additional social events, stay tuned for more details coming soon. Together, these components will create a community space where attendees can learn and grow from connecting with each other, bonding over shared experiences, and learning from each individual’s unique insights into AI, queerness, and beyond!
Indigenous in AI Workshop
With increasing interest in working with Indigenous Communities on data-related projects, Indigenous in AI brings a discussion around data ethics and Indigenous data sovereignty. Hear from Indigenous AI practitioners as they share their ideas and work towards a more equitable data future for Indigenous peoples.
Interdisciplinary ML: Bridging Gaps and Building Graphs
Given the many disciplines that encompass ML, it is important that we as researchers better understand academics with differing backgrounds than our own to produce valued contributions.
In this 2-hour social, we pair participants together based on differing levels of experience in related disciplines of ML. These pairings would be determined by a confidential online form. For example, suppose Researcher A identifies as being highly experienced in Reinforcement Learning but has little to no experience in Neuroscience. Researcher A could then be paired with Researcher B, who has a great background in Neuroscience but has had no exposure to Reinforcement Learning. This cycle could repeat every 10 minutes so that every participant can meet a diverse set of researchers across a multitude of disciplines.
To help guide the discussions in these breakout sessions, they will be provided a script of suggested, generic questions to help bridge the gaps of understanding between their fields of expertise. These questions will cover more formal topics such as finding similarity between their fields of study. Some suggested questions may encompass less formal topics such as which leisure activities are commonly pursued by groups of specific disciplines. Participants are encouraged to document these similarities from session to session.
The final activity will include round-table discussions of researchers who have similar backgrounds. These discussions will allow researchers to share what they found to be similar in groups different than their own. This round-table group will be tasked to produce a graph that connects various aspects from field to field. These graphs can then be shared via social media or other platforms for the rest of the conference to view.
These interactions could allow us to form better connections across disciplines and build a better internal knowledge graph of the vast landscape of ML.
Benign Overfitting
Deep learning has revealed some major surprises from the perspective of statistical complexity: even without any explicit effort to control model complexity, these methods find prediction rules that give a near-perfect fit to noisy training data and yet exhibit excellent prediction performance in practice. This talk surveys work on methods that predict accurately in probabilistic settings despite fitting too well to training data. We present a characterization of linear regression problems for which the minimum norm interpolating prediction rule has near-optimal prediction accuracy. The characterization shows that overparameterization is essential for benign overfitting in this setting: the number of directions in parameter space that are unimportant for prediction must significantly exceed the sample size. We discuss implications for robustness to adversarial examples, and we describe extensions to ridge regression and barriers to analyzing benign overfitting via model-dependent generalization bounds.
WiML Workshop 1
WiML’s purpose is to enhance the experience of women in machine learning. Our flagship event is the annual Women in Machine Learning (WiML) Workshop, typically co-located with NeurIPS. We also organize an “un-workshop” at ICML, as well as small events at other machine learning conferences such as AISTATS, ICLR, etc.
Our mission is to enhance the experience of women in machine learning, and thereby
Increase the number of women in machine learning
Help women in machine learning succeed professionally
Increase the impact of women in machine learning in the community
Toward this goal, we create opportunities for women to engage in substantive technical and professional conversations in a positive, supportive environment (e.g. annual workshop, small events, mentoring program). We also work to increase awareness and appreciation of the achievements of women in machine learning (e.g. directory and profiles of women in machine learning). Our programs help women build their technical confidence and their voice, and our publicity efforts help ensure that women in machine learning and their achievements are known in the community.
ML in India: A Billion Opportunities
Machine learning and data science activity is seeing a massive growth in India with many universities initiating related academic programmes, many research labs setting their base in India, and many ML and AI based start-ups rapidly expanding in India. This social aims to make NeurIPS attendees aware of career opportunities at these places by facilitating interaction between those interested in exploring career opportunities in India (e.g., graduating students and postdocs) and researchers who are currently based in India, and also create more awareness around ML problems that can have significant impact in the Indian context.
Optimal Transport: Past, Present, and Future
At the end of the 18th century, Gaspard Monge introduced the optimal transport problem to understand the most efficient way of transporting a distribution of material from one place to another to build fortifications. In the last 30 years, this theory has found various applications in many areas of mathematics. However, more recently, optimal transport has also become a very powerful tool in many areas of machine learning. In this talk, we will give an overview of optimal transport, with some selected applications.
