NeurIPS 2021

Timezone: »

Competition

Competition Track Day 3: Overviews + Breakout Sessions

Douwe Kiela · Marco Ciccone · Barbara Caputo

Thu Dec 09 10:00 AM -- 03:04 PM (PST) @

Event URL: https://neurips.cc/Conferences/2021/CompetitionTrack »

The program includes a wide variety of exciting competitions in different domains, with some focusing more on applications and others trying to unify fields, focusing on technical challenges or directly tackling important problems in the world. The aim is for the broad program to make it so that anyone who wants to work on or learn from a competition can find something to their liking.

In this session, we have the following competitions:
* Evaluating Approximate Inference in Bayesian Deep Learning
* The NetHack Challenge
* Machine Learning for Combinatorial Optimization
* Traffic4cast 2021 - Temporal and Spatial Few-Shot Transfer Learning in Traffic Map Movie Forecasting
* BASALT: A MineRL Competition on Solving Human-Judged Tasks
* IGLU: Interactive Grounded Language Understanding in a Collaborative Environment

Thu 10:00 a.m. - 10:05 a.m.	Introduction to Competition Day 3 ( Intro ) SlidesLive Video »	Marco Ciccone 🔗
Thu 10:05 a.m. - 10:25 a.m.	Evaluating Approximate Inference in Bayesian Deep Learning + Q&A ( Talk ) link » SlidesLive Video » Understanding the fidelity of approximate inference has extraordinary value beyond the standard approach of measuring generalization on a particular task: if approximate inference is working correctly, then we can expect more reliable and accurate deployment across any number of real-world settings. In this regular competition, we invite the community to evaluate the fidelity of approximate Bayesian inference procedures in deep learning, using as a reference Hamiltonian Monte Carlo (HMC) samples obtained by parallelizing computations over hundreds of tensor processing unit (TPU) devices. We consider a variety of tasks, including image recognition, regression, covariate shift, and medical applications, such as diagnosing diabetic retinopathy. All data are publicly available, and we will release several baselines, including stochastic MCMC, variational methods, and deep ensembles. Link »	Andrew Gordon Wilson · Pavel Izmailov · Matthew Hoffman · Yarin Gal · Yingzhen Li · Melanie F. Pradier · Sharad Vikram · Andrew Foong · Sanae Lotfi · Sebastian Farquhar 🔗
Thu 10:24 a.m. - 1:24 p.m.	Breakout: Evaluating Approximate Inference in Bayesian Deep Learning ( Breakout session )	🔗
Thu 10:25 a.m. - 10:45 a.m.	The NetHack Challenge + Q&A ( Talk ) link » SlidesLive Video » The NetHack Challenge is based on the NetHack Learning Environment (NLE), where teams will compete to build the best agents to play the game of NetHack. NetHack is a ASCII-rendered single-player dungeon crawl game that is one of the oldest and most difficult computer games in history. NetHack is procedurally-generated, with hundreds of different entities and complex environment dynamics, presenting an extremely challenging environment for both current state-of-the-art RL agents and humans, while crucially being lightning-fast to simulate. We are excited that this competition offers machine learning students, researchers and NetHack-bot builders the opportunity to participate in a grand challenge in AI without prohibitive computational costs—and we are eagerly looking forward to the wide variety of submissions. Link »	Eric Hambro · Sharada Mohanty · Dipam Chakrabroty · Edward Grefenstette · Minqi Jiang · Robert Kirk · Vitaly Kurin · Heinrich Kuttler · Vegard Mella · Nantas Nardelli · Jack Parker-Holder · Roberta Raileanu · Tim Rocktäschel · Danielle Rothermel · Mikayel Samvelyan 🔗
Thu 10:44 a.m. - 1:44 p.m.	Breakout: The NetHack Challenge ( Breakout session )	🔗
Thu 10:45 a.m. - 11:05 a.m.	Machine Learning for Combinatorial Optimization + Q&A ( Talk ) link » SlidesLive Video » The Machine Learning for Combinatorial Optimization (ML4CO) competition aims at improving a state-of-the-art mathematical solver by replacing key heuristic components with machine learning models trained on historical data. To that end participants will compete on the three following challenges, each corresponding to a distinct control task arising in a branch-and-bound solver: producing good solutions (primal task), proving optimality via branching (dual task), and choosing the best solver parameters (configuration task). Each task is exposed through an OpenAI-gym Python API build on top of the open-source solver SCIP, using the Ecole library. Participants can compete in any subset of the proposed challenges. While we encourage solutions derived from the reinforcement learning paradigm, any algorithmic solution respecting the competition's API is accepted. Link »	Maxime Gasse · Simon Bowly · Chris Cameron · Quentin Cappart · Jonas Charfreitag · Laurent Charlin · Shipra Agrawal · Didier Chetelat · Justin Dumouchelle · Ambros Gleixner · Aleksandr Kazachkov · Elias Khalil · Pawel Lichocki · Andrea Lodi · Miles Lubin · Christopher Morris · Dimitri Papageorgiou · Augustin Parjadis · Sebastian Pokutta · Antoine Prouvost · Yuandong Tian · Lara Scavuzzo · Giulia Zarpellon 🔗
Thu 11:04 a.m. - 2:04 p.m.	Breakout: Machine Learning for Combinatorial Optimization ( Breakout session ) Schedule (GMT Timezone) 19:05 GMT - competition wrap-up (15 min) 19:20 GMT - presentation from the winners (3x10 min) 19:50 GMT - poster session on Gathertown (30 min) 20:20 GMT - break (15 min) 20:35 GMT - keynote (30 min) - Kate Smith-Miles 21:05 GMT - panel discussion (1 hour) - Andrea Lodi, Timo Berthold, Frank Hutter, Vinod Nair, Kevin Leyton-Brown	🔗
Thu 11:05 a.m. - 11:25 a.m.	Traffic4cast 2021 – Temporal and Spatial Few-Shot Transfer Learning in Traffic Map Movie Forecasting + Q&A ( Talk ) link » SlidesLive Video » Traffic is said to follow `hidden rules' that can be transferred across domain shifts. Our competition sets out to explore this meta topic with two few-shot learning tasks: predictions across a temporal shift brought about by COVID-19 and across a spatio-temporal shift in hitherto unseen cities. We provide an unprecedented, large data set from $10^{12}$ real world GPS probes in $10$ cities binned in space and time into multi-channel movie frames, as well as static data on the basic road connections of the underlying road network. Thus participants can approach these transfer tasks using graph based approaches encoding knowledge about the road network or approaches from computer vision like U-nets, which were highly successful in our previous competitions. Any advance in these questions will have a large impact on smart city planning, on mobility in general and thus, ultimately, our way of living more sustainably. Link »	Moritz Neun · Christian Eichenberger · Henry Martin · Pedro Herruzo · David Jonietz · Fei Tang · Daniel Springer · Markus Spanring · Avi Avidan · Luis Ferro · Ali Soleymani · Rohit Gupta · Bo Xu · Kevin Malm · Aleksandra Gruca · Johannes Brandstetter · Michael Kopp · David Kreil · Sepp Hochreiter 🔗
Thu 11:24 a.m. - 2:24 p.m.	Breakout: Traffic4cast 2021 – Temporal and Spatial Few-Shot Transfer Learning in Traffic Map Movie Forecasting ( Breakout session )	🔗
Thu 11:25 a.m. - 11:45 a.m.	BASALT: A MineRL Competition on Solving Human-Judged Task + Q&A ( Talk ) link » SlidesLive Video » The Benchmark for Agents that Solve Almost-Lifelike Tasks (BASALT) competition aims to promote research in the area of learning from human feedback in order to enable agents that can pursue tasks that do not have crisp, easily defined reward functions. We provide tasks consisting of a simple English language description alongside a Gym environment, without any associated reward function, but with expert demos. Participants will train agents for these tasks using their preferred methods. We expect typical solutions will use imitation learning, or learning from comparisons. Submitted agents will be evaluated based on how well they complete the tasks, as judged by humans given the same description of the tasks. Link »	Rohin Shah · Cody Wild · Steven Wang · Neel Alex · Brandon Houghton · William Guss · Sharada Mohanty · Stephanie Milani · Nicholay Topin · Pieter Abbeel · Stuart Russell · Anca Dragan 🔗
Thu 11:44 a.m. - 2:44 p.m.	Breakout: BASALT: A MineRL Competition on Solving Human-Judged Tasks ( Breakout session )	🔗
Thu 11:45 a.m. - 12:05 p.m.	IGLU: Interactive Grounded Language Understanding in a Collaborative Environment + Q&A ( Talk ) link » SlidesLive Video » Human intelligence has the remarkable ability to quickly adapt to new tasks and environments. Starting from a very young age, humans acquire new skills and learn how to solve new tasks either by imitating the behavior of others or by following provided natural language instructions. To facilitate research in this direction, we propose IGLU: Interactive Grounded Language Understanding in a Collaborative Environment. The primary goal of the competition is to approach the problem of how to build interactive agents that learn to solve a task while provided with grounded natural language instructions in a collaborative environment. Understanding the complexity of the challenge, we split it into sub-tasks to make it feasible for participants. Link »	Julia Kiseleva · Ziming Li · Mohammad Aliannejadi · Maartje Anne ter Hoeve · Mikhail Burtsev · Alexey Skrynnik · Artem Zholus · Aleksandr Panov · Katja Hofmann · Kavya Srinet · arthur szlam · Michel Galley · Ahmed Awadallah 🔗
Thu 12:04 p.m. - 3:04 p.m.	Breakout: IGLU: Interactive Grounded Language Understanding in a Collaborative Environment ( Breakout session )	🔗

Author Information

Douwe Kiela (Facebook AI Research)

Marco Ciccone (Politecnico di Torino)

Barbara Caputo (Politecnico di Torino)

More from the Same Authors