Timezone: »
Demonstrations must show novel technology and must run online during the conference. Unlike poster presentations or slide shows, interaction with the audience is a critical element. Therefore, the creativity of demonstrators to propose new ways in which interaction and engagement can fully leverage this year’s virtual conference format will be particularly relevant for selection. This session has the following demonstrations:
- PYLON: A PyTorch Framework for Learning with Constraints
- Real-Time and Accurate Self-Supervised Monocular Depth Estimation on Mobile Device
- Unsupervised Indoor Wi-Fi Positioning
- Prospective Explanations: An Interactive Mechanism for Model Understanding
|
Thu 8:30 a.m. - 8:35 a.m.
|
Intro
(
Talk
)
|
Marco Ciccone 🔗 |
|
Thu 8:35 a.m. - 8:50 a.m.
|
PYLON: A PyTorch Framework for Learning with Constraints
(
Live Demo
)
link »
Deep learning excels at learning task information from large amounts of data, however, struggles with learning from declarative high-level knowledge that can be more succinctly expressed directly. In this work, we introduce PYLON, a neural-symbolic training framework that builds on PyTorch to augment imperatively trained models with declaratively specified knowledge. PYLON lets users programmatically specify constraints as Python functions and compiles them into a differentiable loss, thus training predictive models that fit the data whilst satisfying the specified constraints. PYLON includes both exact as well as approximate compilers to efficiently compute the loss, employing fuzzy logic, sampling methods,and circuits, ensuring scalability even to complex models and constraints. Crucially, a guiding principle in designing PYLON is the ease with which any existing deep learning codebase can be extended to learn from constraints using only a few lines: a function that expresses the constraint and code to incorporate it as a loss. Our demo comprises of models in NLP, computer vision, logical games, and knowledge graphs that can be interactively trained using constraints as supervision. |
Kareem Ahmed · Tao Li · Nu Mai Thy Ton · Quan Guo · Kai-Wei Chang · Parisa Kordjamshidi · Vivek Srikumar · Guy Van den Broeck · Sameer Singh 🔗 |
|
Thu 8:50 a.m. - 9:05 a.m.
|
Real-Time and Accurate Self-Supervised Monocular Depth Estimation on Mobile Device
(
Live Demo
)
link »
This demonstration showcases our novel innovations on self-supervised monocular depth estimation. First, we enhance self-supervised monocular depth estimation with semantic information during training. This reduces the error by 12% and achieves state-of-the-art performance. Second, we enhance the backbone architecture using a scalable method for neural architecture search which optimizes directly for inference latency on a target device. This enables operation at > 30 FPS. We demonstrate these techniques on a smartphone powered by a Snapdragon® Mobile Platform. |
Hong Cai · Yinhao Zhu · Janarbek Matai · Fatih Porikli · Fei Yin · Tushar Singhal · Bharath Ramaswamy · Frank Mayer · Chirag Patel · Parham Noorzad · Andrii Skliar · Tijmen Blankevoort · Joseph Soriaga · Ron Tindall · Pat Lawlor
|
|
Thu 9:05 a.m. - 9:20 a.m.
|
Unsupervised Indoor Wi-Fi Positioning
(
Live Demo
)
link »
Sensing using radio frequency (RF) signals such as Wi-Fi has garnered significant attention in recent years. They can be used, for instance, for so-called passive indoor positioning of humans. This passive positioning uses the Wi-Fi signal as a bi-static radar to determine the location of a human subject who is not carrying any Wi-Fi device. While previous works have demonstrated that positioning is possible, these algorithms rely on precise position labels for training, and only work in confined laboratory environments that must remain invariant. We recently proposed two novel algorithms for passive positioning. The first is based on a self-supervision signal by a combined clustering and triplet loss. The second is modality-agnostic and is based on a low-dimensional manifold learning facilitated by optimal transport. Neither algorithm requires dense labels as required by state of the art algorithms. In this demo, we demonstrate results of these two algorithms in real-world environments, i.e., outside of carefully controlled labs. The presented results demonstrate that our methods surpass state of the art by a wide margin. |
Farhad G. Zanjani · Ilia Karmanov · Hanno Ackermann · Daniel Dijkman · Max Welling · Ishaque Kadampot · Simone Merlin · Steve Shellhammer · Rui Liang · Brian Buesker · Harshit Joshi · Vamsi Vegunta · Raamkumar Balamurthi · Bibhu Mohanty · Joseph Soriaga · Ron Tindall · Pat Lawlor
|
|
Thu 9:20 a.m. - 9:35 a.m.
|
Prospective Explanations: An Interactive Mechanism for Model Understanding
(
Live Demo
)
link »
We demonstrate a system for prospective explanations of black box models for regression and classification tasks with structured data. Prospective explanations are aimed at showing how models work by highlighting likely changes in model outcomes under changes in input. This in contrast to most post-hoc explanability methods, that aim to provide a justification for a decision retrospectively. Our system is designed to provide fast estimates of changes in outcomes for any arbitrary exploratory query from users. Such queries are typical partial, i.e. involve only a selected number of features, the outcomes labels are shown therefore as likelihoods. Repeated queries can therefore indicate which aspects of the feature space are more likely to influence the target variable. Fast interactive exploration is made possible by a surrogate Bayesian network model trained on model labels with some reasonable assumptions on architectures. The main advantages of our approach are that (a) inference is very fast and supports real-time feedback allowing for interactivity, (b) inference can be done with partial information on features, and (c) any indirect effects are also considered in estimating target class distributions. |
Rahul Nair · Pierpaolo Tommasi 🔗 |
Author Information
Douwe Kiela (Facebook AI Research)
Barbara Caputo (Politecnico di Torino)
Marco Ciccone (Politecnico di Torino)
Marco Ciccone is an ELLIS Postdoctoral Researcher in the VANDAL group at Politecnico di Torino and UCL. His current research interests are in the intersection of meta, continual, and federated learning with a particular focus on modularity and models re-use to scale the training of agents with heterogeneous data and mitigate the effect of catastrophic forgetting and interference across tasks, domains, and devices. He has been NeurIPS Competiton Track co-chair in 2021, 2022 and 2023.
More from the Same Authors
-
2021 : Public Information Representation for Adversarial Team Games »
Luca Carminati · Federico Cacciamani · Marco Ciccone · Nicola Gatti -
2022 : Perturbation Augmentation for Fairer NLP »
Rebecca Qian · Candace Ross · Jude Fernandes · Eric Michael Smith · Douwe Kiela · Adina Williams -
2023 Poster: OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents »
Hugo Laurençon · Lucile Saulnier · Leo Tronchon · Stas Bekman · Amanpreet Singh · Anton Lozhkov · Thomas Wang · Siddharth Karamcheti · Alexander Rush · Douwe Kiela · Matthieu Cord · Victor Sanh -
2023 Poster: DataPerf: Benchmarks for Data-Centric AI Development »
Mark Mazumder · Colby Banbury · Xiaozhe Yao · Bojan Karlaš · William Gaviria Rojas · Sudnya Diamos · Greg Diamos · Lynn He · Alicia Parrish · Hannah Rose Kirk · Jessica Quaye · Charvi Rastogi · Douwe Kiela · David Jurado · David Kanter · Rafael Mosquera · Will Cukierski · Juan Ciro · Lora Aroyo · Bilge Acun · Lingjiao Chen · Mehul Raje · Max Bartolo · Evan Sabri Eyuboglu · Amirata Ghorbani · Emmett Goodman · Addison Howard · Oana Inel · Tariq Kane · Christine R. Kirkpatrick · D. Sculley · Tzu-Sheng Kuo · Jonas Mueller · Tristan Thrush · Joaquin Vanschoren · Margaret Warren · Adina Williams · Serena Yeung · Newsha Ardalani · Praveen Paritosh · Ce Zhang · James Zou · Carole-Jean Wu · Cody Coleman · Andrew Ng · Peter Mattson · Vijay Janapa Reddi -
2022 Workshop: Human Evaluation of Generative Models »
Divyansh Kaushik · Jennifer Hsia · Jessica Huynh · Yonadav Shavit · Samuel Bowman · Ting-Hao Huang · Douwe Kiela · Zachary Lipton · Eric Michael Smith -
2022 Competition: NeurIPS 2022 Competition Track: Overview & Results »
Marco Ciccone · Gustavo Stolovitzky · Jake Albrecht -
2021 : Spotlight Talk: Public Information Representation for Adversarial Team Games »
Luca Carminati · Federico Cacciamani · Marco Ciccone · Nicola Gatti -
2021 : Facebook - Data Centric Infrastructure »
Douwe Kiela -
2021 Demonstration: Demonstrations 4 »
Douwe Kiela · Barbara Caputo · Marco Ciccone -
2021 : Intro »
Marco Ciccone -
2021 : Introduction to Competition Day 4 »
Marco Ciccone -
2021 Competition: Competition Track Day 4: Overviews + Breakout Sessions »
Douwe Kiela · Marco Ciccone · Barbara Caputo -
2021 Poster: True Few-Shot Learning with Language Models »
Ethan Perez · Douwe Kiela · Kyunghyun Cho -
2021 : Invited talk - Douwe Kiela »
Douwe Kiela -
2021 : Introduction to Competition Day 3 »
Marco Ciccone -
2021 Competition: Competition Track Day 3: Overviews + Breakout Sessions »
Douwe Kiela · Marco Ciccone · Barbara Caputo -
2021 : Intro »
Marco Ciccone -
2021 Poster: Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking »
Zhiyi Ma · Kawin Ethayarajh · Tristan Thrush · Somya Jain · Ledell Wu · Robin Jia · Christopher Potts · Adina Williams · Douwe Kiela -
2021 Demonstration: Demonstrations 2 »
Douwe Kiela · Barbara Caputo · Marco Ciccone -
2021 : Intro »
Douwe Kiela -
2021 : Introduction to Competition Day 2 »
Barbara Caputo -
2021 Competition: Competition Track Day 2: Overviews + Breakout Sessions »
Douwe Kiela · Marco Ciccone · Barbara Caputo -
2021 Competition: Competition Track Day 1: Overviews + Breakout Sessions »
Douwe Kiela · Marco Ciccone · Barbara Caputo -
2021 : Introduction Competion Day 1 »
Douwe Kiela -
2021 Poster: Human-Adversarial Visual Question Answering »
Sasha Sheng · Amanpreet Singh · Vedanuj Goswami · Jose Magana · Tristan Thrush · Wojciech Galuba · Devi Parikh · Douwe Kiela -
2021 Demonstration: Demonstrations 1 »
Douwe Kiela · Barbara Caputo · Marco Ciccone -
2021 : Introduction »
Douwe Kiela -
2020 : Q & A and Panel Session with Dan Weld, Kristen Grauman, Scott Yih, Emma Brunskill, and Alex Ratner »
Kristen Grauman · Wen-tau Yih · Alexander Ratner · Emma Brunskill · Douwe Kiela · Daniel S. Weld -
2020 Workshop: HAMLETS: Human And Model in the Loop Evaluation and Training Strategies »
Divyansh Kaushik · Bhargavi Paranjape · Forough Arabshahi · Yanai Elazar · Yixin Nie · Max Bartolo · Polina Kirichenko · Pontus Lars Erik Saito Stenetorp · Mohit Bansal · Zachary Lipton · Douwe Kiela -
2020 : Opening Remarks »
Divyansh Kaushik · Bhargavi Paranjape · Douwe Kiela -
2020 : The Hateful Memes Challenge: Live award ceremony and winner presentations »
Douwe Kiela -
2020 : The Hateful Memes Challenge: Competition Overview »
Douwe Kiela -
2020 Poster: The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes »
Douwe Kiela · Hamed Firooz · Aravind Mohan · Vedanuj Goswami · Amanpreet Singh · Pratik Ringshia · Davide Testuggine -
2020 Poster: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks »
Patrick Lewis · Ethan Perez · Aleksandra Piktus · Fabio Petroni · Vladimir Karpukhin · Naman Goyal · Heinrich Küttler · Mike Lewis · Wen-tau Yih · Tim Rocktäschel · Sebastian Riedel · Douwe Kiela -
2020 Poster: Learning Optimal Representations with the Decodable Information Bottleneck »
Yann Dubois · Douwe Kiela · David Schwab · Ramakrishna Vedantam -
2020 Spotlight: Learning Optimal Representations with the Decodable Information Bottleneck »
Yann Dubois · Douwe Kiela · David Schwab · Ramakrishna Vedantam -
2019 : Audrey Durand, Douwe Kiela, Kamalika Chaudhuri moderated by Yann Dauphin »
Audrey Durand · Kamalika Chaudhuri · Yann Dauphin · Orhan Firat · Dilan Gorur · Douwe Kiela -
2019 : Douwe Kiela - Benchmarking Progress in AI: A New Benchmark for Natural Language Understanding »
Douwe Kiela -
2019 Workshop: Emergent Communication: Towards Natural Language »
Abhinav Gupta · Michael Noukhovitch · Cinjon Resnick · Natasha Jaques · Angelos Filos · Marie Ossenkopf · Angeliki Lazaridou · Jakob Foerster · Ryan Lowe · Douwe Kiela · Kyunghyun Cho -
2019 Poster: Hyperbolic Graph Neural Networks »
Qi Liu · Maximilian Nickel · Douwe Kiela -
2018 Workshop: Emergent Communication Workshop »
Jakob Foerster · Angeliki Lazaridou · Ryan Lowe · Igor Mordatch · Douwe Kiela · Kyunghyun Cho -
2018 : Panel Discussion »
Antonio Torralba · Douwe Kiela · Barbara Landau · Angeliki Lazaridou · Joyce Chai · Christopher Manning · Stevan Harnad · Roozbeh Mottaghi -
2018 : Douwe Kiela - Learning Multimodal Embeddings »
Douwe Kiela -
2018 Poster: NAIS-Net: Stable Deep Networks from Non-Autonomous Differential Equations »
Marco Ciccone · Marco Gallieri · Jonathan Masci · Christian Osendorfer · Faustino Gomez -
2017 Workshop: Emergent Communication Workshop »
Jakob Foerster · Igor Mordatch · Angeliki Lazaridou · Kyunghyun Cho · Douwe Kiela · Pieter Abbeel -
2017 Poster: Poincaré Embeddings for Learning Hierarchical Representations »
Maximilian Nickel · Douwe Kiela -
2017 Spotlight: Poincaré Embeddings for Learning Hierarchical Representations »
Maximilian Nickel · Douwe Kiela -
2009 Workshop: Learning from Multiple Sources with Applications to Robotics »
Barbara Caputo · Nicolò Cesa-Bianchi · David R Hardoon · Gayle Leen · Francesco Orabona · Jaakko Peltonen · Simon Rogers -
2009 Poster: Who’s Doing What: Joint Modeling of Names and Verbs for Simultaneous Face and Pose Annotation »
Jie Luo · Barbara Caputo · Vittorio Ferrari