Timezone: »
Demonstrations must show novel technology and must run online during the conference. Unlike poster presentations or slide shows, interaction with the audience is a critical element. Therefore, the creativity of demonstrators to propose new ways in which interaction and engagement can fully leverage this year’s virtual conference format will be particularly relevant for selection. This session has the following demonstrations:
- Automated Evaluation of GNN Explanations with Neuro Symbolic Reasoning
- AIMEE: Interactive model maintenance with rule-based surrogates
- AME: Interpretable Almost Exact Matching for Causal Inference
- Exploring Conceptual Soundness with TruLens
- An Interactive Tool for Computation with Assemblies of Neurons
|
Wed 8:30 a.m. - 8:35 a.m.
|
Intro
(
Talk
)
|
Douwe Kiela 🔗 |
|
Wed 8:35 a.m. - 8:50 a.m.
|
Automated Evaluation of GNN Explanations with Neuro Symbolic Reasoning
(
Live Demo
)
link »
Automatically evaluating explanations of GNN model predictions can significantly increase the adoption of GNN models and explainability solutions in real world applications. In this demo, we present a method to use GNN predictions and symbolic reasoners, to automatically evaluate the explanations from GNN explainability techniques. |
Vanya BK · Muhammed Ameen · Balaji Ganesan · Devbrat Sharma · Arvind Agarwal 🔗 |
|
Wed 8:50 a.m. - 9:05 a.m.
|
AIMEE: Interactive model maintenance with rule-based surrogates
(
Live Demo
)
link »
In real-world applications, such as loan approvals or claims management, machine learning (ML) models need to be updated or retrained to adhere to new rules and regulations. But how can a new model be built and new decision boundaries be formed without having new training data available? We present AI Model Explorer and Editor tool (AIMEE) for model exploration and model editing using human understandable rules. It addresses the problem of changing decision boundaries by leveraging user-specified feedback rules that are used to pre-process training data such that a retrained model will reflect user changes. The pre-processing step using synthetic oversampling and relabeling and assumes white box access to the model. AIMEE provides interactive methods to edit rule sets, visualize changes to decision boundaries, and generates interpretable comparisons of model changes so that users see their feedback reflected in the updated model. The demo shows and end-to-end solution that supports the full update lifecycle of an ML model. |
Owen Cornec · Rahul Nair · Oznur Alkan · Dennis Wei · Elizabeth Daly 🔗 |
|
Wed 9:05 a.m. - 9:20 a.m.
|
AME: Interpretable Almost Exact Matching for Causal Inference
(
Live Demo
)
link »
AME (Almost Matching Exactly) is an interactive web-based application that allows users to perform matching for observational causal inference on large datasets. The AME application is powered by Fast Large-Scale Almost Matching Exactly (FLAME) (JMLR’21) and Dynamic Almost Matching Exactly (DAME) (AISTATS’19) algorithms that match treatment and control units in a way that is interpretable, because the matches are made directly on covariates, high-quality, because machine learning is used to determine which covariates are important to match on, and scalable, using techniques from data management. Our demonstration shows the usefulness of these algorithms and allows easy interactive explorations for treatment effect estimates and corresponding matched groups of units with a suite of visualizations providing detailed insights to users. |
Haoning Jiang · Thomas Howell · Neha Gupta · Vittorio Orlandi · Sudeepa Roy · Marco Morucci · Harsh Parikh · Alexander Volfovsky · Cynthia Rudin 🔗 |
|
Wed 9:20 a.m. - 9:35 a.m.
|
Exploring Conceptual Soundness with TruLens
(
Live Demo
)
link »
As machine learning has become increasingly ubiquitous, there has been a growing need to assess the trustworthiness of learned models. One important aspect to model trust is conceptual soundness, i.e., the extent to which a model uses features that are appropriate for its intended task. We present TruLens, a new cross-platform framework for explaining deep network behavior. In our demonstration, we provide an interactive application built on TruLens that we use to explore the conceptual soundness of various pre-trained models. Throughout the presentation, we take the unique perspective that robustness to small-norm adversarial examples is a necessary condition for conceptual soundness; we demonstrate this by comparing explanations on models trained with and without a robust objective. Our demonstration will focus on our end-to-end application, which will be made accessible for the audience to interact with; but we will also provide details on its open-source components, including the TruLens library and the code used to train robust networks. |
Anupam Datta · Matt Fredrikson · Klas Leino · Kaiji Lu · Shayak Sen · Ricardo C Shih · Zifan Wang 🔗 |
|
Wed 9:35 a.m. - 9:50 a.m.
|
An Interactive Tool for Computation with Assemblies of Neurons
(
Live Demo
)
link »
While Artificial Neural Networks have reached once unimaginable levels of performance, it is also broadly recognized that they lag far behind biological brains in important aspects (besides biological plausibility), such as interpretability and adaptability to new tasks as well as power and data usage. During the past two decades, the study of the animal brain has also progressed tremendously through powerful recording techniques and interpretation methodology --- in recent years assisted greatly by machine learning. However, this progress has not brought us closer to answering this field's own overarching interpretability question: how exactly does the activity of neurons and synapses result in high-level cognitive functions, especially in the human brain? The Assembly Calculus (AC) is a novel framework intended to bridge the gap between the level of neuron and synapses, and that of cognition. AC is a computational system entailing a basic data item called an assembly, a stable set of neurons explained below; a set of operations that create and manipulate assemblies; and an execution model squarely based on basic tenets of neuroscience. Importantly, it allows the creation of biologically plausible, flexible and interpretable programs, enabling one to develop tangible hypotheses on how specific brain functions may work. To facilitate such experimentation, we present here a tool which in real-time allows the simulation, modification and visualization of this computational system, including several prepared examples. Our tool is a web application which greatly aids in the creation and analysis of algorithms within the assembly calculus by allowing the user to both visualize neurons and their connections and also present a simple interface to dynamically modify and run code on assemblies. The interface can be accessed here: http://brain.cc.gatech.edu |
Seung Je Jung · Christos Papadimitriou · Santosh Vempala 🔗 |
Author Information
Douwe Kiela (Facebook AI Research)
Barbara Caputo (Politecnico di Torino)
Marco Ciccone (Politecnico di Torino)
Marco Ciccone is an ELLIS Postdoctoral Researcher in the VANDAL group at Politecnico di Torino and UCL. His current research interests are in the intersection of meta, continual, and federated learning with a particular focus on modularity and models re-use to scale the training of agents with heterogeneous data and mitigate the effect of catastrophic forgetting and interference across tasks, domains, and devices. He has been NeurIPS Competiton Track co-chair in 2021, 2022 and 2023.
More from the Same Authors
-
2021 : Public Information Representation for Adversarial Team Games »
Luca Carminati · Federico Cacciamani · Marco Ciccone · Nicola Gatti -
2022 : Perturbation Augmentation for Fairer NLP »
Rebecca Qian · Candace Ross · Jude Fernandes · Eric Michael Smith · Douwe Kiela · Adina Williams -
2023 Poster: OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents »
Hugo Laurençon · Lucile Saulnier · Leo Tronchon · Stas Bekman · Amanpreet Singh · Anton Lozhkov · Thomas Wang · Siddharth Karamcheti · Alexander Rush · Douwe Kiela · Matthieu Cord · Victor Sanh -
2023 Poster: DataPerf: Benchmarks for Data-Centric AI Development »
Mark Mazumder · Colby Banbury · Xiaozhe Yao · Bojan Karlaš · William Gaviria Rojas · Sudnya Diamos · Greg Diamos · Lynn He · Alicia Parrish · Hannah Rose Kirk · Jessica Quaye · Charvi Rastogi · Douwe Kiela · David Jurado · David Kanter · Rafael Mosquera · Will Cukierski · Juan Ciro · Lora Aroyo · Bilge Acun · Lingjiao Chen · Mehul Raje · Max Bartolo · Evan Sabri Eyuboglu · Amirata Ghorbani · Emmett Goodman · Addison Howard · Oana Inel · Tariq Kane · Christine Kirkpatrick · D. Sculley · Tzu-Sheng Kuo · Jonas Mueller · Tristan Thrush · Joaquin Vanschoren · Margaret Warren · Adina Williams · Serena Yeung · Newsha Ardalani · Praveen Paritosh · Ce Zhang · James Zou · Carole-Jean Wu · Cody Coleman · Andrew Ng · Peter Mattson · Vijay Janapa Reddi -
2022 Workshop: Human Evaluation of Generative Models »
Divyansh Kaushik · Jennifer Hsia · Jessica Huynh · Yonadav Shavit · Samuel Bowman · Ting-Hao Huang · Douwe Kiela · Zachary Lipton · Eric Michael Smith -
2022 Competition: NeurIPS 2022 Competition Track: Overview & Results »
Marco Ciccone · Gustavo Stolovitzky · Jake Albrecht -
2021 : Spotlight Talk: Public Information Representation for Adversarial Team Games »
Luca Carminati · Federico Cacciamani · Marco Ciccone · Nicola Gatti -
2021 : Facebook - Data Centric Infrastructure »
Douwe Kiela -
2021 Demonstration: Demonstrations 4 »
Douwe Kiela · Barbara Caputo · Marco Ciccone -
2021 : Intro »
Marco Ciccone -
2021 : Introduction to Competition Day 4 »
Marco Ciccone -
2021 Competition: Competition Track Day 4: Overviews + Breakout Sessions »
Douwe Kiela · Marco Ciccone · Barbara Caputo -
2021 Poster: True Few-Shot Learning with Language Models »
Ethan Perez · Douwe Kiela · Kyunghyun Cho -
2021 : Invited talk - Douwe Kiela »
Douwe Kiela -
2021 : Introduction to Competition Day 3 »
Marco Ciccone -
2021 Competition: Competition Track Day 3: Overviews + Breakout Sessions »
Douwe Kiela · Marco Ciccone · Barbara Caputo -
2021 Demonstration: Demonstrations 3 »
Douwe Kiela · Barbara Caputo · Marco Ciccone -
2021 : Intro »
Marco Ciccone -
2021 Poster: Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking »
Zhiyi Ma · Kawin Ethayarajh · Tristan Thrush · Somya Jain · Ledell Wu · Robin Jia · Christopher Potts · Adina Williams · Douwe Kiela -
2021 : Intro »
Douwe Kiela -
2021 : Introduction to Competition Day 2 »
Barbara Caputo -
2021 Competition: Competition Track Day 2: Overviews + Breakout Sessions »
Douwe Kiela · Marco Ciccone · Barbara Caputo -
2021 Competition: Competition Track Day 1: Overviews + Breakout Sessions »
Douwe Kiela · Marco Ciccone · Barbara Caputo -
2021 : Introduction Competion Day 1 »
Douwe Kiela -
2021 Poster: Human-Adversarial Visual Question Answering »
Sasha Sheng · Amanpreet Singh · Vedanuj Goswami · Jose Magana · Tristan Thrush · Wojciech Galuba · Devi Parikh · Douwe Kiela -
2021 Demonstration: Demonstrations 1 »
Douwe Kiela · Barbara Caputo · Marco Ciccone -
2021 : Introduction »
Douwe Kiela -
2020 : Q & A and Panel Session with Dan Weld, Kristen Grauman, Scott Yih, Emma Brunskill, and Alex Ratner »
Kristen Grauman · Wen-tau Yih · Alexander Ratner · Emma Brunskill · Douwe Kiela · Daniel S. Weld -
2020 Workshop: HAMLETS: Human And Model in the Loop Evaluation and Training Strategies »
Divyansh Kaushik · Bhargavi Paranjape · Forough Arabshahi · Yanai Elazar · Yixin Nie · Max Bartolo · Polina Kirichenko · Pontus Lars Erik Saito Stenetorp · Mohit Bansal · Zachary Lipton · Douwe Kiela -
2020 : Opening Remarks »
Divyansh Kaushik · Bhargavi Paranjape · Douwe Kiela -
2020 : The Hateful Memes Challenge: Live award ceremony and winner presentations »
Douwe Kiela -
2020 : The Hateful Memes Challenge: Competition Overview »
Douwe Kiela -
2020 Poster: The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes »
Douwe Kiela · Hamed Firooz · Aravind Mohan · Vedanuj Goswami · Amanpreet Singh · Pratik Ringshia · Davide Testuggine -
2020 Poster: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks »
Patrick Lewis · Ethan Perez · Aleksandra Piktus · Fabio Petroni · Vladimir Karpukhin · Naman Goyal · Heinrich Küttler · Mike Lewis · Wen-tau Yih · Tim Rocktäschel · Sebastian Riedel · Douwe Kiela -
2020 Poster: Learning Optimal Representations with the Decodable Information Bottleneck »
Yann Dubois · Douwe Kiela · David Schwab · Ramakrishna Vedantam -
2020 Spotlight: Learning Optimal Representations with the Decodable Information Bottleneck »
Yann Dubois · Douwe Kiela · David Schwab · Ramakrishna Vedantam -
2019 : Audrey Durand, Douwe Kiela, Kamalika Chaudhuri moderated by Yann Dauphin »
Audrey Durand · Kamalika Chaudhuri · Yann Dauphin · Orhan Firat · Dilan Gorur · Douwe Kiela -
2019 : Douwe Kiela - Benchmarking Progress in AI: A New Benchmark for Natural Language Understanding »
Douwe Kiela -
2019 Workshop: Emergent Communication: Towards Natural Language »
Abhinav Gupta · Michael Noukhovitch · Cinjon Resnick · Natasha Jaques · Angelos Filos · Marie Ossenkopf · Angeliki Lazaridou · Jakob Foerster · Ryan Lowe · Douwe Kiela · Kyunghyun Cho -
2019 Poster: Hyperbolic Graph Neural Networks »
Qi Liu · Maximilian Nickel · Douwe Kiela -
2018 Workshop: Emergent Communication Workshop »
Jakob Foerster · Angeliki Lazaridou · Ryan Lowe · Igor Mordatch · Douwe Kiela · Kyunghyun Cho -
2018 : Panel Discussion »
Antonio Torralba · Douwe Kiela · Barbara Landau · Angeliki Lazaridou · Joyce Chai · Christopher Manning · Stevan Harnad · Roozbeh Mottaghi -
2018 : Douwe Kiela - Learning Multimodal Embeddings »
Douwe Kiela -
2018 Poster: NAIS-Net: Stable Deep Networks from Non-Autonomous Differential Equations »
Marco Ciccone · Marco Gallieri · Jonathan Masci · Christian Osendorfer · Faustino Gomez -
2017 Workshop: Emergent Communication Workshop »
Jakob Foerster · Igor Mordatch · Angeliki Lazaridou · Kyunghyun Cho · Douwe Kiela · Pieter Abbeel -
2017 Poster: Poincaré Embeddings for Learning Hierarchical Representations »
Maximilian Nickel · Douwe Kiela -
2017 Spotlight: Poincaré Embeddings for Learning Hierarchical Representations »
Maximilian Nickel · Douwe Kiela -
2009 Workshop: Learning from Multiple Sources with Applications to Robotics »
Barbara Caputo · Nicolò Cesa-Bianchi · David R Hardoon · Gayle Leen · Francesco Orabona · Jaakko Peltonen · Simon Rogers -
2009 Poster: Who’s Doing What: Joint Modeling of Names and Verbs for Simultaneous Face and Pose Annotation »
Jie Luo · Barbara Caputo · Vittorio Ferrari