Timezone: »
Humans conversing naturally with machines is a staple of science fiction. Building agents capable of mutually coordinating their states and actions via communication, in conjunction with human agents, would be one of the Average engineering feats of human history. In addition to the tremendous economic potential of this technology, the ability to converse appears intimately related to the overall goal of AI.
Although dialogue has been an active area within the linguistics and NLP communities for decades, the wave of optimism in the machine learning community has inspired increased interest from researchers, companies, and foundations. The NLP community has enthusiastically embraced and innovated neural information processing systems, resulting in substantial relevant activity published outside of NIPS. A forum for increased interaction (dialogue!) with these communities at NIPS will accelerate creativity and progress.
We plan to focus on the following issues:
1. How to be data-driven
a. What are tractable and useful intermediate tasks on the path to truly conversant machines? How can we leverage existing benchmark tasks and competitions? What design criteria would we like to see for the next set of benchmark tasks and competitions?
b. How do we assess performance? What can and cannot be done with offline evaluation on fixed data sets? How can we facilitate development of these offline evaluation tasks in the public domain? What is the role of online evaluation as a benchmark, and how would we make it accessible to the general community? Is there a role for simulated environments, or tasks where machines communicate solely with each other?
2. How to build applications
a. What unexpected problem aspects arise in situated systems? human-hybrid systems? systems learning from adversarial inputs?
b. Can we divide and conquer? Do we need to a irreducible end-to-end system, or can we define modules with abstractions that do not leak?
c. How do we ease the burden on the human designer of specifying or bootstrapping the system?
3. Architectural and algorithmic innovation
a. What are the associated requisite capabilities for learning architectures, and where are the deficiencies in our current architectures? How can we leverage recent advances in reasoning, attention, and memory architectures? How can we beneficially incorporate linguistic knowledge into our architectures?
b. How far can we get with current optimization techniques? To learn requisite competencies, do we need advances in discrete optimization? curriculum learning? (inverse) reinforcement learning?
Fri 11:20 p.m. - 11:30 p.m.
|
Opening
(
Overview
)
|
🔗 |
Fri 11:25 p.m. - 11:25 p.m.
|
Building Complete Systems
(
Section
)
This set of talks and panel session is organized around the theme of building end-to-end dialog systems. |
🔗 |
Fri 11:30 p.m. - 12:10 a.m.
|
Evolvable Dialogue Systems
(
Invited Talk
)
|
Milica Gasic 🔗 |
Sat 12:10 a.m. - 12:50 a.m.
|
The Missing Pieces for a Full-Fledged Dialog Agent
(
Invited Talk
)
|
Baroni Marco 🔗 |
Sat 12:50 a.m. - 1:30 a.m.
|
Authoring End-to-End Dialog Systems ( Invited Talk ) link » | Jason Williams 🔗 |
Sat 1:30 a.m. - 2:00 a.m.
|
Break
Workshop coffee break. |
🔗 |
Sat 2:00 a.m. - 2:20 a.m.
|
Panel Session 1
(
Panel Session
)
|
🔗 |
Sat 2:20 a.m. - 4:00 a.m.
|
Lunch
|
🔗 |
Sat 3:55 a.m. - 3:55 a.m.
|
Leveraging Linguistics
(
Section
)
This set of talks and panel session is organized around the theme of leveraging linguistics to build, improve, and understand dialog systems. |
🔗 |
Sat 4:00 a.m. - 4:40 a.m.
|
Coordination and Learning in Human Dialogue
(
Invited Talk
)
|
Raquel Fernández 🔗 |
Sat 4:40 a.m. - 5:20 a.m.
|
Domain Adaptation using Linguistic Knowledge
(
Invited Talk
)
|
Nina Dethlefs 🔗 |
Sat 5:20 a.m. - 5:40 a.m.
|
Bootstrapping Incremental Dialogue Systems: Using Linguistic Knowledge to Learn from Minimal Data ( Contributed Talk ) link » | Dimitriοs Kalatzis · Arash Eshghi 🔗 |
Sat 5:40 a.m. - 6:00 a.m.
|
Multi-Agent Communication and the Emergence of (Natural) Language
(
Contributed Talk
)
|
Angeliki Lazaridou 🔗 |
Sat 6:00 a.m. - 6:30 a.m.
|
Break 2
Workshop coffee break. |
🔗 |
Sat 6:30 a.m. - 6:50 a.m.
|
Panel Session 2
(
Panel Session
)
|
🔗 |
Sat 6:45 a.m. - 6:45 a.m.
|
Modeling Techniques
(
Section
)
This set of talks and panel session is organized around the theme of modeling dialogue using machine learning techniques: in particular, what architectures to use, and how to evaluate performance. |
🔗 |
Sat 6:50 a.m. - 7:30 a.m.
|
Evaluating End-to-End Goal Oriented Dialog Systems
(
Invited Talk
)
Traditional dialog systems used in goal-oriented applications require a lot of domain-specific handcrafting, which hinders scaling up to new domains. End- to-end dialog systems, in which all components are trained from the dialogs themselves, escape this limitation. But the encouraging successes recently obtained in chit-chat dialog may not carry over to goal-oriented settings. In this talk, we will discuss how to evaluate end-to-end goal oriented dialog systems in a robust and reproducible manner. We will also present a new testbed designed to that end. On this new dataset, we show that an end-to-end dialog system based on Memory Networks can reach promising, yet imperfect, performance and learn to perform non-trivial operations. We confirm those results by comparing our system to a hand-crafted slot-filling baseline on data from the second Dialog State Tracking Challenge (Henderson et al., 2014a) and show similar result patterns on data extracted from an online concierge service. |
Antoine Bordes 🔗 |
Sat 7:30 a.m. - 8:10 a.m.
|
Awkward Silence? The Evaluation of Social Dialogue Systems
(
Invited Talk
)
Non-goal orientated dialogue systems can provide users with key information, support decision making, facilitate user action or simply chat for the sake of having some company. The key question is how do we define what it means to be an engaging conversationalist and how can we measure this automatically? This talk will take a multidisciplinary approach to evaluating data-driven, non-goal orientated dialogue systems, taking insights from the gaming industry, HCI, psychology and cognitive theory. |
Helen Hastie 🔗 |
Sat 8:10 a.m. - 8:30 a.m.
|
Learning Goal-oriented Dialog using Gated End-to-End Memory Networks
(
Contributed Talk
)
link »
In this paper, we introduce a novel memory network model using an end-to-end differentiable memory access regulation mechanism. It is inspired by the current progress on the connection short-cutting principle in the field of computer vision. We name it Gated End-to-End Memory Network (GMemN2N). From the machine learning perspective, this new capability is learned in an end-to-end fashion without the use of any additional supervision signal which is, as far as our knowledge goes, the first of its kind. Our experiments show improvements on all of the Dialog bAbI tasks, particularly on the real human-bot conversion-based Dialog State Tracking Challenge (DSTC2) dataset. This method does not require the use of any domain knowledge. Our model sets a new state of the art of end-to-end trainable dialog systems on this dataset. |
Julien Perez 🔗 |
Sat 8:30 a.m. - 8:50 a.m.
|
Generative Deep Neural Networks for Dialogue: A Short Review
(
Contributed Talk
)
link »
Researchers have recently started investigating deep neural networks for dialogue applications. In particular, generative sequence-to-sequence (Seq2Seq) models have shown promising results for unstructured tasks, such as word-level dialogue response generation. The hope is that such models will be able to leverage massive amounts of data to learn meaningful natural language representations and response generation strategies, while requiring a minimum amount of domain knowledge and hand-crafting. We review recently proposed models based on generative encoder-decoder neural network architectures, and show that these models have better ability to incorporate long-term dialogue history, to model uncertainty and ambiguity in dialogue, and to generate responses with high-level compositional structure. |
Iulian Vlad Serban 🔗 |
Sat 8:50 a.m. - 9:00 a.m.
|
Mini Break
|
🔗 |
Sat 9:00 a.m. - 9:20 a.m.
|
Panel Session 3
(
Panel Session
)
|
🔗 |
Sat 9:20 a.m. - 9:20 a.m.
|
La Fin
(
Closing
)
|
🔗 |
Author Information
Hal Daumé III (Microsoft Research & University of Maryland)
Hal Daumé III wields a professor appointment in Computer Science and Language Science at the University of Maryland, and spends time as a principal researcher in the machine learning group and fairness group at Microsoft Research in New York City. He and his wonderful advisees study questions related to how to get machines to become more adept at human language, by developing models and algorithms that allow them to learn from data. The two major questions that really drive their research these days are: (1) how can we get computers to learn language through natural interaction with people/users? and (2) how can we do this in a way that promotes fairness, transparency and explainability in the learned models?
Paul Mineiro (Microsoft)
Amanda Stent (Yahoo, Inc.)
Jason E Weston (Facebook AI Research)
Jason Weston received a PhD. (2000) from Royal Holloway, University of London under the supervision of Vladimir Vapnik. From 2000 to 2002, he was a researcher at Biowulf technologies, New York, applying machine learning to bioinformatics. From 2002 to 2003 he was a research scientist at the Max Planck Institute for Biological Cybernetics, Tuebingen, Germany. From 2004 to June 2009 he was a research staff member at NEC Labs America, Princeton. From July 2009 onwards he has been a research scientist at Google, New York. Jason Weston's current research focuses on various aspects of statistical machine learning and its applications, particularly in text and images.
More from the Same Authors
-
2020 : Invited Talk 4 Presentation - Jason Weston - (Towards) Learning from Conversing »
Jason E Weston -
2021 Spotlight: Hash Layers For Large Sparse Models »
Stephen Roller · Sainbayar Sukhbaatar · arthur szlam · Jason Weston -
2021 : Poster: The Many Roles that Causal Reasoning Plays in Reasoning about Fairness in Machine Learning »
Irene Y Chen · Hal Daumé III · Solon Barocas -
2022 : Learning to Reason and Memorize with Self-Questioning »
Jack Lanchantin · Shubham Toshniwal · Jason E Weston · arthur szlam · Sainbayar Sukhbaatar -
2022 : Towards Data-Driven Offline Simulations for Online Reinforcement Learning »
Shengpu Tang · Felipe Vieira Frujeri · Dipendra Misra · Alex Lamb · John Langford · Paul Mineiro · Sebastian Kochman -
2023 Poster: Practical Contextual Bandits with Feedback Graphs »
Mengxiao Zhang · Yuheng Zhang · Olga Vrousgou · Haipeng Luo · Paul Mineiro -
2023 Poster: $\texttt{TACO}$: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learning »
Ruijie Zheng · Xiyao Wang · Yanchao Sun · Shuang Ma · Jieyu Zhao · Huazhe Xu · Hal Daumé III · Furong Huang -
2023 Poster: Learning to Reason and Memorize with Self-Notes »
Jack Lanchantin · Shubham Toshniwal · Jason E Weston · arthur szlam · Sainbayar Sukhbaatar -
2023 Poster: Time-uniform confidence bands for the CDF under nonstationarity »
Paul Mineiro · Steven Howard -
2023 Poster: ASL Citizen: A Community-Sourced Dataset for Advancing Isolated Sign Language Recognition »
Aashaka Desai · Lauren Berger · Fyodor Minakov · Nessa Milano · Chinmay Singh · Kriston Pumphrey · Richard Ladner · Hal Daumé III · Alex X Lu · Naomi Caselli · Danielle Bragg -
2022 Workshop: HCAI@NeurIPS 2022, Human Centered AI »
Michael Muller · Plamen P Angelov · Hal Daumé III · Shion Guha · Q.Vera Liao · Nuria Oliver · David Piorkowski -
2022 : Invited Keynote by Jason Weston »
Jason Weston -
2022 : Learning to Reason and Memorize with Self-Questioning »
Jack Lanchantin · Shubham Toshniwal · Jason E Weston · arthur szlam · Sainbayar Sukhbaatar -
2022 Poster: Interaction-Grounded Learning with Action-Inclusive Feedback »
Tengyang Xie · Akanksha Saran · Dylan J Foster · Lekan Molu · Ida Momennejad · Nan Jiang · Paul Mineiro · John Langford -
2022 Poster: Staircase Attention for Recurrent Processing of Sequences »
Da JU · Stephen Roller · Sainbayar Sukhbaatar · Jason E Weston -
2021 : The Many Roles that Causal Reasoning Plays in Reasoning about Fairness in Machine Learning »
Irene Y Chen · Hal Daumé III · Solon Barocas -
2021 Poster: Bellman-consistent Pessimism for Offline Reinforcement Learning »
Tengyang Xie · Ching-An Cheng · Nan Jiang · Paul Mineiro · Alekh Agarwal -
2021 Poster: Hash Layers For Large Sparse Models »
Stephen Roller · Sainbayar Sukhbaatar · arthur szlam · Jason Weston -
2021 Oral: Bellman-consistent Pessimism for Offline Reinforcement Learning »
Tengyang Xie · Ching-An Cheng · Nan Jiang · Paul Mineiro · Alekh Agarwal -
2020 Workshop: Wordplay: When Language Meets Games »
Prithviraj Ammanabrolu · Matthew Hausknecht · Xingdi Yuan · Marc-Alexandre Côté · Adam Trischler · Kory Mathewson @korymath · John Urbanek · Jason Weston · Mark Riedl -
2020 : Panel »
Maxine Eskenazi · Ankur Parikh · Govindarajan Thattai · Alexander Rudnicky · Jason E Weston -
2020 : Invited Talk 4 Q/A - Jason Weston »
Jason E Weston -
2020 Poster: Empirical Likelihood for Contextual Bandits »
Nikos Karampatziakis · John Langford · Paul Mineiro -
2020 Memorial: In Memory of Olivier Chapelle »
Bernhard Schölkopf · Andre Elisseeff · Olivier Bousquet · Vladimir Vapnik · Jason E Weston -
2020 : Real World RL with Vowpal Wabbit: Beyond Contextual Bandits »
John Langford · Marek Wydmuch · Maryam Majzoubi · Adith Swaminathan · · Dylan Foster · Paul Mineiro -
2019 : Poster and Coffee Break 2 »
Karol Hausman · Kefan Dong · Ken Goldberg · Lihong Li · Lin Yang · Lingxiao Wang · Lior Shani · Liwei Wang · Loren Amdahl-Culleton · Lucas Cassano · Marc Dymetman · Marc Bellemare · Marcin Tomczak · Margarita Castro · Marius Kloft · Marius-Constantin Dinu · Markus Holzleitner · Martha White · Mengdi Wang · Michael Jordan · Mihailo Jovanovic · Ming Yu · Minshuo Chen · Moonkyung Ryu · Muhammad Zaheer · Naman Agarwal · Nan Jiang · Niao He · Nikolaus Yasui · Nikos Karampatziakis · Nino Vieillard · Ofir Nachum · Olivier Pietquin · Ozan Sener · Pan Xu · Parameswaran Kamalaruban · Paul Mineiro · Paul Rolland · Philip Amortila · Pierre-Luc Bacon · Prakash Panangaden · Qi Cai · Qiang Liu · Quanquan Gu · Raihan Seraj · Richard Sutton · Rick Valenzano · Robert Dadashi · Rodrigo Toro Icarte · Roshan Shariff · Roy Fox · Ruosong Wang · Saeed Ghadimi · Samuel Sokota · Sean Sinclair · Sepp Hochreiter · Sergey Levine · Sergio Valcarcel Macua · Sham Kakade · Shangtong Zhang · Sheila McIlraith · Shie Mannor · Shimon Whiteson · Shuai Li · Shuang Qiu · Wai Lok Li · Siddhartha Banerjee · Sitao Luan · Tamer Basar · Thinh Doan · Tianhe Yu · Tianyi Liu · Tom Zahavy · Toryn Klassen · Tuo Zhao · Vicenç Gómez · Vincent Liu · Volkan Cevher · Wesley Suttle · Xiao-Wen Chang · Xiaohan Wei · Xiaotong Liu · Xingguo Li · Xinyi Chen · Xingyou Song · Yao Liu · YiDing Jiang · Yihao Feng · Yilun Du · Yinlam Chow · Yinyu Ye · Yishay Mansour · · Yonathan Efroni · Yongxin Chen · Yuanhao Wang · Bo Dai · Chen-Yu Wei · Harsh Shrivastava · Hongyang Zhang · Qinqing Zheng · SIDDHARTHA SATPATHI · Xueqing Liu · Andreu Vall -
2019 : Poster Spotlight 1 »
David Brandfonbrener · Joan Bruna · Tom Zahavy · Haim Kaplan · Yishay Mansour · Nikos Karampatziakis · John Langford · Paul Mineiro · Donghwan Lee · Niao He -
2019 Poster: Reinforcement Learning with Convex Constraints »
Sobhan Miryoosefi · Kianté Brantley · Hal Daumé III · Miro Dudik · Robert Schapire -
2019 Tutorial: Imitation Learning and its Application to Natural Language Generation »
Kyunghyun Cho · Hal Daumé III -
2018 : Teaching through Dialogue and Games »
Jason E Weston -
2018 : Humans and models as embodied dialogue agents in text-based games »
Jason Weston -
2018 Workshop: Wordplay: Reinforcement and Language Learning in Text-based Games »
Adam Trischler · Angeliki Lazaridou · Yonatan Bisk · Wendy Tay · Nate Kushman · Marc-Alexandre Côté · Alessandro Sordoni · Daniel Ricks · Tom Zahavy · Hal Daumé III -
2018 : The Conversational Intelligence Challenge 2 (ConvAI2) : Setup, Opening Words »
Jason Weston -
2017 : Competition V: Human-Computer Question Answering »
Jordan Boyd-Graber · Hal Daumé III · He He · Mohit Iyyer · Pedro Rodriguez -
2016 : Jason Weston »
Jason E Weston -
2016 Poster: Dialog-based Language Learning »
Jason E Weston -
2016 Poster: A Credit Assignment Compiler for Joint Prediction »
Kai-Wei Chang · He He · Stephane Ross · Hal Daumé III · John Langford -
2015 Workshop: Reasoning, Attention, Memory (RAM) Workshop »
Jason E Weston · Sumit Chopra · Antoine Bordes -
2015 : Evaluating Prerequisite Qualities For End-to-End Dialog Systems »
Jason E Weston -
2015 Poster: End-To-End Memory Networks »
Sainbayar Sukhbaatar · arthur szlam · Jason Weston · Rob Fergus -
2015 Oral: End-To-End Memory Networks »
Sainbayar Sukhbaatar · arthur szlam · Jason Weston · Rob Fergus -
2014 Workshop: Second Workshop on Transfer and Multi-Task Learning: Theory meets Practice »
Urun Dogan · Tatiana Tommasi · Yoshua Bengio · Francesco Orabona · Marius Kloft · Andres Munoz · Gunnar Rätsch · Hal Daumé III · Mehryar Mohri · Xuezhi Wang · Daniel Hernández-lobato · Song Liu · Thomas Unterthiner · Pascal Germain · Vinay P Namboodiri · Michael Goetz · Christopher Berlind · Sigurd Spieckermann · Marta Soare · Yujia Li · Vitaly Kuznetsov · Wenzhao Lian · Daniele Calandriello · Emilie Morvant -
2014 Workshop: Representation and Learning Methods for Complex Outputs »
Richard Zemel · Dale Schuurmans · Kilian Q Weinberger · Yuhong Guo · Jia Deng · Francesco Dinuzzo · Hal Daumé III · Honglak Lee · Noah A Smith · Richard Sutton · Jiaqian YU · Vitaly Kuznetsov · Luke Vilnis · Hanchen Xiong · Calvin Murdock · Thomas Unterthiner · Jean-Francis Roy · Martin Renqiang Min · Hichem SAHBI · Fabio Massimo Zanzotto -
2014 Workshop: 4th Workshop on Automated Knowledge Base Construction (AKBC) »
Sameer Singh · Fabian M Suchanek · Sebastian Riedel · Partha Pratim Talukdar · Kevin Murphy · Christopher Ré · William Cohen · Tom Mitchell · Andrew McCallum · Jason E Weston · Ramanathan Guha · Boyan Onyshkevych · Hoifung Poon · Oren Etzioni · Ari Kobren · Arvind Neelakantan · Peter Clark -
2014 Poster: Learning to Search in Branch and Bound Algorithms »
He He · Hal Daumé III · Jason Eisner -
2013 Poster: Binary to Bushy: Bayesian Hierarchical Clustering with the Beta Coalescent »
Yuening Hu · Jordan Boyd-Graber · Hal Daumé III · Z. Irene Ying -
2012 Poster: Imitation Learning by Coaching »
He He · Hal Daumé III · Jason Eisner -
2012 Poster: Simultaneously Leveraging Output and Task Structures for Multiple-Output Regression »
Piyush Rai · Abhishek Kumar · Hal Daumé III -
2012 Poster: Learned Prioritization for Trading Off Accuracy and Speed »
Jiarong Jiang · Adam Teichert · Hal Daumé III · Jason Eisner -
2011 Workshop: Learning Semantics »
Antoine Bordes · Jason E Weston · Ronan Collobert · Leon Bottou -
2011 Poster: Message-Passing for Approximate MAP Inference with Latent Variables »
Jiarong Jiang · Piyush Rai · Hal Daumé III -
2011 Poster: Co-regularized Multi-view Spectral Clustering »
Abhishek Kumar · Piyush Rai · Hal Daumé III -
2010 Poster: Label Embedding Trees for Large Multi-Class Tasks »
Samy Bengio · Jason E Weston · David Grangier -
2010 Poster: Learning Multiple Tasks using Manifold Regularization »
Arvind Agarwal · Hal Daumé III · Samuel Gerber -
2010 Poster: Co-regularization Based Semi-supervised Domain Adaptation »
Hal Daumé III · Abhishek Kumar · Avishek Saha -
2009 Poster: Multi-Label Prediction via Sparse Infinite CCA »
Piyush Rai · Hal Daumé III -
2009 Poster: Polynomial Semantic Indexing »
Bing Bai · Jason E Weston · David Grangier · Ronan Collobert · Kunihiko Sadamasa · Yanjun Qi · Corinna Cortes · Mehryar Mohri -
2009 Tutorial: Deep Learning in Natural Language Processing »
Ronan Collobert · Jason E Weston -
2008 Poster: Nonparametric Bayesian Sparse Hierarchical Factor Modeling and Regression »
Piyush Rai · Hal Daumé III -
2007 Poster: Bayesian Agglomerative Clustering with Coalescents »
Yee Whye Teh · Hal Daumé III · Daniel Roy -
2007 Oral: Bayesian Agglomerative Clustering with Coalescents »
Yee Whye Teh · Hal Daumé III · Daniel Roy