Timezone: »
Reinforcement Learning (RL) has had numerous successes in recent years in solving complex problem domains. However, this progress has been largely limited to domains where a simulator is available or the real environment is quick and easy to access. This is one of a number of challenges that are bottlenecks to deploying RL agents on real-world systems. Two recent papers identify nine important challenges that, if solved, will take a big step towards enabling RL agents to be deployed to real-world systems (Dulac et. al. 2019, 2020).The goals of this workshop are four-fold: (1) Providing a forum for researchers in academia, industry researchers as well as industry practitioners from diverse backgrounds to discuss the challenges faced in real-world systems; (2) discuss and prioritize the nine research challenges. This includes determining which challenges we should focus on next, whether any new challenges should be added to the list or existing ones removed from this list; (3) Discuss problem formulations for the various challenges and critique these formulations or develop new ones. This is especially important for more abstract challenges such as explainability. We should also be asking ourselves whether the current Markov Decision Process (MDP) formulation is sufficient for solving these problems or whether modifications need to be made. (4) Discuss approaches to solving combinations of these challenges.
Sat 8:30 a.m. - 8:40 a.m.
|
Introduction and Overview ( Introduction ) link » | Daniel Mankowitz · Gabriel Dulac-Arnold 🔗 |
Sat 8:40 a.m. - 9:20 a.m.
|
Keynote: Aviv Tamar
(
Talk
)
SlidesLive Video » Real World RL Challenges |
Aviv Tamar 🔗 |
Sat 9:20 a.m. - 10:00 a.m.
|
Keynote: Emma Brunskill
(
Talk
)
SlidesLive Video » More practical Batch Offline Reinforcement Learning |
Emma Brunskill 🔗 |
Sat 10:00 a.m. - 10:40 a.m.
|
Keynote: Jost Tobias Springenberg
(
Talk
)
link »
Challenges for RL in Robotics |
Jost Tobias Springenberg 🔗 |
Sat 10:40 a.m. - 11:20 a.m.
|
Mini-panel discussion 1 - Bridging the gap between theory and practice
(
Discussion Panel
)
|
Aviv Tamar · Emma Brunskill · Jost Tobias Springenberg · Omer Gottesman · Daniel Mankowitz 🔗 |
Sat 11:20 a.m. - 11:50 a.m.
|
Poster session 1
(
Poster Session and Coffee Break
)
link »
You can now chat to the paper authors by clicking the above Gather.town link Links to individual poster presentations can be found here: https://sites.google.com/corp/view/neurips2020rwrl#h.ey6lwdtrdt7c |
🔗 |
Sat 11:50 a.m. - 12:30 p.m.
|
Keynote: Franziska Meier
(
Talk
)
SlidesLive Video » Challenges of Model-based Inverse Reinforcement Learning |
🔗 |
Sat 12:30 p.m. - 1:10 p.m.
|
Keynote: Marc Raibert, Scott Kuindersma
(
Talk
)
link »
SlidesLive Video » Boston Dynamics |
🔗 |
Sat 1:10 p.m. - 1:50 p.m.
|
Mini-panel discussion 2 - Real World RL: An industry perspective
(
Discussion Panel
)
The following speakers that will be at this event do not have Neurips profiles: Franziska Meier - fmeier@fb.com Marc Reibert - marcraibert@bostondynamics.com Scott Kuindersma - skuindersma@bostondynamics.com |
Franziska Meier · Gabriel Dulac-Arnold · Shie Mannor · Timothy A Mann 🔗 |
Sat 1:50 p.m. - 3:20 p.m.
|
Lunch
(
Lunch Break
)
Enjoy your lunch break. If you intend to attend the 3rd mini-panel session, we encourage you to watch the talks of Anca Dragan and Angela Schoellig during lunch as their keynote talks will only occur after the mini-panel session. Thus, if you want to ask them questions, please take the time to watch the talks now. |
🔗 |
Sat 3:20 p.m. - 4:00 p.m.
|
Spotlight Talks
(
Talk
)
link »
We have 4 spotlight talks. These talks can be found at the following link: https://sites.google.com/corp/view/neurips2020rwrl#h.9w5kdo7eecim |
🔗 |
Sat 4:00 p.m. - 4:40 p.m.
|
Keynote: Tom Diettrich
(
Talk
)
SlidesLive Video » Applying RL to Ecosystem Management: Lessons Learned |
Thomas Dietterich 🔗 |
Sat 4:40 p.m. - 5:20 p.m.
|
Keynote: Chelsea Finn
(
Talk
)
SlidesLive Video » Reinforcement Learning for Real Robots |
Chelsea Finn 🔗 |
Sat 5:20 p.m. - 6:00 p.m.
|
Mini-panel discussion 3 - Prioritizing Real World RL Challenges
(
Discussion Panel
)
|
Chelsea Finn · Thomas Dietterich · Angela Schoellig · Anca Dragan · Anusha Nagabandi · Doina Precup 🔗 |
Sat 6:00 p.m. - 6:30 p.m.
|
Poster session 2
(
Poster Session and Coffee Break
)
link »
You can now chat to the paper authors by clicking the above Gather.town link Links to individual poster presentations can be found here: https://sites.google.com/corp/view/neurips2020rwrl#h.ey6lwdtrdt7c |
🔗 |
Sat 6:30 p.m. - 7:10 p.m.
|
Keynote: Angela Schoellig
(
Talk
)
link »
Machine Learning for Safety-Critical Robotics Applications |
Angela Schoellig 🔗 |
Sat 7:10 p.m. - 7:50 p.m.
|
Keynote: Anca Dragan
(
Talk
)
SlidesLive Video » Reinforcement Learning that optimizes what people really want |
Anca Dragan 🔗 |
Author Information
Daniel Mankowitz (DeepMind)
Gabriel Dulac-Arnold (Google Research)
Shie Mannor (Technion)
Omer Gottesman (Harvard)
Anusha Nagabandi (UC Berkeley)
Doina Precup (DeepMind)
Timothy A Mann (DeepMind)
Gabriel Dulac-Arnold (Google Research)
More from the Same Authors
-
2021 Spotlight: Flexible Option Learning »
Martin Klissarov · Doina Precup -
2021 : Bandits with Partially Observable Confounded Data »
Guy Tennenholtz · Uri Shalit · Shie Mannor · Yonathan Efroni -
2021 : Covariate Shift of Latent Confounders in Imitation and Reinforcement Learning »
Guy Tennenholtz · Assaf Hallak · Gal Dalal · Shie Mannor · Gal Chechik · Uri Shalit -
2021 : Policy Gradients Incorporating the Future »
David Venuto · Elaine Lau · Doina Precup · Ofir Nachum -
2021 : A Consciousness-Inspired Planning Agent for Model-Based Reinforcement Learning »
Mingde Zhao · Zhen Liu · Sitao Luan · Shuyuan Zhang · Doina Precup · Yoshua Bengio -
2021 : Covariate Shift of Latent Confounders in Imitation and Reinforcement Learning »
Guy Tennenholtz · Assaf Hallak · Gal Dalal · Shie Mannor · Gal Chechik · Uri Shalit -
2021 : Latent Geodesics of Model Dynamics for Offline Reinforcement Learning »
Guy Tennenholtz · Nir Baram · Shie Mannor -
2022 : Controlling Commercial Cooling Systems Using Reinforcement Learning »
Jerry Luo · Cosmin Paduraru · Octavian Voicu · Yuri Chervonyi · Scott Munns · Jerry Li · Crystal Qian · Praneet Dutta · Daniel Mankowitz · Jared Quincy Davis · Ningjia Wu · Xingwei Yang · Chu-Ming Chang · Ted Li · Rob Rose · Mingyan Fan · Hootan Nakhost · Tinglin Liu · Deeni Fatiha · Neil Satra · Juliet Rothenberg · Molly Carlin · Satish Tallapaka · Sims Witherspoon · David Parish · Peter Dolan · Chenyu Zhao -
2022 : Controlling Commercial Cooling Systems Using Reinforcement Learning »
Jerry Luo · Cosmin Paduraru · Octavian Voicu · Yuri Chervonyi · Scott Munns · Jerry Li · Crystal Qian · Praneet Dutta · Daniel Mankowitz · Jared Quincy Davis · Ningjia Wu · Xingwei Yang · Chu-Ming Chang · Ted Li · Rob Rose · Mingyan Fan · Hootan Nakhost · Tinglin Liu · Deeni Fatiha · Neil Satra · Juliet Rothenberg · Molly Carlin · Satish Tallapaka · Sims Witherspoon · David Parish · Peter Dolan · Chenyu Zhao -
2021 : Invited Speaker Panel »
Sham Kakade · Minmin Chen · Philip Thomas · Angela Schoellig · Barbara Engelhardt · Doina Precup · George Tucker -
2021 Poster: On the Expressivity of Markov Reward »
David Abel · Will Dabney · Anna Harutyunyan · Mark Ho · Michael Littman · Doina Precup · Satinder Singh -
2021 Poster: Gradient Starvation: A Learning Proclivity in Neural Networks »
Mohammad Pezeshki · Oumar Kaba · Yoshua Bengio · Aaron Courville · Doina Precup · Guillaume Lajoie -
2021 Poster: Flexible Option Learning »
Martin Klissarov · Doina Precup -
2021 Poster: A Consciousness-Inspired Planning Agent for Model-Based Reinforcement Learning »
Mingde Zhao · Zhen Liu · Sitao Luan · Shuyuan Zhang · Doina Precup · Yoshua Bengio -
2021 Poster: Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation »
Emmanuel Bengio · Moksh Jain · Maksym Korablyov · Doina Precup · Yoshua Bengio -
2021 Poster: Temporally Abstract Partial Models »
Khimya Khetarpal · Zafarali Ahmed · Gheorghe Comanici · Doina Precup -
2021 Poster: Active Offline Policy Selection »
Ksenia Konyushova · Yutian Chen · Thomas Paine · Caglar Gulcehre · Cosmin Paduraru · Daniel Mankowitz · Misha Denil · Nando de Freitas -
2021 Poster: Data Augmentation Can Improve Robustness »
Sylvestre-Alvise Rebuffi · Sven Gowal · Dan Andrei Calian · Florian Stimberg · Olivia Wiles · Timothy A Mann -
2021 Poster: Improving Robustness using Generated Data »
Sven Gowal · Sylvestre-Alvise Rebuffi · Olivia Wiles · Florian Stimberg · Dan Andrei Calian · Timothy A Mann -
2021 Oral: On the Expressivity of Markov Reward »
David Abel · Will Dabney · Anna Harutyunyan · Mark Ho · Michael Littman · Doina Precup · Satinder Singh -
2020 : Mini-panel discussion 3 - Prioritizing Real World RL Challenges »
Chelsea Finn · Thomas Dietterich · Angela Schoellig · Anca Dragan · Anusha Nagabandi · Doina Precup -
2020 : Mini-panel discussion 2 - Real World RL: An industry perspective »
Franziska Meier · Gabriel Dulac-Arnold · Shie Mannor · Timothy A Mann -
2020 : Mini-panel discussion 1 - Bridging the gap between theory and practice »
Aviv Tamar · Emma Brunskill · Jost Tobias Springenberg · Omer Gottesman · Daniel Mankowitz -
2020 : Introduction and Overview »
Daniel Mankowitz · Gabriel Dulac-Arnold -
2020 : Panel discussion »
Pierre-Yves Oudeyer · Marc Bellemare · Peter Stone · Matt Botvinick · Susan Murphy · Anusha Nagabandi · Ashley Edwards · Karen Liu · Pieter Abbeel -
2020 : Invited talk: Anusha Nagabandi "Model-based Deep Reinforcement Learning for Robotic Systems" »
Anusha Nagabandi -
2020 Poster: Value-driven Hindsight Modelling »
Arthur Guez · Fabio Viola · Theophane Weber · Lars Buesing · Steven Kapturowski · Doina Precup · David Silver · Nicolas Heess -
2020 Poster: RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learning »
Caglar Gulcehre · Ziyu Wang · Alexander Novikov · Thomas Paine · Sergio Gómez · Konrad Zolna · Rishabh Agarwal · Josh Merel · Daniel Mankowitz · Cosmin Paduraru · Gabriel Dulac-Arnold · Jerry Li · Mohammad Norouzi · Matthew Hoffman · Nicolas Heess · Nando de Freitas -
2020 Poster: On Efficiency in Hierarchical Reinforcement Learning »
Zheng Wen · Doina Precup · Morteza Ibrahimi · Andre Barreto · Benjamin Van Roy · Satinder Singh -
2020 Poster: Online Planning with Lookahead Policies »
Yonathan Efroni · Mohammad Ghavamzadeh · Shie Mannor -
2020 Poster: Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model »
Alex X. Lee · Anusha Nagabandi · Pieter Abbeel · Sergey Levine -
2020 Spotlight: On Efficiency in Hierarchical Reinforcement Learning »
Zheng Wen · Doina Precup · Morteza Ibrahimi · Andre Barreto · Benjamin Van Roy · Satinder Singh -
2019 : Poster Session »
Matthia Sabatelli · Adam Stooke · Amir Abdi · Paulo Rauber · Leonard Adolphs · Ian Osband · Hardik Meisheri · Karol Kurach · Johannes Ackermann · Matt Benatan · GUO ZHANG · Chen Tessler · Dinghan Shen · Mikayel Samvelyan · Riashat Islam · Murtaza Dalal · Luke Harries · Andrey Kurenkov · Konrad Żołna · Sudeep Dasari · Kristian Hartikainen · Ofir Nachum · Kimin Lee · Markus Holzleitner · Vu Nguyen · Francis Song · Christopher Grimm · Felipe Leno da Silva · Yuping Luo · Yifan Wu · Alex Lee · Thomas Paine · Wei-Yang Qu · Daniel Graves · Yannis Flet-Berliac · Yunhao Tang · Suraj Nair · Matthew Hausknecht · Akhil Bagaria · Simon Schmitt · Bowen Baker · Paavo Parmas · Benjamin Eysenbach · Lisa Lee · Siyu Lin · Daniel Seita · Abhishek Gupta · Riley Simmons-Edler · Yijie Guo · Kevin Corder · Vikash Kumar · Scott Fujimoto · Adam Lerer · Ignasi Clavera Gilaberte · Nicholas Rhinehart · Ashvin Nair · Ge Yang · Lingxiao Wang · Sungryull Sohn · J. Fernando Hernandez-Garcia · Xian Yeow Lee · Rupesh Srivastava · Khimya Khetarpal · Chenjun Xiao · Luckeciano Carvalho Melo · Rishabh Agarwal · Tianhe Yu · Glen Berseth · Devendra Singh Chaplot · Jie Tang · Anirudh Srinivasan · Tharun Kumar Reddy Medini · Aaron Havens · Misha Laskin · Asier Mujika · Rohan Saphal · Joseph Marino · Alex Ray · Joshua Achiam · Ajay Mandlekar · Zhuang Liu · Danijar Hafner · Zhiwen Tang · Ted Xiao · Michael Walton · Jeff Druce · Ferran Alet · Zhang-Wei Hong · Stephanie Chan · Anusha Nagabandi · Hao Liu · Hao Sun · Ge Liu · Dinesh Jayaraman · John Co-Reyes · Sophia Sanborn -
2019 Poster: Distributional Policy Optimization: An Alternative Approach for Continuous Control »
Chen Tessler · Guy Tennenholtz · Shie Mannor -
2019 Poster: Adaptive Temporal-Difference Learning for Policy Evaluation with Per-State Uncertainty Estimates »
Carlos Riquelme · Hugo Penedones · Damien Vincent · Hartmut Maennel · Sylvain Gelly · Timothy A Mann · Andre Barreto · Gergely Neu -
2019 Demonstration: The Option Keyboard: Combining Skills in Reinforcement Learning »
Daniel Toyama · Shaobo Hou · Gheorghe Comanici · Andre Barreto · Doina Precup · Shibl Mourad · Eser Aygün · Philippe Hamel -
2019 Poster: The Option Keyboard: Combining Skills in Reinforcement Learning »
Andre Barreto · Diana Borsa · Shaobo Hou · Gheorghe Comanici · Eser Aygün · Philippe Hamel · Daniel Toyama · jonathan j hunt · Shibl Mourad · David Silver · Doina Precup -
2019 Poster: Value Propagation for Decentralized Networked Deep Multi-agent Reinforcement Learning »
Chao Qu · Shie Mannor · Huan Xu · Yuan Qi · Le Song · Junwu Xiong -
2019 Poster: Hindsight Credit Assignment »
Anna Harutyunyan · Will Dabney · Thomas Mesnard · Mohammad Gheshlaghi Azar · Bilal Piot · Nicolas Heess · Hado van Hasselt · Gregory Wayne · Satinder Singh · Doina Precup · Remi Munos -
2019 Spotlight: Hindsight Credit Assignment »
Anna Harutyunyan · Will Dabney · Thomas Mesnard · Mohammad Gheshlaghi Azar · Bilal Piot · Nicolas Heess · Hado van Hasselt · Gregory Wayne · Satinder Singh · Doina Precup · Remi Munos -
2018 : Discussion Panel: Ryan Adams, Nicolas Heess, Leslie Kaelbling, Shie Mannor, Emo Todorov (moderator: Roy Fox) »
Ryan Adams · Nicolas Heess · Leslie Kaelbling · Shie Mannor · Emo Todorov · Roy Fox -
2018 : Hierarchical RL: From Prior Knowledge to Policies (Shie Mannor) »
Shie Mannor -
2018 : Learning to Adapt in Dynamic, Real-World Environments via Meta-Reinforcement Learning »
Anusha Nagabandi -
2018 : Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL »
Anusha Nagabandi -
2018 Poster: Learn What Not to Learn: Action Elimination with Deep Reinforcement Learning »
Tom Zahavy · Matan Haroush · Nadav Merlis · Daniel J Mankowitz · Shie Mannor -
2017 Workshop: Hierarchical Reinforcement Learning »
Andrew G Barto · Doina Precup · Shie Mannor · Tom Schaul · Roy Fox · Carlos Florensa