Workshop
3rd Offline Reinforcement Learning Workshop: Offline RL as a "Launchpad"
Aviral Kumar 路 Rishabh Agarwal 路 Aravind Rajeswaran 路 Wenxuan Zhou 路 George Tucker 路 Doina Precup 路 Aviral Kumar
Room 291 - 292
Fri 2 Dec, 6:20 a.m. PST
While offline RL focuses on learning solely from fixed datasets, one of the main learning points from the previous edition of offline RL workshop was that large-scale RL applications typically want to use offline RL as part of a bigger system as opposed to being the end-goal in itself. Thus, we propose to shift the focus from algorithm design and offline RL applications to how offline RL can be a launchpad , i.e., a tool or a starting point, for solving challenges in sequential decision-making such as exploration, generalization, transfer, safety, and adaptation. Particularly, we are interested in studying and discussing methods for learning expressive models, policies, skills and value functions from data that can help us make progress towards efficiently tackling these challenges, which are otherwise often intractable.
Submission site: https://openreview.net/group?id=NeurIPS.cc/2022/Workshop/Offline_RL. The submission deadline is September 25, 2022 (Anywhere on Earth). Please refer to the submission page for more details.
Schedule
Fri 6:20 a.m. - 6:30 a.m.
|
Opening Remarks
(
Opening Remarks
)
>
SlidesLive Video |
馃敆 |
Fri 6:30 a.m. - 7:00 a.m.
|
Offline RL in the context of "Collect and Infer" (Martin Riedmiller)
(
Invited Talk
)
>
SlidesLive Video |
馃敆 |
Fri 7:00 a.m. - 7:10 a.m.
|
Efficient Planning in a Compact Latent Action Space
(
Contributed Talk
)
>
SlidesLive Video |
馃敆 |
Fri 7:10 a.m. - 7:20 a.m.
|
Control Graph as Unified IO for Morphology-Task Generalization
(
Contributed Talk
)
>
SlidesLive Video |
馃敆 |
Fri 7:20 a.m. - 7:30 a.m.
|
Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training
(
Contributed Talk
)
>
SlidesLive Video |
馃敆 |
Fri 7:35 a.m. - 8:05 a.m.
|
AV2.0: Learning to Drive at a Global Scale (Alex Kendall)
(
Invited Talk
)
>
SlidesLive Video |
馃敆 |
Fri 8:05 a.m. - 9:10 a.m.
|
Poster Session 1
(
Poster Session
)
>
|
馃敆 |
Fri 9:10 a.m. - 9:40 a.m.
|
Learning from Suboptimal Demonstrations with No Rewards (Dorsa Sadigh)
(
Invited Talk
)
>
SlidesLive Video |
馃敆 |
Fri 9:40 a.m. - 10:30 a.m.
|
Break
|
馃敆 |
Fri 10:45 a.m. - 11:30 a.m.
|
Panel Discussion 1 - Applications
(
Panel Discussion
)
>
SlidesLive Video |
馃敆 |
Fri 11:30 a.m. - 11:40 a.m.
|
Choreographer: Learning and Adapting Skills in Imagination
(
Contributed Talk
)
>
SlidesLive Video |
馃敆 |
Fri 11:40 a.m. - 11:50 a.m.
|
Provable Benefits of Representational Transfer in Reinforcement Learning
(
Contributed Talk
)
>
SlidesLive Video |
馃敆 |
Fri 11:50 a.m. - 12:00 p.m.
|
Pareto-Efficient Decision Agents for Offline Multi-Objective Reinforcement Learning
(
Contributed Talk
)
>
SlidesLive Video |
馃敆 |
Fri 12:00 p.m. - 1:00 p.m.
|
Poster Session 2
(
Poster Session
)
>
|
馃敆 |
Fri 1:00 p.m. - 1:30 p.m.
|
Reinforcement Learning and LTV at Spotify (Tony Jebara)
(
Invited Talk
)
>
SlidesLive Video |
馃敆 |
Fri 1:30 p.m. - 2:00 p.m.
|
Hybrid RL: Using Both Offline and Online Data Can Make RL Efficient (Wen Sun)
(
Invited Talk
)
>
SlidesLive Video |
馃敆 |
Fri 2:00 p.m. - 3:00 p.m.
|
Panel Discussion 2 - Research
(
Panel Discussion
)
>
SlidesLive Video |
馃敆 |
Fri 3:00 p.m. - 3:30 p.m.
|
Identification of Dead-ends in Safety-Critical Offline RL (Talyor Killian)
(
Invited Talk
)
>
SlidesLive Video |
馃敆 |
-
|
Agent-Controller Representations: Principled Offline RL with Rich Exogenous Information ( Poster ) > link |
11 presentersRiashat Islam 路 Manan Tomar 路 Alex Lamb 路 Hongyu Zang 路 Yonathan Efroni 路 Dipendra Misra 路 Aniket Didolkar 路 Xin Li 路 Harm Van Seijen 路 Remi Tachet des Combes 路 John Langford |
-
|
Proto-Value Networks: Scaling Representation Learning with Auxiliary Tasks ( Poster ) > link | Jesse Farebrother 路 Joshua Greaves 路 Rishabh Agarwal 路 Charline Le Lan 路 Ross Goroshin 路 Pablo Samuel Castro 路 Marc Bellemare 馃敆 |
-
|
Confidence-Conditioned Value Functions for Offline Reinforcement Learning ( Poster ) > link | Joey Hong 路 Aviral Kumar 路 Sergey Levine 馃敆 |
-
|
Efficient Deep Reinforcement Learning Requires Regulating Statistical Overfitting ( Poster ) > link | Qiyang Li 路 Aviral Kumar 路 Ilya Kostrikov 路 Sergey Levine 馃敆 |
-
|
Domain Generalization for Robust Model-Based Offline RL
(
Poster
)
>
link
SlidesLive Video |
Alan Clark 路 Shoaib Siddiqui 路 Robert Kirk 路 Usman Anwar 路 Stephen Chung 路 David Krueger 馃敆 |
-
|
Squeezing more value out of your historical data: data-augmented behavioural cloning as launchpad for reinforcement learning
(
Poster
)
>
link
SlidesLive Video |
Charles Hepburn 路 Giovanni Montana 馃敆 |
-
|
Keep Calm and Carry Offline: Policy refinement in offline reinforcement learning
(
Poster
)
>
link
SlidesLive Video |
Alex Beeson 路 Giovanni Montana 馃敆 |
-
|
Guiding Offline Reinforcement Learning Using a Safety Expert ( Poster ) > link | Richa Verma 路 Kartik Bharadwaj 路 Harshad Khadilkar 路 Balaraman Ravindran 馃敆 |
-
|
Pareto-Efficient Decision Agents for Offline Multi-Objective Reinforcement Learning
(
Poster
)
>
link
SlidesLive Video |
Baiting Zhu 路 Meihua Dang 路 Aditya Grover 馃敆 |
-
|
Revisiting Bellman Errors for Offline Model Selection ( Poster ) > link | Joshua Zitovsky 路 Rishabh Agarwal 路 Daniel de Marchi 路 Michael Kosorok 馃敆 |
-
|
Boosting Offline Reinforcement Learning via Data Resampling ( Poster ) > link | Yang Yue 路 Bingyi Kang 路 Xiao Ma 路 Zhongwen Xu 路 Gao Huang 路 Shuicheng Yan 馃敆 |
-
|
General policy mapping: online continual reinforcement learning inspired on the insect brain
(
Poster
)
>
link
SlidesLive Video |
Angel Yanguas-Gil 路 Sandeep Madireddy 馃敆 |
-
|
Offline Reinforcement Learning with Closed-Form Policy Improvement Operators
(
Poster
)
>
link
SlidesLive Video |
Jiachen Li 路 Edwin Zhang 路 Ming Yin 路 Qinxun Bai 路 Yu-Xiang Wang 路 William Yang Wang 馃敆 |
-
|
On- and Offline Multi-agent Reinforcement Learning for Disease Mitigation using Human Mobility Data ( Poster ) > link | Sofia Hurtado 路 Radu Marculescu 馃敆 |
-
|
Contrastive Example-Based Control ( Poster ) > link | Kyle Hatch 路 Sarthak J Shetty 路 Benjamin Eysenbach 路 Tianhe Yu 路 Rafael Rafailov 路 Russ Salakhutdinov 路 Sergey Levine 路 Chelsea Finn 馃敆 |
-
|
Offline Policy Evaluation for Reinforcement Learning with Adaptively Collected Data ( Poster ) > link | Sunil Madhow 路 Dan Qiao 路 Yu-Xiang Wang 馃敆 |
-
|
Bridging the Gap Between Offline and Online Reinforcement Learning Evaluation Methodologies
(
Poster
)
>
link
SlidesLive Video |
Shivakanth Sujit 路 Pedro Braga 路 J枚rg Bornschein 路 Samira Ebrahimi Kahou 馃敆 |
-
|
Offline Policy Comparison with Confidence: Benchmarks and Baselines ( Poster ) > link | Anurag Koul 路 Mariano Phielipp 路 Alan Fern 馃敆 |
-
|
Residual Model-Based Reinforcement Learning for Physical Dynamics
(
Poster
)
>
link
SlidesLive Video |
Zakariae EL ASRI 路 Cl茅ment Rambour 路 Vincent LE GUEN 路 Nicolas THOME 馃敆 |
-
|
Raisin: Residual Algorithms for Versatile Offline Reinforcement Learning ( Poster ) > link | Braham Snyder 路 Yuke Zhu 馃敆 |
-
|
Collaborative symmetricity exploitation for offline learning of hardware design solver ( Poster ) > link | HAEYEON KIM 路 Minsu Kim 路 joungho kim 路 Jinkyoo Park 馃敆 |
-
|
SPRINT: Scalable Semantic Policy Pre-training via Language Instruction Relabeling
(
Poster
)
>
link
SlidesLive Video |
Jesse Zhang 路 Karl Pertsch 路 Jiahui Zhang 路 Taewook Nam 路 Sung Ju Hwang 路 Xiang Ren 路 Joseph Lim 馃敆 |
-
|
Bayesian Q-learning With Imperfect Expert Demonstrations
(
Poster
)
>
link
SlidesLive Video |
Fengdi Che 路 Xiru Zhu 路 Doina Precup 路 David Meger 路 Gregory Dudek 馃敆 |
-
|
Can Active Sampling Reduce Causal Confusion in Offline Reinforcement Learning? ( Poster ) > link | Gunshi Gupta 路 Tim G. J. Rudner 路 Rowan McAllister 路 Adrien Gaidon 路 Yarin Gal 馃敆 |
-
|
Trajectory-based Explainability Framework for Offline RL ( Poster ) > link | Shripad Deshmukh 路 Arpan Dasgupta 路 Chirag Agarwal 路 Nan Jiang 路 Balaji Krishnamurthy 路 Georgios Theocharous 路 Jayakumar Subramanian 馃敆 |
-
|
AMORE: A Model-based Framework for Improving Arbitrary Baseline Policies with Offline Data ( Poster ) > link | Tengyang Xie 路 Mohak Bhardwaj 路 Nan Jiang 路 Ching-An Cheng 馃敆 |
-
|
Balanced Off-Policy Evaluation for Personalized Pricing ( Poster ) > link | Adam N. Elmachtoub 路 Vishal Gupta 路 YUNFAN ZHAO 馃敆 |
-
|
ABC: Adversarial Behavioral Cloning for Offline Mode-Seeking Imitation Learning ( Poster ) > link | Eddy Hudson 路 Ishan Durugkar 路 Garrett Warnell 路 Peter Stone 馃敆 |
-
|
Dynamics-Augmented Decision Transformer for Offline Dynamics Generalization ( Poster ) > link | Changyeon Kim 路 Junsu Kim 路 Younggyo Seo 路 Kimin Lee 路 Honglak Lee 路 Jinwoo Shin 馃敆 |
-
|
Offline Reinforcement Learning on Real Robot with Realistic Data Sources
(
Poster
)
>
link
SlidesLive Video |
Gaoyue Zhou 路 Liyiming Ke 路 Siddhartha Srinivasa 路 Abhinav Gupta 路 Aravind Rajeswaran 路 Vikash Kumar 馃敆 |
-
|
Let Offline RL Flow: Training Conservative Agents in the Latent Space of Normalizing Flows
(
Poster
)
>
link
SlidesLive Video |
Dmitry Akimov 路 Alexander Nikulin 路 Vladislav Kurenkov 路 Denis Tarasov 路 Sergey Kolesnikov 馃敆 |
-
|
Matrix Estimation for Offline Evaluation in Reinforcement Learning with Low-Rank Structure ( Poster ) > link | Xumei Xi 路 Christina Yu 路 Yudong Chen 馃敆 |
-
|
Train Offline, Test Online: A Real Robot Learning Benchmark
(
Poster
)
>
link
SlidesLive Video |
12 presentersGaoyue Zhou 路 Victoria Dean 路 Mohan Kumar Srirama 路 Aravind Rajeswaran 路 Jyothish Pari 路 Kyle Hatch 路 Aryan Jain 路 Tianhe Yu 路 Pieter Abbeel 路 Lerrel Pinto 路 Chelsea Finn 路 Abhinav Gupta |
-
|
Hybrid RL: Using both offline and online data can make RL efficient
(
Poster
)
>
link
SlidesLive Video |
Yuda Song 路 Yifei Zhou 路 Ayush Sekhari 路 J. Bagnell 路 Akshay Krishnamurthy 路 Wen Sun 馃敆 |
-
|
Choreographer: Learning and Adapting Skills in Imagination
(
Poster
)
>
link
SlidesLive Video |
Pietro Mazzaglia 路 Tim Verbelen 路 Bart Dhoedt 路 Alexandre Lacoste 路 Sai Rajeswar Mudumba 馃敆 |
-
|
CORL: Research-oriented Deep Offline Reinforcement Learning Library
(
Poster
)
>
link
SlidesLive Video |
Denis Tarasov 路 Alexander Nikulin 路 Dmitry Akimov 路 Vladislav Kurenkov 路 Sergey Kolesnikov 馃敆 |
-
|
Q-Ensemble for Offline RL: Don't Scale the Ensemble, Scale the Batch Size
(
Poster
)
>
link
SlidesLive Video |
Alexander Nikulin 路 Vladislav Kurenkov 路 Denis Tarasov 路 Dmitry Akimov 路 Sergey Kolesnikov 馃敆 |
-
|
Offline Reinforcement Learning for Customizable Visual Navigation ( Poster ) > link | Dhruv Shah 路 Arjun Bhorkar 路 Hrishit Leen 路 Ilya Kostrikov 路 Nicholas Rhinehart 路 Sergey Levine 馃敆 |
-
|
Efficient Planning in a Compact Latent Action Space ( Poster ) > link | zhengyao Jiang 路 Tianjun Zhang 路 Michael Janner 路 Yueying (Lisa) Li 路 Tim Rockt盲schel 路 Edward Grefenstette 路 Yuandong Tian 馃敆 |
-
|
User-Interactive Offline Reinforcement Learning
(
Poster
)
>
link
SlidesLive Video |
Phillip Swazinna 路 Steffen Udluft 路 Thomas Runkler 馃敆 |
-
|
Does Zero-Shot Reinforcement Learning Exist? ( Poster ) > link | Ahmed Touati 路 J茅r茅my Rapin 路 Yann Ollivier 馃敆 |
-
|
State Advantage Weighting for Offline RL ( Poster ) > link | Jiafei Lyu 路 aicheng Gong 路 Le Wan 路 Zongqing Lu 路 Xiu Li 馃敆 |
-
|
Optimal Transport for Offline Imitation Learning
(
Poster
)
>
link
SlidesLive Video |
Yicheng Luo 路 zhengyao Jiang 路 Samuel Cohen 路 Edward Grefenstette 路 Marc Deisenroth 馃敆 |
-
|
Control Graph as Unified IO for Morphology-Task Generalization
(
Poster
)
>
link
SlidesLive Video |
Hiroki Furuta 路 Yusuke Iwasawa 路 Yutaka Matsuo 路 Shixiang (Shane) Gu 馃敆 |
-
|
Mutual Information Regularized Offline Reinforcement Learning ( Poster ) > link | Xiao Ma 路 Bingyi Kang 路 Zhongwen Xu 路 Min Lin 路 Shuicheng Yan 馃敆 |
-
|
Uncertainty-Driven Pessimistic Q-Ensemble for Offline-to-Online Reinforcement Learning ( Poster ) > link | Ingook Jang 路 Seonghyun Kim 馃敆 |
-
|
Offline Robot Reinforcement Learning with Uncertainty-Guided Human Expert Sampling ( Poster ) > link | Ashish Kumar 路 Ilya Kuzovkin 馃敆 |
-
|
Near-Optimal Deployment Efficiency in Reward-Free Reinforcement Learning with Linear Function Approximation ( Poster ) > link | Dan Qiao 路 Yu-Xiang Wang 馃敆 |
-
|
Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training
(
Poster
)
>
link
SlidesLive Video |
Jason Yecheng Ma 路 Shagun Sodhani 路 Dinesh Jayaraman 路 Osbert Bastani 路 Vikash Kumar 路 Amy Zhang 馃敆 |
-
|
Imitation from Observation With Bootstrapped Contrastive Learning ( Poster ) > link | Medric Sonwa 路 Johanna Hansen 路 Eugene Belilovsky 馃敆 |
-
|
Provable Benefits of Representational Transfer in Reinforcement Learning ( Poster ) > link | Alekh Agarwal 路 Yuda Song 路 Kaiwen Wang 路 Mengdi Wang 路 Wen Sun 路 Xuezhou Zhang 馃敆 |
-
|
A Connection between One-Step Regularization and Critic Regularization in Reinforcement Learning ( Poster ) > link | Benjamin Eysenbach 路 Matthieu Geist 路 Sergey Levine 路 Russ Salakhutdinov 馃敆 |
-
|
Offline evaluation in RL: soft stability weighting to combine fitted Q-learning and model-based methods ( Poster ) > link | Briton Park 路 Xian Wu 路 Bin Yu 路 Angela Zhou 馃敆 |
-
|
Using Confounded Data in Offline RL ( Poster ) > link | Maxime Gasse 路 Damien GRASSET 路 Guillaume Gaudron 路 Pierre-Yves Oudeyer 馃敆 |
-
|
Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement
(
Poster
)
>
link
SlidesLive Video |
Michael Chang 路 Alyssa L Dayan 路 Franziska Meier 路 Tom Griffiths 路 Sergey Levine 路 Amy Zhang 馃敆 |
-
|
Visual Backtracking Teleoperation: A Data Collection Protocol for Offline Image-Based RL
(
Poster
)
>
link
SlidesLive Video |
David Brandfonbrener 路 Stephen Tu 路 Avi Singh 路 Stefan Welker 路 Chad Boodoo 路 Nikolai Matni 路 Jake Varley 馃敆 |
-
|
Towards Data-Driven Offline Simulations for Online Reinforcement Learning ( Poster ) > link | Shengpu Tang 路 Felipe Vieira Frujeri 路 Dipendra Misra 路 Alex Lamb 路 John Langford 路 Paul Mineiro 路 Sebastian Kochman 馃敆 |
-
|
Scaling Marginalized Importance Sampling to High-Dimensional State-Spaces via State Abstraction ( Poster ) > link | Brahma Pavse 路 Josiah Hanna 馃敆 |
-
|
Benchmarking Offline Reinforcement Learning Algorithms for E-Commerce Order Fraud Evaluation ( Poster ) > link | Soysal Degirmenci 路 Christopher S Jones 馃敆 |
-
|
Sparse Q-Learning: Offline Reinforcement Learning with Implicit Value Regularization ( Poster ) > link | Haoran Xu 路 Li Jiang 路 Li Jianxiong 路 Zhuoran Yang 路 Zhaoran Wang 路 Xianyuan Zhan 馃敆 |