Timezone: »
The purpose of multi-task reinforcement learning (MTRL) is to train a single policy that can be applied to a set of different tasks. Sharing parameters allows us to take advantage of the similarities among tasks. However, the gaps between contents and difficulties of different tasks bring us challenges on both which tasks should share the parameters and what parameters should be shared, as well as the optimization challenges due to parameter sharing. In this work, we introduce a parameter-compositional approach (PaCo) as an attempt to address these challenges. In this framework, a policy subspace represented by a set of parameters is learned. Policies for all the single tasks lie in this subspace and can be composed by interpolating with the learned set. It allows not only flexible parameter sharing, but also a natural way to improve training.We demonstrate the state-of-the-art performance on Meta-World benchmarks, verifying the effectiveness of the proposed approach.
Author Information
Lingfeng Sun (UC Berkeley)
Haichao Zhang (Horizon Robotics)
Wei Xu (Horizon Robotics)
Masayoshi TOMIZUKA (MSC Lab)
More from the Same Authors
-
2021 : Hierarchical Adaptable and Transferable Networks (HATN) for Driving Behavior Prediction »
Letian Wang · Yeping Hu · Liting Sun · Wei Zhan · Masayoshi TOMIZUKA · Changliu Liu -
2021 : Causal-based Time Series Domain Generalization for Vehicle Intention Prediction »
Yeping Hu · Xiaogang Jia · Masayoshi TOMIZUKA · Wei Zhan -
2022 Poster: Towards Safe Reinforcement Learning with a Safety Editor Policy »
Haonan Yu · Wei Xu · Haichao Zhang -
2022 Poster: Society of Agents: Regret Bounds of Concurrent Thompson Sampling »
Yan Chen · Perry Dong · Qinxun Bai · Maria Dimakopoulou · Wei Xu · Zhengyuan Zhou -
2021 Poster: TAAC: Temporally Abstract Actor-Critic for Continuous Control »
Haonan Yu · Wei Xu · Haichao Zhang -
2019 Poster: Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Training »
Haichao Zhang · Jianyu Wang -
2018 Poster: Adversarial Text Generation via Feature-Mover's Distance »
Liqun Chen · Shuyang Dai · Chenyang Tao · Haichao Zhang · Zhe Gan · Dinghan Shen · Yizhe Zhang · Guoyin Wang · Dinghan Shen · Lawrence Carin -
2017 : Break + Poster (1) »
Devendra Singh Chaplot · CHIH-YAO MA · Simon Brodeur · Eri Matsuo · Ichiro Kobayashi · Seitaro Shinagawa · Koichiro Yoshino · Yuhong Guo · Ben Murdoch · Kanthashree Mysore Sathyendra · Daniel Ricks · Haichao Zhang · Joshua Peterson · Li Zhang · Mircea Mironenco · Peter Anderson · Mark Johnson · Kang Min Yoo · Guntis Barzdins · Ahmed H Zaidi · Martin Andrews · Sam Witteveen · SUBBAREDDY OOTA · Prashanth Vijayaraghavan · Ke Wang · Yan Zhu · Renars Liepins · Max Quinn · Amit Raj · Vincent Cartillier · Eric Chu · Ethan Caballero · Fritz Obermeyer -
2014 Poster: Scale Adaptive Blind Deblurring »
Haichao Zhang · Jianchao Yang -
2013 Poster: Non-Uniform Camera Shake Removal Using a Spatially-Adaptive Sparse Penalty »
Haichao Zhang · David Wipf -
2013 Oral: Non-Uniform Camera Shake Removal Using a Spatially-Adaptive Sparse Penalty »
Haichao Zhang · David Wipf