Timezone: »

 
Poster
Offline Model-based Adaptable Policy Learning
Xiong-Hui Chen · Yang Yu · Qingyang Li · Fan-Ming Luo · Zhiwei Qin · Wenjie Shang · Jieping Ye

Tue Dec 07 08:30 AM -- 10:00 AM (PST) @
in Poster Session 1 »

In reinforcement learning, a promising direction to avoid online trial-and-error costs is learning from an offline dataset. Current offline reinforcement learning methods commonly learn in the policy space constrained to in-support regions by the offline dataset, in order to ensure the robustness of the outcome policies. Such constraints, however, also limit the potential of the outcome policies. In this paper, to release the potential of offline policy learning, we investigate the decision-making problems in out-of-support regions directly and propose offline Model-based Adaptable Policy LEarning (MAPLE). By this approach, instead of learning in in-support regions, we learn an adaptable policy that can adapt its behavior in out-of-support regions when deployed. We conduct experiments on MuJoCo controlling tasks with offline datasets. The results show that the proposed method can make robust decisions in out-of-support regions and achieve better performance than SOTA algorithms.

Keywords: Reinforcement Learning and Planning · Robustness

Author Information

Xiong-Hui Chen (Nanjing University)
Yang Yu (Nanjing University)
Qingyang Li (Didi AI Labs)

https://scholar.google.com/citations?user=Pd50HpAAAAAJ&hl=en

Fan-Ming Luo (Nanjing University)
Zhiwei Qin (Didi Research America)
Wenjie Shang (Nanjing University)
Jieping Ye (University of Michigan)

More from the Same Authors