Timezone: »

 
Poster
Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learning
Zhecheng Yuan · Zhengrong Xue · Bo Yuan · Xueqian Wang · YI WU · Yang Gao · Huazhe Xu

@

Learning generalizable policies that can adapt to unseen environments remains challenging in visual Reinforcement Learning (RL). Existing approaches try to acquire a robust representation via diversifying the appearances of in-domain observations for better generalization. Limited by the specific observations of the environment, these methods ignore the possibility of exploring diverse real-world image datasets. In this paper, we investigate how a visual RL agent would benefit from the off-the-shelf visual representations. Surprisingly, we find that the early layers in an ImageNet pre-trained ResNet model could provide rather generalizable representations for visual RL. Hence, we propose Pre-trained Image Encoder for Generalizable visual reinforcement learning (PIE-G), a simple yet effective framework that can generalize to the unseen visual scenarios in a zero-shot manner. Extensive experiments are conducted on DMControl Generalization Benchmark, DMControl Manipulation Tasks, Drawer World, and CARLA to verify the effectiveness of PIE-G. Empirical evidence suggests PIE-G improves sample efficiency and significantly outperforms previous state-of-the-art methods in terms of generalization performance. In particular, PIE-G boasts a 55% generalization performance gain on average in the challenging video background setting. Project Page: https://sites.google.com/view/pie-g/home.

Keywords: pre-trained models · Visual RL · Generealization
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Zhecheng Yuan (Tsinghua University, Tsinghua University)
Zhengrong Xue (Shanghai Jiao Tong University)
Bo Yuan (Qianyuan Institute of Sciences)
Xueqian Wang (Tsinghua University, Tsinghua University)
YI WU (UC Berkeley)
Yang Gao (Tsinghua University)
Huazhe Xu (Tsinghua University)

More from the Same Authors