Timezone: »

 
SEM2: Enhance Sample Efficiency and Robustness of End-to-end Urban Autonomous Driving via Semantic Masked World Model
Zeyu Gao · Yao Mu · Ruoyan Shen · Chen Chen · Yangang Ren · Jianyu Chen · Shengbo Li · Ping Luo · Yanfeng Lu

@
in Deep Reinforcement Learning Workshop »
Event URL: https://openreview.net/forum?id=W0g6TJFeKY »

End-to-end autonomous driving provides a feasible way to automatically maximize overall driving system performance by directly mapping the raw pixels from a front-facing camera to control signals. Recent advanced methods construct a latent world model to map the high dimensional observations into compact latent space. However, the latent states embedded by the world model proposed in previous works may contain a large amount of task-irrelevant information, resulting in low sampling efficiency and poor robustness to input perturbations. Meanwhile, the training data distribution is usually unbalanced, and the learned policy is hard to cope with the corner cases during the driving process. To solve the above challenges, we present a semantic masked recurrent world model (SEM2), which introduces a latent filter to extract key task-relevant features and reconstruct a semantic mask via the filtered features, and is trained with a multi-source data sampler, which aggregates common data and multiple corner case data in a single batch, to balance the data distribution. Extensive experiments on CARLA show that our method outperforms the state-of-the-art approaches in terms of sample efficiency and robustness to input permutations.

[ Poster]  [ OpenReview]  [ Topia] 

Author Information

Zeyu Gao (Harbin Institute of Technology)
Yao Mu (The University of Hong Kong)

I am currently a Ph.D. Candidate of Computer Science at the University of Hong Kong. I graduated with a Master Degree from Tsinghua University in June 2021. My research interests include Reinforcement Learning, Representation Learning, Autonomous Driving, Optimal Control, and Computer Vision.

Ruoyan Shen (Harbin Institute of Technology)
Chen Chen
Yangang Ren
Jianyu Chen (Tsinghua University)
Shengbo Li (Tsinghua University, Tsinghua University)
Ping Luo (The University of Hong Kong)
Yanfeng Lu (Institute of automation, Chinese Academy of Sciences)

More from the Same Authors