Timezone: »

 
Poster
Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learning
Junsu Kim · Younggyo Seo · Jinwoo Shin

Thu Dec 09 12:30 AM -- 02:00 AM (PST) @
in Poster Session 5 »

Goal-conditioned hierarchical reinforcement learning (HRL) has shown promising results for solving complex and long-horizon RL tasks. However, the action space of high-level policy in the goal-conditioned HRL is often large, so it results in poor exploration, leading to inefficiency in training. In this paper, we present HIerarchical reinforcement learning Guided by Landmarks (HIGL), a novel framework for training a high-level policy with a reduced action space guided by landmarks, i.e., promising states to explore. The key component of HIGL is twofold: (a) sampling landmarks that are informative for exploration and (b) encouraging the high level policy to generate a subgoal towards a selected landmark. For (a), we consider two criteria: coverage of the entire visited state space (i.e., dispersion of states) and novelty of states (i.e., prediction error of a state). For (b), we select a landmark as the very first landmark in the shortest path in a graph whose nodes are landmarks. Our experiments demonstrate that our framework outperforms prior-arts across a variety of control tasks, thanks to efficient exploration guided by landmarks.

Keywords: Reinforcement Learning and Planning · Graph Learning

Author Information

Junsu Kim (KAIST)
Younggyo Seo (KAIST)
Jinwoo Shin (KAIST)

More from the Same Authors