Timezone: »
Inverse reinforcement learning (IRL) seeks to infer a cost function that explains the underlying goals and preferences of expert demonstrations. This paper presents Receding Horizon Inverse Reinforcement Learning (RHIRL), a new IRL algorithm for high-dimensional, noisy, continuous systems with black-box dynamic models. RHIRL addresses two key challenges of IRL: scalability and robustness. To handle high-dimensional continuous systems, RHIRL matches the induced optimal trajectories with expert demonstrations locally in a receding horizon manner and stitches'' together the local solutions to learn the cost; it thereby avoids the
curse of dimensionality''. This contrasts sharply with earlier algorithms that match with expert demonstrations globally over the entire high-dimensional state space. To be robust against imperfect expert demonstrations and control noise, RHIRL learns a state-dependent cost function ``disentangled'' from system dynamics under mild conditions. Experiments on benchmark tasks show that RHIRL outperforms several leading IRL algorithms in most instances. We also prove that the cumulative error of RHIRL grows linearly with the task duration.
Author Information
Yiqing Xu (national university of singaore, National University of Singapore)
Wei Gao (NUS)
David Hsu (National University of Singapore)
More from the Same Authors
-
2018 Workshop: Reinforcement Learning under Partial Observability »
Joni Pajarinen · Chris Amato · Pascal Poupart · David Hsu -
2017 Poster: QMDP-Net: Deep Learning for Planning under Partial Observability »
Peter Karkus · David Hsu · Wee Sun Lee -
2015 Poster: Adaptive Stochastic Optimization: From Sets to Paths »
Zhan Wei Lim · David Hsu · Wee Sun Lee -
2013 Poster: DESPOT: Online POMDP Planning with Regularization »
Adhiraj Somani · Nan Ye · David Hsu · Wee Sun Lee -
2011 Poster: Monte Carlo Value Iteration with Macro-Actions »
Zhan Wei Lim · David Hsu · Wee Sun Lee -
2007 Spotlight: What makes some POMDP problems easy to approximate? »
David Hsu · Wee Sun Lee · Nan Rong -
2007 Poster: What makes some POMDP problems easy to approximate? »
David Hsu · Wee Sun Lee · Nan Rong