Timezone: »

 
Poster
Curriculum Design for Teaching via Demonstrations: Theory and Applications
Gaurav Yengera · Rati Devidze · Parameswaran Kamalaruban · Adish Singla

Fri Dec 10 08:30 AM -- 10:00 AM (PST) @
in Poster Session 8 »

We consider the problem of teaching via demonstrations in sequential decision-making settings. In particular, we study how to design a personalized curriculum over demonstrations to speed up the learner's convergence. We provide a unified curriculum strategy for two popular learner models: Maximum Causal Entropy Inverse Reinforcement Learning (MaxEnt-IRL) and Cross-Entropy Behavioral Cloning (CrossEnt-BC). Our unified strategy induces a ranking over demonstrations based on a notion of difficulty scores computed w.r.t. the teacher's optimal policy and the learner's current policy. Compared to the state of the art, our strategy doesn't require access to the learner's internal dynamics and still enjoys similar convergence guarantees under mild technical conditions. Furthermore, we adapt our curriculum strategy to the setting where no teacher agent is present using task-specific difficulty scores. Experiments on a synthetic car driving environment and navigation-based environments demonstrate the effectiveness of our curriculum strategy.

Keywords: Theory · Reinforcement Learning and Planning

Author Information

Gaurav Yengera (Amazon)

Computer Science master's student at Saarland University, Germany. Presently carrying out research in reinforcement learning at the Max Planck Institute for Software Systems, with prior research experience in deep learning for healthcare. Previously graduated with a bachelor's degree in electrical engineering from the Indian Institute of Technology, Varanasi.

Rati Devidze (MPI-SWS)
Parameswaran Kamalaruban (EPFL)
Adish Singla (MPI-SWS)

More from the Same Authors