Timezone: »

 
Poster
Error Bounds of Imitating Policies and Environments
Tian Xu · Ziniu Li · Yang Yu

Tue Dec 08 09:00 PM -- 11:00 PM (PST) @ Poster Session 2 #605
in Poster Session 3 »

Imitation learning trains a policy by mimicking expert demonstrations. Various imitation methods were proposed and empirically evaluated, meanwhile, their theoretical understanding needs further studies. In this paper, we firstly analyze the value gap between the expert policy and imitated policies by two imitation methods, behavioral cloning and generative adversarial imitation. The results support that generative adversarial imitation can reduce the compounding errors compared to behavioral cloning, and thus has a better sample complexity. Noticed that by considering the environment transition model as a dual agent, imitation learning can also be used to learn the environment model. Therefore, based on the bounds of imitating policies, we further analyze the performance of imitating environments. The results show that environment models can be more effectively imitated by generative adversarial imitation than behavioral cloning, suggesting a novel application of adversarial imitation for model-based reinforcement learning. We hope these results could inspire future advances in imitation learning and model-based reinforcement learning.

Author Information

Tian Xu (Nanjing University)
Ziniu Li (Nanjing University)
Yang Yu (Nanjing University)

More from the Same Authors