Timezone: »

 
Poster
Generating Long-term Trajectories Using Deep Hierarchical Networks
Stephan Zheng · Yisong Yue · Patrick Lucey

Mon Dec 05 09:00 AM -- 12:30 PM (PST) @ Area 5+6+7+8 #72
in Monday Posters »

We study the problem of modeling spatiotemporal trajectories over long time horizons using expert demonstrations. For instance, in sports, agents often choose action sequences with long-term goals in mind, such as achieving a certain strategic position. Conventional policy learning approaches, such as those based on Markov decision processes, generally fail at learning cohesive long-term behavior in such high-dimensional state spaces, and are only effective when fairly myopic decision-making yields the desired behavior. The key difficulty is that conventional models are ``single-scale'' and only learn a single state-action policy. We instead propose a hierarchical policy class that automatically reasons about both long-term and short-term goals, which we instantiate as a hierarchical neural network. We showcase our approach in a case study on learning to imitate demonstrated basketball trajectories, and show that it generates significantly more realistic trajectories compared to non-hierarchical baselines as judged by professional sports analysts.

Keywords: Deep Learning or Neural Networks · Reinforcement Learning Algorithms
[ PDF] 

Author Information

Stephan Zheng (Caltech)
Yisong Yue (Caltech)
Patrick Lucey (Stats)

More from the Same Authors