Poster
Completing State Representations using Spectral Learning
Nan Jiang · Alex Kulesza · Satinder Singh

Tue Dec 4th 05:00 -- 07:00 PM @ Room 517 AB #168
in Tue Poster Session B »

A central problem in dynamical system modeling is state discovery—that is, finding a compact summary of the past that captures the information needed to predict the future. Predictive State Representations (PSRs) enable clever spectral methods for state discovery; however, while consistent in the limit of infinite data, these methods often suffer from poor performance in the low data regime. In this paper we develop a novel algorithm for incorporating domain knowledge, in the form of an imperfect state representation, as side information to speed spectral learning for PSRs. We prove theoretical results characterizing the relevance of a user-provided state representation, and design spectral algorithms that can take advantage of a relevant representation. Our algorithm utilizes principal angles to extract the relevant components of the representation, and is robust to misspecification. Empirical evaluation on synthetic HMMs, an aircraft identification domain, and a gene splice dataset shows that, even with weak domain knowledge, the algorithm can significantly outperform standard PSR learning.

Keywords: Spectral Methods · Reinforcement Learning

Author Information

Nan Jiang (University of Illinois at Urbana-Champaign)
Alex Kulesza (Google)
Satinder Singh (University of Michigan)

More from the Same Authors