Timezone: »

 
Poster
Exponential Family Model-Based Reinforcement Learning via Score Matching
Gene Li · Junbo Li · Anmol Kabra · Nati Srebro · Zhaoran Wang · Zhuoran Yang

Tue Nov 29 09:00 AM -- 11:00 AM (PST) @ Hall J #343
in Poster Session 1 »
We propose an optimistic model-based algorithm, dubbed SMRL, for finite-horizon episodic reinforcement learning (RL) when the transition model is specified by exponential family distributions with $d$ parameters and the reward is bounded and known. SMRL uses score matching, an unnormalized density estimation technique that enables efficient estimation of the model parameter by ridge regression. Under standard regularity assumptions, SMRL achieves $\tilde O(d\sqrt{H^3T})$ online regret, where $H$ is the length of each episode and $T$ is the total number of interactions (ignoring polynomial dependence on structural scale parameters).
Keywords: Model-Based RL · online reinforcement learning · exponential family
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Gene Li (Toyota Technological Institute at Chicago)
Junbo Li (University of California, Santa Cruz)
Anmol Kabra (TTIC)
Nati Srebro (TTI-Chicago)
Zhaoran Wang (Northwestern University)
Zhuoran Yang (Yale University)

More from the Same Authors