Timezone: »

 
Poster
Regret Bounds for Learning State Representations in Reinforcement Learning
Ronald Ortner · Matteo Pirotta · Alessandro Lazaric · Ronan Fruit · Odalric-Ambrym Maillard

Tue Dec 10 10:45 AM -- 12:45 PM (PST) @ East Exhibition Hall B + C #196
in Reinforcement Learning and Planning -- Exploration »

We consider the problem of online reinforcement learning when several state representations (mapping histories to a discrete state space) are available to the learning agent. At least one of these representations is assumed to induce a Markov decision process (MDP), and the performance of the agent is measured in terms of cumulative regret against the optimal policy giving the highest average reward in this MDP representation. We propose an algorithm (UCB-MS) with O(sqrt(T)) regret in any communicating Markov decision process. The regret bound shows that UCB-MS automatically adapts to the Markov model. This improves over the currently known best results in the literature that gave regret bounds of order O(T^(2/3)).

Keywords: Markov Decision Processes · Computational Complexity · Reinforcement Learning and Planning · Algorithms -> Model Selection and Structure Learning; Algorithms -> Representation Learning; Theory
[ Paper]  [ Slides] 

Author Information

Ronald Ortner (Montanuniversitaet Leoben)
Matteo Pirotta (Facebook AI Research)
Alessandro Lazaric (Facebook Artificial Intelligence Research)
Ronan Fruit (Inria Lille)
Odalric-Ambrym Maillard (INRIA)

More from the Same Authors