Timezone: »

 
Poster
Continuous-time Value Function Approximation in Reproducing Kernel Hilbert Spaces
Motoya Ohnishi · Masahiro Yukawa · Mikael Johansson · Masashi Sugiyama

Wed Dec 05 02:00 PM -- 04:00 PM (PST) @ Room 210 #98
in Wed Poster Session B »

Motivated by the success of reinforcement learning (RL) for discrete-time tasks such as AlphaGo and Atari games, there has been a recent surge of interest in using RL for continuous-time control of physical systems (cf. many challenging tasks in OpenAI Gym and DeepMind Control Suite). Since discretization of time is susceptible to error, it is methodologically more desirable to handle the system dynamics directly in continuous time. However, very few techniques exist for continuous-time RL and they lack flexibility in value function approximation. In this paper, we propose a novel framework for model-based continuous-time value function approximation in reproducing kernel Hilbert spaces. The resulting framework is so flexible that it can accommodate any kind of kernel-based approach, such as Gaussian processes and kernel adaptive filters, and it allows us to handle uncertainties and nonstationarity without prior knowledge about the environment or what basis functions to employ. We demonstrate the validity of the presented framework through experiments.

Keywords: Kernel Methods · Model-Based RL

Author Information

Motoya Ohnishi (Keio University/KTH Royal Institute of Technology/RIKEN)

M. Ohnishi received the B.S. degree in electronics and electrical engineering from Keio University, Tokyo, Japan, in 2016. He is currently working toward the M.S. degrees both in electronics and electrical engineering with Keio University, and in electrical engineering with the KTH Royal Institute of Technology, Stockholm, Sweden. He was a Research Assistant with the Department of Automatic Control, KTH Royal Institute of Technology, and was a Visiting Researcher with the GRITS Lab, Georgia Institute of Technology, Atlanta, GA, USA, in 2017. He is currently a Research Assistant with the RIKEN Center for Advanced Intelligence Project, Tokyo, Japan. His research interests include mathematical signal processing, machine learning, and robotics.

Masahiro Yukawa (Keio University)
Mikael Johansson (KTH - Royal Institute of Technology)
Masashi Sugiyama (RIKEN / University of Tokyo)

More from the Same Authors