Skip to yearly menu bar Skip to main content


Incremental Variational Sparse Gaussian Process Regression

Ching-An Cheng · Byron Boots

Area 5+6+7+8 #12

Keywords: [ Gaussian Processes ] [ Variational Inference ] [ (Other) Regression ] [ Large Scale Learning and Big Data ] [ Online Learning ] [ Kernel Methods ]


Recent work on scaling up Gaussian process regression (GPR) to large datasets has primarily focused on sparse GPR, which leverages a small set of basis functions to approximate the full Gaussian process during inference. However, the majority of these approaches are batch methods that operate on the entire training dataset at once, precluding the use of datasets that are streaming or too large to fit into memory. Although previous work has considered incrementally solving variational sparse GPR, most algorithms fail to update the basis functions and therefore perform suboptimally. We propose a novel incremental learning algorithm for variational sparse GPR based on stochastic mirror ascent of probability densities in reproducing kernel Hilbert space. This new formulation allows our algorithm to update basis functions online in accordance with the manifold structure of probability densities for fast convergence. We conduct several experiments and show that our proposed approach achieves better empirical performance in terms of prediction error than the recent state-of-the-art incremental solutions to variational sparse GPR.

Live content is unavailable. Log in and register to view live content