Fantasizing with Dual GPs in Bayesian Optimization and Active Learning

Gaussian Processes (GPs) are popular surrogate models for sequential decision making tasks such as Bayesian Optimization and Active Learning. Such frameworks often exploit well-known cheap methods for conditioning a GP posterior on new data. However, these standard methods cannot be applied to popular but more complex models such as sparse GPs or for non-conjugate likelihoods due to a lack of such update formulas. Using an alternative sparse Dual GP parameterization, we show that these costly computations can be avoided, whilst enjoying one-step updates for non-Gaussian likelihoods. The resulting algorithms allow for cheap batch formulations that work with most acquisition functions.