Timezone: »

The Art of Gaussian Processes: Classical and Contemporary
César Lincoln C Mattos · Felipe Tobar

Mon Dec 06 09:00 AM -- 01:47 PM (PST) @ None

Gaussian processes (GP) are Bayesian nonparametric models for continuous functions which allow for uncertainty quantification, interpretability, and the incorporation of expert knowledge. The theory and practice of GPs have flourished in the last decade, where researchers have looked into the expressiveness and efficiency of GP-based models and practitioners have applied them to a plethora of disciplines. This tutorial presents both the foundational theory and modern developments of data modelling using GPs, following step by step intuitions, illustrations and real-world examples. The tutorial will start with emphasis on the building blocks of the GP model, to then move onto the choice of the kernel function, cost-effective training strategies and non-Gaussian extensions. The second part of the tutorial will showcase more recent advances, such as latent variable models, deep GPs, current trends on kernel design and connections between GPs and deep neural networks. We hope that this exhibition, featuring classic and contemporary GP works, inspires attendees to incorporate GPs into their applications and motivates them to continue learning and contributing to the current developments in the field.

Mon 9:00 a.m. - 9:05 a.m.

Welcome message, presentation of the speakers and brief overview of the talk.

Felipe Tobar, César Lincoln C Mattos
Mon 9:05 a.m. - 9:20 a.m.

We revise the main components of Bayesian inference underpinning the treatment of the Gaussian process. Adopting an illustrative approach, we review statistical models, the approaches to learning the model parameters and the effect of the parameters’ cardinality.

Felipe Tobar
Mon 9:20 a.m. - 9:35 a.m.

We motivate the use of Gaussian random variables (RVs) from conceptual and practical perspectives. We examine the main properties of the Gaussian model and show how Gaussian RVs can be combined to construct Gaussian vectors

Felipe Tobar
Mon 9:35 a.m. - 9:50 a.m.
Mon 9:50 a.m. - 10:10 a.m.

We study the covariance function and its marginalisation properties to motivate the existence of an infinite collection of Gaussian RVs, that is, the Gaussian process. We then present the prior and posterior GP in an illustrative manner.

Felipe Tobar
Mon 10:10 a.m. - 10:25 a.m.

Building on the use of a Gaussian likelihood, we present a learning objective and refer to the problem of choosing a kernel and optimising its parameters. We revise different kernel functions and their properties.

Felipe Tobar
Mon 10:25 a.m. - 10:35 a.m.

We show how to implement a vanilla GP from scratch, we refer to the particulars of coding the kernel, the likelihood function and the train() function. We apply this minimal GP toolbox to a couple of datasets to illustrate the GP’s ease of use and modelling abilities

Felipe Tobar
Mon 10:35 a.m. - 10:50 a.m.
Mon 10:50 a.m. - 11:00 a.m.
Mon 11:00 a.m. - 11:15 a.m.

We expand the applicability of GP models by introducing non-Gaussian likelihoods. We show why inference becomes intractable and how it can be approximated. Via the use of illustrations, we focus on the binary classification learning scenario, warping functions, and heteroscedastic models.

César Lincoln C Mattos
Mon 11:15 a.m. - 11:37 a.m.

Standard GP presents scalability issues, thus, in this section we present how sparse approximations enable the use of GPs for large datasets. We consider, in particular, the variational approach based on inducing variables, which results in tractable approximations for performing predictions and model selection via optimization of the evidence lower bound (ELBO).

César Lincoln C Mattos
Mon 11:37 a.m. - 11:52 a.m.
Mon 11:52 a.m. - 12:05 p.m.

We highlight some recent advances to kernel design that enhance the use of GP models in complex data settings. From automatic composition of simple kernels to deep kernel learning and (graph) convolutional kernels, the works revised in this section indicate how fast-paced the research on kernel design is.

César Lincoln C Mattos
Mon 12:05 p.m. - 12:27 p.m.

We consider the unsupervised setting, more specifically, the task of nonlinear dimensionality reduction. We summarize the GP latent variable model (GPLVM) and indicate how it can be used as a building block for other generative models, such as the deep GP and other flexible probabilistic models.

César Lincoln C Mattos
Mon 12:27 p.m. - 12:42 p.m.

We motivate the need for MOGPs and show how they can be constructed by mixing independent GPs. Then, we revise standard approaches to covariance design and their implications. We conclude this section with real-world examples.

Felipe Tobar
Mon 12:47 p.m. - 1:47 p.m.

Final remarks: the take-home message, what has been left out of the tutorial and proposed avenues for further study

Felipe Tobar, César Lincoln C Mattos
Mon 12:47 p.m. - 1:00 p.m.

Author Information

César Lincoln C Mattos (Federal University of Ceará)

César Lincoln Cavalcante Mattos is an associate professor at the Department of Computer Science, at Federal University of Ceará (UFC), Brazil. He is also an associate researcher at the Logics and Artificial Intelligence Group (LOGIA). He has research interests in the broad fields of machine learning and probabilistic modeling, such as Gaussian processes, deep (probabilistic) learning, approximate inference and system identification. He has been applying learning methods in several research and development collaborations in areas such as dynamical system modeling, health risk analysis, software repository mining and anomaly detection.

Felipe Tobar (Universidad de Chile)

Felipe Tobar is an Assistant Professor at the Data & AI Initiative at Universidad de Chile. He holds Researcher positions at the Center for Mathematical Modeling and the Advanced Center for Electrical Engineering. Felipe received the BSc/MSc degrees in Electrical Engineering (U. de Chile, 2010) and a PhD in Signal Processing (Imperial College London, 2014), and he was an Associate Researcher in Machine Learning at the University of Cambridge (2014-2015). Felipe teaches Statistics and Machine Learning courses at undergraduate, graduate and professional levels. His research interests lie in the interface between Machine Learning and Statistical Signal Processing, including Gaussian processes, spectral estimation, approximate inference, Bayesian nonparametrics, and optimal transport.

More from the Same Authors