Welcome message, presentation of the speakers and brief overview of the talk.

We revise the main components of Bayesian inference underpinning the treatment of the Gaussian process. Adopting an illustrative approach, we review statistical models, the approaches to learning the model parameters and the effect of the parameters’ cardinality.

We motivate the use of Gaussian random variables (RVs) from conceptual and practical perspectives. We examine the main properties of the Gaussian model and show how Gaussian RVs can be combined to construct Gaussian vectors

We study the covariance function and its marginalisation properties to motivate the existence of an infinite collection of Gaussian RVs, that is, the Gaussian process. We then present the prior and posterior GP in an illustrative manner.

Building on the use of a Gaussian likelihood, we present a learning objective and refer to the problem of choosing a kernel and optimising its parameters. We revise different kernel functions and their properties.

We show how to implement a vanilla GP from scratch, we refer to the particulars of coding the kernel, the likelihood function and the train() function. We apply this minimal GP toolbox to a couple of datasets to illustrate the GP’s ease of use and modelling abilities

We expand the applicability of GP models by introducing non-Gaussian likelihoods. We show why inference becomes intractable and how it can be approximated. Via the use of illustrations, we focus on the binary classification learning scenario, warping functions, and heteroscedastic models.

Standard GP presents scalability issues, thus, in this section we present how sparse approximations enable the use of GPs for large datasets. We consider, in particular, the variational approach based on inducing variables, which results in tractable approximations for performing predictions and model selection via optimization of the evidence lower bound (ELBO).

We highlight some recent advances to kernel design that enhance the use of GP models in complex data settings. From automatic composition of simple kernels to deep kernel learning and (graph) convolutional kernels, the works revised in this section indicate how fast-paced the research on kernel design is.

We consider the unsupervised setting, more specifically, the task of nonlinear dimensionality reduction. We summarize the GP latent variable model (GPLVM) and indicate how it can be used as a building block for other generative models, such as the deep GP and other flexible probabilistic models.

We motivate the need for MOGPs and show how they can be constructed by mixing independent GPs. Then, we revise standard approaches to covariance design and their implications. We conclude this section with real-world examples.

Final remarks: the take-home message, what has been left out of the tutorial and proposed avenues for further study