Timezone: »
Conditional Density Estimation (CDE) models deal with estimating conditional distributions. The conditions imposed on the distribution are the inputs of the model. CDE is a challenging task as there is a fundamental trade-off between model complexity, representational capacity and overfitting. In this work, we propose to extend the model's input with latent variables and use Gaussian processes (GP) to map this augmented input onto samples from the conditional distribution. Our Bayesian approach allows for the modeling of small datasets, but we also provide the machinery for it to be applied to big data using stochastic variational inference. Our approach can be used to model densities even in sparse data regions, and allows for sharing learned structure between conditions. We illustrate the effectiveness and wide-reaching applicability of our model on a variety of real-world problems, such as spatio-temporal density estimation of taxi drop-offs, non-Gaussian noise modeling, and few-shot learning on omniglot images.
Author Information
Vincent Dutordoir (PROWLER.io)
Hugh Salimbeni (Imperial College London)
James Hensman (PROWLER.io)
Marc Deisenroth (Imperial College London)

Professor Marc Deisenroth is the DeepMind Chair in Artificial Intelligence at University College London and the Deputy Director of UCL's Centre for Artificial Intelligence. He also holds a visiting faculty position at the University of Johannesburg and Imperial College London. Marc's research interests center around data-efficient machine learning, probabilistic modeling and autonomous decision making. Marc was Program Chair of EWRL 2012, Workshops Chair of RSS 2013, EXPO-Co-Chair of ICML 2020, and Tutorials Co-Chair of NeurIPS 2021. In 2019, Marc co-organized the Machine Learning Summer School in London. He received Paper Awards at ICRA 2014, ICCAS 2016, and ICML 2020. He is co-author of the book [Mathematics for Machine Learning](https://mml-book.github.io) published by Cambridge University Press (2020).
More from the Same Authors
-
2021 : Imitation Learning from Pixel Observations for Continuous Control »
Samuel Cohen · Brandon Amos · Marc Deisenroth · Mikael Henaff · Eugene Vinitsky · Denis Yarats -
2021 : On Combining Expert Demonstrations in Imitation Learning via Optimal Transport »
ilana sebag · Samuel Cohen · Marc Deisenroth -
2021 : Sliced Multi-Marginal Optimal Transport »
Samuel Cohen · Alexander Terenin · Yannik Pitcan · Brandon Amos · Marc Deisenroth · Senanayak Sesh Kumar Karri -
2021 Meetup: Cambridge, UK »
Vincent Dutordoir -
2022 : Actually Sparse Variational Gaussian Processes »
Jake Cunningham · So Takao · Mark van der Wilk · Marc Deisenroth -
2022 : Short-term Prediction and Filtering of Solar Power Using State-Space Gaussian Processes »
So Takao · Sean Nassimiha · Peter Dudfield · Jack Kelly · Marc Deisenroth -
2022 : Optimal Transport for Offline Imitation Learning »
Yicheng Luo · zhengyao Jiang · Samuel Cohen · Edward Grefenstette · Marc Deisenroth -
2023 Poster: Thin and deep Gaussian processes »
Daniel Augusto de Souza · Alexander Nikitin · ST John · Magnus Ross · Mauricio A Álvarez · Marc Deisenroth · João Paulo Gomes · Diego Mesquita · César Lincoln Mattos -
2023 Poster: Geometric Neural Diffusion Processes »
Emile Mathieu · Vincent Dutordoir · Michael Hutchinson · Valentin De Bortoli · Yee Whye Teh · Richard Turner -
2023 Poster: Cauchy–Schwarz Regularized Autoencoder »
Linh Tran · Maja Pantic · Marc Deisenroth -
2021 Poster: Vector-valued Gaussian Processes on Riemannian Manifolds via Gauge Independent Projected Kernels »
Michael Hutchinson · Alexander Terenin · Slava Borovitskiy · So Takao · Yee Teh · Marc Deisenroth -
2020 : GENNI: Visualising the Geometry of Equivalences for Neural Network Identifiability »
Arinbjörn Kolbeinsson · Nicholas Jennings · Marc Deisenroth · Daniel Lengyel · Janith Petangoda · Michalis Lazarou · Kate Highnam · John IF Falk -
2020 Poster: Matérn Gaussian Processes on Riemannian Manifolds »
Slava Borovitskiy · Alexander Terenin · Peter Mostowsky · Marc Deisenroth -
2020 Session: Orals & Spotlights Track 25: Probabilistic Models/Statistics »
Marc Deisenroth · Matthew D. Hoffman -
2020 Poster: Probabilistic Active Meta-Learning »
Jean Kaddour · Steindor Saemundsson · Marc Deisenroth -
2020 Tutorial: (Track1) There and Back Again: A Tale of Slopes and Expectations Q&A »
Marc Deisenroth · Cheng Soon Ong -
2020 : Discussion Panel: Hugo Larochelle, Finale Doshi-Velez, Devi Parikh, Marc Deisenroth, Julien Mairal, Katja Hofmann, Phillip Isola, and Michael Bowling »
Hugo Larochelle · Finale Doshi-Velez · Marc Deisenroth · Devi Parikh · Julien Mairal · Katja Hofmann · Phillip Isola · Michael Bowling -
2020 Tutorial: (Track1) There and Back Again: A Tale of Slopes and Expectations »
Marc Deisenroth · Cheng Soon Ong -
2019 : Invited Talk - Marc Deisenroth »
Marc Deisenroth -
2019 Poster: Pseudo-Extended Markov chain Monte Carlo »
Christopher Nemeth · Fredrik Lindsten · Maurizio Filippone · James Hensman -
2018 Poster: Infinite-Horizon Gaussian Processes »
Arno Solin · James Hensman · Richard Turner -
2018 Poster: Maximizing acquisition functions for Bayesian optimization »
James Wilson · Frank Hutter · Marc Deisenroth -
2018 Poster: Learning Invariances using the Marginal Likelihood »
Mark van der Wilk · Matthias Bauer · ST John · James Hensman -
2018 Poster: Orthogonally Decoupled Variational Gaussian Processes »
Hugh Salimbeni · Ching-An Cheng · Byron Boots · Marc Deisenroth -
2017 Poster: Convolutional Gaussian Processes »
Mark van der Wilk · Carl Edward Rasmussen · James Hensman -
2017 Poster: Doubly Stochastic Variational Inference for Deep Gaussian Processes »
Hugh Salimbeni · Marc Deisenroth -
2017 Spotlight: Doubly Stochastic Variational Inference for Deep Gaussian Processes »
Hugh Salimbeni · Marc Deisenroth -
2017 Oral: Convolutional Gaussian Processes »
Mark van der Wilk · Carl Edward Rasmussen · James Hensman -
2017 Poster: Identification of Gaussian Process State Space Models »
Stefanos Eleftheriadis · Tom Nicholson · Marc Deisenroth · James Hensman -
2015 : Applications of Bayesian Optimization to Systems »
Marc Deisenroth -
2015 Poster: MCMC for Variationally Sparse Gaussian Processes »
James Hensman · Alexander Matthews · Maurizio Filippone · Zoubin Ghahramani -
2014 Workshop: Novel Trends and Applications in Reinforcement Learning »
Csaba Szepesvari · Marc Deisenroth · Sergey Levine · Pedro Ortega · Brian Ziebart · Emma Brunskill · Naftali Tishby · Gerhard Neumann · Daniel Lee · Sridhar Mahadevan · Pieter Abbeel · David Silver · Vicenç Gómez -
2013 Workshop: Advances in Machine Learning for Sensorimotor Control »
Thomas Walsh · Alborz Geramifard · Marc Deisenroth · Jonathan How · Jan Peters -
2013 Workshop: Probabilistic Models for Big Data »
Neil D Lawrence · Joaquin Quiñonero-Candela · Tianshi Gao · James Hensman · Zoubin Ghahramani · Max Welling · David Blei · Ralf Herbrich -
2013 Session: Tutorial Session A »
James Hensman -
2012 Poster: Expectation Propagation in Gaussian Process Dynamical Systems »
Marc Deisenroth · Shakir Mohamed -
2012 Poster: Fast Variational Inference in the Conjugate Exponential Family »
James Hensman · Magnus Rattray · Neil D Lawrence -
2009 Workshop: Probabilistic Approaches for Control and Robotics »
Marc Deisenroth · Hilbert J Kappen · Emo Todorov · Duy Nguyen-Tuong · Carl Edward Rasmussen · Jan Peters