Timezone: »
Gaussian Processes (GPs) have been widely used in machine learning to model distributions over functions, with applications including multi-modal regression, time-series prediction, and few-shot learning. GPs are particularly useful in the last application since they rely on Normal distributions and enable closed-form computation of the posterior probability function. Unfortunately, because the resulting posterior is not flexible enough to capture complex distributions, GPs assume high similarity between subsequent tasks - a requirement rarely met in real-world conditions. In this work, we address this limitation by leveraging the flexibility of Normalizing Flows to modulate the posterior predictive distribution of the GP. This makes the GP posterior locally non-Gaussian, therefore we name our method Non-Gaussian Gaussian Processes (NGGPs). More precisely, we propose an invertible ODE-based mapping that operates on each component of the random variable vectors and shares the parameters across all of them. We empirically tested the flexibility of NGGPs on various few-shot learning regression datasets, showing that the mapping can incorporate context embedding information to model different noise levels for periodic functions. As a result, our method shares the structure of the problem between subsequent tasks, but the contextualization allows for adaptation to dissimilarities. NGGPs outperform the competing state-of-the-art approaches on a diversified set of benchmarks and applications.
Author Information
Marcin Sendera (Jagiellonian University)
Jacek Tabor (Jagiellonian University)
Aleksandra Nowak (Jagiellonian Univeristy)
Andrzej Bedychaj (None)
Massimiliano Patacchiola (University of Cambridge)
Massimiliano is a postdoctoral researcher at the University of Cambridge in the Machine Learning Group. He is interested in efficient learning (few-shot, self-supervised, meta-learning), Bayesian methods (Gaussian processes), and reinforcement learning. Previously he has been a postdoctoral researcher at the University of Edinburgh and an intern in the Camera Platform team at Snapchat.
Tomasz Trzcinski (EPFL)
Przemysław Spurek (Jagiellonian University)
Maciej Zieba (Wroclaw University of Science and Technology, Tooploox)
More from the Same Authors
-
2020 : Defining Benchmarks for Continual Few-Shot Learning »
Massimiliano Patacchiola -
2022 : Trust Your $\nabla$: Gradient-based Intervention Targeting for Causal Discovery »
Mateusz Olko · Michał Zając · Aleksandra Nowak · Nino Scherrer · Yashas Annadani · Stefan Bauer · Łukasz Kuciński · Piotr Miłoś -
2022 : HyperSound: Generating Implicit Neural Representations of Audio Signals with Hypernetworks »
Filip Szatkowski · Karol J. Piczak · Przemysław Spurek · Jacek Tabor · Tomasz Trzcinski -
2022 Poster: FlowHMM: Flow-based continuous hidden Markov models »
Pawel Lorek · Rafal Nowak · Tomasz Trzcinski · Maciej Zieba -
2021 Poster: Zero Time Waste: Recycling Predictions in Early Exit Neural Networks »
Maciej Wołczyk · Bartosz Wójcik · Klaudia Bałazy · Igor T Podolak · Jacek Tabor · Marek Śmieja · Tomasz Trzcinski -
2021 Poster: Memory Efficient Meta-Learning with Large Images »
John Bronskill · Daniela Massiceti · Massimiliano Patacchiola · Katja Hofmann · Sebastian Nowozin · Richard Turner -
2020 Poster: Self-Supervised Relational Reasoning for Representation Learning »
Massimiliano Patacchiola · Amos Storkey -
2020 Spotlight: Self-Supervised Relational Reasoning for Representation Learning »
Massimiliano Patacchiola · Amos Storkey -
2020 Poster: Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels »
Massimiliano Patacchiola · Jack Turner · Elliot Crowley · Michael O'Boyle · Amos Storkey -
2020 Poster: UCSG-NET- Unsupervised Discovering of Constructive Solid Geometry Tree »
Kacper Kania · Maciej Zieba · Tomasz Kajdanowicz -
2020 Spotlight: Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels »
Massimiliano Patacchiola · Jack Turner · Elliot Crowley · Michael O'Boyle · Amos Storkey -
2019 : Poster Session »
Pravish Sainath · Mohamed Akrout · Charles Delahunt · Nathan Kutz · Guangyu Robert Yang · Joseph Marino · L F Abbott · Nicolas Vecoven · Damien Ernst · andrew warrington · Michael Kagan · Kyunghyun Cho · Kameron Harris · Leopold Grinberg · John J. Hopfield · Dmitry Krotov · Taliah Muhammad · Erick Cobos · Edgar Walker · Jacob Reimer · Andreas Tolias · Alexander Ecker · Janaki Sheth · Yu Zhang · Maciej Wołczyk · Jacek Tabor · Szymon Maszke · Roman Pogodin · Dane Corneil · Wulfram Gerstner · Baihan Lin · Guillermo Cecchi · Jenna M Reinen · Irina Rish · Guillaume Bellec · Darjan Salaj · Anand Subramoney · Wolfgang Maass · Yueqi Wang · Ari Pakman · Jin Hyung Lee · Liam Paninski · Bryan Tripp · Colin Graber · Alex Schwing · Luke Prince · Gabriel Ocker · Michael Buice · Benjamin Lansdell · Konrad Kording · Jack Lindsey · Terrence Sejnowski · Matthew Farrell · Eric Shea-Brown · Nicolas Farrugia · Victor Nepveu · Jiwoong Im · Kristin Branson · Brian Hu · Ramakrishnan Iyer · Stefan Mihalas · Sneha Aenugu · Hananel Hazan · Sihui Dai · Tan Nguyen · Doris Tsao · Richard Baraniuk · Anima Anandkumar · Hidenori Tanaka · Aran Nayebi · Stephen Baccus · Surya Ganguli · Dean Pospisil · Eilif Muller · Jeffrey S Cheng · Gaël Varoquaux · Kamalaker Dadi · Dimitrios C Gklezakos · Rajesh PN Rao · Anand Louis · Christos Papadimitriou · Santosh Vempala · Naganand Yadati · Daniel Zdeblick · Daniela M Witten · Nicholas Roberts · Vinay Prabhu · Pierre Bellec · Poornima Ramesh · Jakob H Macke · Santiago Cadena · Guillaume Bellec · Franz Scherr · Owen Marschall · Robert Kim · Hannes Rapp · Marcio Fonseca · Oliver Armitage · Jiwoong Im · Thomas Hardcastle · Abhishek Sharma · Wyeth Bair · Adrian Valente · Shane Shang · Merav Stern · Rutuja Patil · Peter Wang · Sruthi Gorantla · Peter Stratton · Tristan Edwards · Jialin Lu · Martin Ester · Yurii Vlasov · Siavash Golkar -
2019 : Coffee Break & Poster Session 1 »
Yan Zhang · Jonathon Hare · Adam Prugel-Bennett · Po Leung · Patrick Flaherty · Pitchaya Wiratchotisatian · Alessandro Epasto · Silvio Lattanzi · Sergei Vassilvitskii · Morteza Zadimoghaddam · Theja Tulabandhula · Fabian Fuchs · Adam Kosiorek · Ingmar Posner · William Hang · Anna Goldie · Sujith Ravi · Azalia Mirhoseini · Yuwen Xiong · Mengye Ren · Renjie Liao · Raquel Urtasun · Haici Zhang · Michele Borassi · Shengda Luo · Andrew Trapp · Geoffroy Dubourg-Felonneau · Yasmeen Kussad · Christopher Bender · Manzil Zaheer · Junier Oliva · Michał Stypułkowski · Maciej Zieba · Austin Dill · Chun-Liang Li · Songwei Ge · Eunsu Kang · Oiwi Parker Jones · Kelvin Ka Wing Wong · Joshua Payne · Yang Li · Azade Nazi · Erkut Erdem · Aykut Erdem · Kevin O'Connor · Juan J Garcia · Maciej Zamorski · Jan Chorowski · Deeksha Sinha · Harry Clifford · John W Cassidy -
2018 Poster: Processing of missing data by neural networks »
Marek Śmieja · Łukasz Struski · Jacek Tabor · Bartosz Zieliński · Przemysław Spurek -
2018 Poster: BinGAN: Learning Compact Binary Descriptors with a Regularized GAN »
Maciej Zieba · Piotr Semberecki · Tarek El-Gaaly · Tomasz Trzcinski -
2012 Poster: Learning Image Descriptors with the Boosting-Trick »
Tomasz Trzcinski · Christos M Christoudias · Vincent Lepetit · Pascal Fua