Timezone: »
We present a novel model architecture which leverages deep learning tools to perform exact Bayesian inference on sets of high dimensional, complex observations. Our model is provably exchangeable, meaning that the joint distribution over observations is invariant under permutation: this property lies at the heart of Bayesian inference. The model does not require variational approximations to train, and new samples can be generated conditional on previous samples, with cost linear in the size of the conditioning set. The advantages of our architecture are demonstrated on learning tasks that require generalisation from short observed sequences while modelling sequence variability, such as conditional image generation, few-shot learning, and anomaly detection.
Author Information
Iryna Korshunova (Ghent University)
Jonas Degrave (Deepmind)
Ferenc Huszar (Twitter)
Yarin Gal (University of OXford)
Arthur Gretton (Gatsby Unit, UCL)
Arthur Gretton is a Professor with the Gatsby Computational Neuroscience Unit at UCL. He received degrees in Physics and Systems Engineering from the Australian National University, and a PhD with Microsoft Research and the Signal Processing and Communications Laboratory at the University of Cambridge. He previously worked at the MPI for Biological Cybernetics, and at the Machine Learning Department, Carnegie Mellon University. Arthur's recent research interests in machine learning include the design and training of generative models, both implicit (e.g. GANs) and explicit (high/infinite dimensional exponential family models), nonparametric hypothesis testing, and kernel methods. He has been an associate editor at IEEE Transactions on Pattern Analysis and Machine Intelligence from 2009 to 2013, an Action Editor for JMLR since April 2013, an Area Chair for NeurIPS in 2008 and 2009, a Senior Area Chair for NeurIPS in 2018, an Area Chair for ICML in 2011 and 2012, and a member of the COLT Program Committee in 2013. Arthur was program chair for AISTATS in 2016 (with Christian Robert), tutorials chair for ICML 2018 (with Ruslan Salakhutdinov), workshops chair for ICML 2019 (with Honglak Lee), program chair for the Dali workshop in 2019 (with Krikamol Muandet and Shakir Mohammed), and co-organsier of the Machine Learning Summer School 2019 in London (with Marc Deisenroth).
Joni Dambre (Ghent University)
More from the Same Authors
-
2020 : Paper 40: Real2sim: Automatic Generation of Open Street Map Towns For Autonomous Driving Benchmarks »
Panagiotis Tigas · Yarin Gal -
2021 : Depth without the Magic: Inductive Biases of Natural Gradient Descent »
Anna Mészáros · Anna Kerekes · Ferenc Huszar -
2021 : Kernel Methods for Multistage Causal Inference: Mediation Analysis and Dynamic Treatment Effects »
Rahul Singh · Ritsugen Jo · Arthur Gretton -
2021 : Return Dispersion as an Estimator of Learning Potential for Prioritized Level Replay »
Iryna Korshunova · Minqi Jiang · Jack Parker-Holder · Tim Rocktäschel · Edward Grefenstette -
2021 : Composite Goodness-of-fit Tests with Kernels »
Oscar Key · Tamara Fernandez · Arthur Gretton · Francois-Xavier Briol -
2022 : Discovering Long-period Exoplanets using Deep Learning with Citizen Science Labels »
Shreshth A Malik · Nora Eisner · Chris Lintott · Yarin Gal -
2022 : Rethinking Sharpness-Aware Minimization as Variational Inference »
Szilvia Ujváry · Zsigmond Telek · Anna Kerekes · Anna Mészáros · Ferenc Huszar -
2022 : TranceptEVE: Combining Family-specific and Family-agnostic Models of Protein Sequences for Improved Fitness Prediction »
Pascal Notin · Lodevicus van Niekerk · Aaron Kollasch · Daniel Ritter · Yarin Gal · Debora Marks -
2022 : Can Active Sampling Reduce Causal Confusion in Offline Reinforcement Learning? »
Gunshi Gupta · Tim G. J. Rudner · Rowan McAllister · Adrien Gaidon · Yarin Gal -
2022 : Can Active Sampling Reduce Causal Confusion in Offline Reinforcement Learning? »
Gunshi Gupta · Tim G. J. Rudner · Rowan McAllister · Adrien Gaidon · Yarin Gal -
2022 : What 'Out-of-distribution' Is and Is Not »
Sebastian Farquhar · Yarin Gal -
2022 : Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation »
Lorenz Kuhn · Yarin Gal · Sebastian Farquhar -
2022 : Can Active Sampling Reduce Causal Confusion in Offline Reinforcement Learning? »
Gunshi Gupta · Tim G. J. Rudner · Rowan McAllister · Adrien Gaidon · Yarin Gal -
2023 Poster: Causal de Finetti: On the Identification of Invariant Causal Structure in Exchangeable Data »
Siyuan Guo · Viktor Toth · Bernhard Schölkopf · Ferenc Huszar -
2023 Poster: Nonlinear Meta-Learning Can Guarantee Faster Rates »
Dimitri Meunier · Zhu Li · Arthur Gretton · Samory Kpotufe -
2023 Poster: MMD-Fuse: Learning and Combining Kernels for Two-Sample Testing Without Data Splitting »
Felix Biggs · Antonin Schrab · Arthur Gretton -
2023 Poster: FedL2P: Federated Learning to Personalize »
Royson Lee · Minyoung Kim · Da Li · Xinchi Qiu · Timothy Hospedales · Ferenc Huszar · Nicholas Lane -
2023 Poster: ProteinNPT: Improving protein property prediction and design with non-parametric transformers »
Pascal Notin · Ruben Weitzman · Debora Marks · Yarin Gal -
2023 Poster: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design »
Pascal Notin · Aaron Kollasch · Daniel Ritter · Lodevicus van Niekerk · Nathan Rollins · Steffanie Paul · Ada Shaw · Ruben Weitzman · Jonathan Frazer · Mafalda Dias · Dinko Franceschi · Rose Orenbuch · Han Spinner · Yarin Gal · Debora Marks -
2023 Poster: MMD Aggregated Two-Sample Test »
Antonin Schrab · Ilmun Kim · Mélisande Albert · Béatrice Laurent · Benjamin Guedj · Arthur Gretton -
2022 Poster: Tractable Function-Space Variational Inference in Bayesian Neural Networks »
Tim G. J. Rudner · Zonghao Chen · Yee Whye Teh · Yarin Gal -
2022 Poster: Optimal Rates for Regularized Conditional Mean Embedding Learning »
Zhu Li · Dimitri Meunier · Mattes Mollenhauer · Arthur Gretton -
2022 Poster: Scalable Sensitivity and Uncertainty Analyses for Causal-Effect Estimates of Continuous-Valued Interventions »
Andrew Jesson · Alyson Douglas · Peter Manshausen · Maëlys Solal · Nicolai Meinshausen · Philip Stier · Yarin Gal · Uri Shalit -
2022 Poster: KSD Aggregated Goodness-of-fit Test »
Antonin Schrab · Benjamin Guedj · Arthur Gretton -
2022 Poster: Efficient Aggregated Kernel Tests using Incomplete $U$-statistics »
Antonin Schrab · Ilmun Kim · Benjamin Guedj · Arthur Gretton -
2022 Poster: Interventions, Where and How? Experimental Design for Causal Models at Scale »
Panagiotis Tigas · Yashas Annadani · Andrew Jesson · Bernhard Schölkopf · Yarin Gal · Stefan Bauer -
2022 Poster: Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation »
Jannik Kossen · Sebastian Farquhar · Yarin Gal · Thomas Rainforth -
2021 Workshop: Machine Learning Meets Econometrics (MLECON) »
David Bruns-Smith · Arthur Gretton · Limor Gultchin · Niki Kilbertus · Krikamol Muandet · Evan Munro · Angela Zhou -
2021 Poster: KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint Support »
Pierre Glaser · Michael Arbel · Arthur Gretton -
2021 Poster: Deep Proxy Causal Learning and its Application to Confounded Bandit Policy Evaluation »
Ritsugen Jo · Heishiro Kanagawa · Arthur Gretton -
2021 Poster: Self-Supervised Learning with Kernel Dependence Maximization »
Yazhe Li · Roman Pogodin · Danica J. Sutherland · Arthur Gretton -
2020 Poster: A Non-Asymptotic Analysis for Stein Variational Gradient Descent »
Anna Korba · Adil Salim · Michael Arbel · Giulia Luise · Arthur Gretton -
2020 Poster: A kernel test for quasi-independence »
Tamara Fernandez · Wenkai Xu · Marc Ditzhaus · Arthur Gretton -
2020 Spotlight: A kernel test for quasi-independence »
Tamara Fernandez · Wenkai Xu · Marc Ditzhaus · Arthur Gretton -
2019 Poster: Exponential Family Estimation via Adversarial Dynamics Embedding »
Bo Dai · Zhen Liu · Hanjun Dai · Niao He · Arthur Gretton · Le Song · Dale Schuurmans -
2019 Poster: Discriminative Topic Modeling with Logistic LDA »
Iryna Korshunova · Hanchen Xiong · Mateusz Fedoryszak · Lucas Theis -
2019 Poster: Maximum Mean Discrepancy Gradient Flow »
Michael Arbel · Anna Korba · Adil Salim · Arthur Gretton -
2019 Poster: Kernel Instrumental Variable Regression »
Rahul Singh · Maneesh Sahani · Arthur Gretton -
2019 Oral: Kernel Instrumental Variable Regression »
Rahul Singh · Maneesh Sahani · Arthur Gretton -
2019 Tutorial: Interpretable Comparison of Distributions and Models »
Wittawat Jitkrittum · Danica J. Sutherland · Arthur Gretton -
2018 : Spotlights 2 »
Aditya Gopalan · Sungjoon Choi · Thomas Ringstrom · Roy Fox · Jonas Degrave · Xiya Cao · Karl Pertsch · Maximilian Igl · Brian Ichter -
2018 : Poster Session 1 »
Kyle H Ambert · Brandon Araki · Xiya Cao · Sungjoon Choi · Hao(Jackson) Cui · Jonas Degrave · Yaqi Duan · Mattie Fellows · Carlos Florensa · Karan Goel · Aditya Gopalan · Ming-Xu Huang · Jonathan Hunt · Cyril Ibrahim · Brian Ichter · Maximilian Igl · Zheng Tracy Ke · Igor Kiselev · Anuj Mahajan · Arash Mehrjou · Karl Pertsch · Alexandre Piche · Nicholas Rhinehart · Thomas Ringstrom · Reazul Hasan Russel · Oleh Rybkin · Ion Stoica · Sharad Vikram · Angelina Wang · Ting-Han Wei · Abigail H Wen · I-Chen Wu · Zhengwei Wu · Linhai Xie · Dinghan Shen -
2018 Poster: Informative Features for Model Comparison »
Wittawat Jitkrittum · Heishiro Kanagawa · Patsorn Sangkloy · James Hays · Bernhard Schölkopf · Arthur Gretton -
2018 Poster: On gradient regularizers for MMD GANs »
Michael Arbel · Danica J. Sutherland · Mikołaj Bińkowski · Arthur Gretton -
2017 : Conditional Densities and Efficient Models in Infinite Exponential Families »
Arthur Gretton -
2017 Poster: A Linear-Time Kernel Goodness-of-Fit Test »
Wittawat Jitkrittum · Wenkai Xu · Zoltan Szabo · Kenji Fukumizu · Arthur Gretton -
2017 Oral: A Linear-Time Kernel Goodness-of-Fit Test »
Wittawat Jitkrittum · Wenkai Xu · Zoltan Szabo · Kenji Fukumizu · Arthur Gretton -
2016 Workshop: Adaptive and Scalable Nonparametric Methods in Machine Learning »
Aaditya Ramdas · Arthur Gretton · Bharath Sriperumbudur · Han Liu · John Lafferty · Samory Kpotufe · Zoltán Szabó -
2016 : Discussion panel »
Ian Goodfellow · Soumith Chintala · Arthur Gretton · Sebastian Nowozin · Aaron Courville · Yann LeCun · Emily Denton -
2016 : Learning features to distinguish distributions »
Arthur Gretton -
2016 Oral: Interpretable Distribution Features with Maximum Testing Power »
Wittawat Jitkrittum · Zoltán Szabó · Kacper P Chwialkowski · Arthur Gretton -
2016 Poster: Interpretable Distribution Features with Maximum Testing Power »
Wittawat Jitkrittum · Zoltán Szabó · Kacper P Chwialkowski · Arthur Gretton -
2015 : *Arthur Gretton* Learning with Probabilities as Inputs, Using Kernels »
Arthur Gretton -
2015 Poster: Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families »
Heiko Strathmann · Dino Sejdinovic · Samuel Livingstone · Zoltan Szabo · Arthur Gretton -
2015 Poster: Fast Two-Sample Testing with Analytic Representations of Probability Measures »
Kacper P Chwialkowski · Aaditya Ramdas · Dino Sejdinovic · Arthur Gretton -
2014 Workshop: Modern Nonparametrics 3: Automating the Learning Pipeline »
Eric Xing · Mladen Kolar · Arthur Gretton · Samory Kpotufe · Han Liu · Zoltán Szabó · Alan Yuille · Andrew G Wilson · Ryan Tibshirani · Sasha Rakhlin · Damian Kozbur · Bharath Sriperumbudur · David Lopez-Paz · Kirthevasan Kandasamy · Francesco Orabona · Andreas Damianou · Wacha Bounliphone · Yanshuai Cao · Arijit Das · Yingzhen Yang · Giulia DeSalvo · Dmitry Storcheus · Roberto Valerio -
2014 Poster: A Wild Bootstrap for Degenerate Kernel Tests »
Kacper P Chwialkowski · Dino Sejdinovic · Arthur Gretton -
2014 Oral: A Wild Bootstrap for Degenerate Kernel Tests »
Kacper P Chwialkowski · Dino Sejdinovic · Arthur Gretton -
2013 Workshop: New Directions in Transfer and Multi-Task: Learning Across Domains and Tasks »
Urun Dogan · Marius Kloft · Tatiana Tommasi · Francesco Orabona · Massimiliano Pontil · Sinno Jialin Pan · Shai Ben-David · Arthur Gretton · Fei Sha · Marco Signoretto · Rajhans Samdani · Yun-Qian Miao · Mohammad Gheshlaghi azar · Ruth Urner · Christoph Lampert · Jonathan How -
2013 Workshop: Modern Nonparametric Methods in Machine Learning »
Arthur Gretton · Mladen Kolar · Samory Kpotufe · John Lafferty · Han Liu · Bernhard Schölkopf · Alexander Smola · Rob Nowak · Mikhail Belkin · Lorenzo Rosasco · peter bickel · Yue Zhao -
2013 Poster: B-test: A Non-parametric, Low Variance Kernel Two-sample Test »
Wojciech Zaremba · Arthur Gretton · Matthew B Blaschko -
2013 Poster: A Kernel Test for Three-Variable Interactions »
Dino Sejdinovic · Arthur Gretton · Wicher Bergsma -
2013 Oral: A Kernel Test for Three-Variable Interactions »
Dino Sejdinovic · Arthur Gretton · Wicher Bergsma -
2012 Workshop: Confluence between Kernel Methods and Graphical Models »
Le Song · Arthur Gretton · Alexander Smola -
2012 Workshop: Modern Nonparametric Methods in Machine Learning »
Sivaraman Balakrishnan · Arthur Gretton · Mladen Kolar · John Lafferty · Han Liu · Tong Zhang -
2012 Poster: Optimal kernel choice for large-scale two-sample tests »
Arthur Gretton · Bharath Sriperumbudur · Dino Sejdinovic · Heiko Strathmann · Sivaraman Balakrishnan · Massimiliano Pontil · Kenji Fukumizu -
2011 Poster: Kernel Bayes' Rule »
Kenji Fukumizu · Le Song · Arthur Gretton -
2010 Workshop: Low-rank Methods for Large-scale Machine Learning »
Arthur Gretton · Michael W Mahoney · Mehryar Mohri · Ameet S Talwalkar -
2009 Workshop: Temporal Segmentation: Perspectives from Statistics, Machine Learning, and Signal Processing »
Stephane Canu · Olivier Cappé · Arthur Gretton · Zaid Harchaoui · Alain Rakotomamonjy · Jean-Philippe Vert -
2009 Workshop: Large-Scale Machine Learning: Parallelism and Massive Datasets »
Alexander Gray · Arthur Gretton · Alexander Smola · Joseph E Gonzalez · Carlos Guestrin -
2009 Session: Oral session 10: Neural Modeling and Imaging »
Arthur Gretton -
2009 Poster: Kernel Choice and Classifiability for RKHS Embeddings of Probability Distributions »
Bharath Sriperumbudur · Kenji Fukumizu · Arthur Gretton · Gert Lanckriet · Bernhard Schölkopf -
2009 Oral: Kernel Choice and Classifiability for RKHS Embeddings of Probability Distributions »
Bharath Sriperumbudur · Kenji Fukumizu · Arthur Gretton · Gert Lanckriet · Bernhard Schölkopf -
2009 Poster: Nonlinear directed acyclic structure learning with weakly additive noise models »
Robert E Tillman · Arthur Gretton · Peter Spirtes -
2009 Poster: A Fast, Consistent Kernel Two-Sample Test »
Arthur Gretton · Kenji Fukumizu · Zaid Harchaoui · Bharath Sriperumbudur -
2009 Spotlight: A Fast, Consistent Kernel Two-Sample Test »
Arthur Gretton · Kenji Fukumizu · Zaid Harchaoui · Bharath Sriperumbudur -
2008 Workshop: Kernel Learning: Automatic Selection of Optimal Kernels »
Corinna Cortes · Arthur Gretton · Gert Lanckriet · Mehryar Mohri · Afshin Rostamizadeh -
2008 Poster: Kernel Measures of Independence for non-iid Data »
Xinhua Zhang · Le Song · Arthur Gretton · Alexander Smola -
2008 Poster: Characteristic Kernels on Groups and Semigroups »
Kenji Fukumizu · Bharath Sriperumbudur · Arthur Gretton · Bernhard Schölkopf -
2008 Spotlight: Kernel Measures of Independence for non-iid Data »
Xinhua Zhang · Le Song · Arthur Gretton · Alexander Smola -
2008 Oral: Characteristic Kernels on Groups and Semigroups »
Kenji Fukumizu · Bharath Sriperumbudur · Arthur Gretton · Bernhard Schölkopf -
2008 Session: Oral session 2: Sensorimotor Control »
Arthur Gretton -
2008 Poster: Learning Taxonomies by Dependence Maximization »
Matthew B Blaschko · Arthur Gretton -
2007 Workshop: Representations and Inference on Probability Distributions »
Kenji Fukumizu · Arthur Gretton · Alexander Smola -
2007 Spotlight: Kernel Measures of Conditional Dependence »
Kenji Fukumizu · Arthur Gretton · Xiaohai Sun · Bernhard Schölkopf -
2007 Poster: Kernel Measures of Conditional Dependence »
Kenji Fukumizu · Arthur Gretton · Xiaohai Sun · Bernhard Schölkopf -
2007 Spotlight: A Kernel Statistical Test of Independence »
Arthur Gretton · Kenji Fukumizu · Choon Hui Teo · Le Song · Bernhard Schölkopf · Alexander Smola -
2007 Oral: Colored Maximum Variance Unfolding »
Le Song · Alexander Smola · Karsten Borgwardt · Arthur Gretton -
2007 Poster: Colored Maximum Variance Unfolding »
Le Song · Alexander Smola · Karsten Borgwardt · Arthur Gretton -
2007 Poster: A Kernel Statistical Test of Independence »
Arthur Gretton · Kenji Fukumizu · Choon Hui Teo · Le Song · Bernhard Schölkopf · Alexander Smola -
2006 Poster: A Kernel Method for the Two-Sample-Problem »
Arthur Gretton · Karsten Borgwardt · Malte J Rasch · Bernhard Schölkopf · Alexander Smola -
2006 Poster: Correcting Sample Selection Bias by Unlabeled Data »
Jiayuan Huang · Alexander Smola · Arthur Gretton · Karsten Borgwardt · Bernhard Schölkopf -
2006 Spotlight: Correcting Sample Selection Bias by Unlabeled Data »
Jiayuan Huang · Alexander Smola · Arthur Gretton · Karsten Borgwardt · Bernhard Schölkopf -
2006 Talk: A Kernel Method for the Two-Sample-Problem »
Arthur Gretton · Karsten Borgwardt · Malte J Rasch · Bernhard Schölkopf · Alexander Smola