Timezone: »
We introduce RNADE, a new model for joint density estimation of real-valued vectors. Our model calculates the density of a datapoint as the product of one-dimensional conditionals modeled using mixture density networks with shared parameters. RNADE learns a distributed representation of the data, while having a tractable expression for the calculation of densities. A tractable likelihood allows direct comparison with other methods and training by standard gradient-based optimizers. We compare the performance of RNADE on several datasets of heterogeneous and perceptual data, finding it outperforms mixture models in all but one case.
Author Information
Benigno Uria (University of Edinburgh)
Iain Murray (University of Edinburgh)
Iain Murray is a SICSA Lecturer in Machine Learning at the University of Edinburgh. Iain was introduced to machine learning by David MacKay and Zoubin Ghahramani, both previous NIPS tutorial speakers. He obtained his PhD in 2007 from the Gatsby Computational Neuroscience Unit at UCL. His thesis on Monte Carlo methods received an honourable mention for the ISBA Savage Award. He was a commonwealth fellow in Machine Learning at the University of Toronto, before moving to Edinburgh in 2010. Iain's research interests include building flexible probabilistic models of data, and probabilistic inference from indirect and uncertain observations. Iain is passionate about teaching. He has lectured at several Summer schools, is listed in the top 15 authors on videolectures.net, and was awarded the EUSA Van Heyningen Award for Teaching in Science and Engineering in 2015.
Hugo Larochelle (Google DeepMind)
More from the Same Authors
-
2021 Spotlight: Maximum Likelihood Training of Score-Based Diffusion Models »
Yang Song · Conor Durkan · Iain Murray · Stefano Ermon -
2021 Poster: Maximum Likelihood Training of Score-Based Diffusion Models »
Yang Song · Conor Durkan · Iain Murray · Stefano Ermon -
2019 Poster: Neural Spline Flows »
Conor Durkan · Artur Bekasov · Iain Murray · George Papamakarios -
2017 Workshop: Workshop on Meta-Learning »
Roberto Calandra · Frank Hutter · Hugo Larochelle · Sergey Levine -
2017 : Panel session »
Iain Murray · Max Welling · Juan Carrasquilla · Anatole von Lilienfeld · Gilles Louppe · Kyle Cranmer -
2017 : Invited talk 3: Learning priors, likelihoods, or posteriors »
Iain Murray -
2017 : Invited talk: Iain Murray (TBA) »
Iain Murray -
2017 Oral: Masked Autoregressive Flow for Density Estimation »
George Papamakarios · Iain Murray · Theo Pavlakou -
2017 Poster: Masked Autoregressive Flow for Density Estimation »
George Papamakarios · Iain Murray · Theo Pavlakou -
2016 Poster: Fast ε-free Inference of Simulation Models with Bayesian Conditional Density Estimation »
George Papamakarios · Iain Murray -
2015 Tutorial: Monte Carlo Inference Methods »
Iain Murray -
2014 Session: Oral Session 3 »
Hugo Larochelle -
2014 Poster: An Autoencoder Approach to Learning Bilingual Word Representations »
Sarath Chandar · Stanislas Lauly · Hugo Larochelle · Mitesh Khapra · Balaraman Ravindran · Vikas C Raykar · Amrita Saha -
2013 Workshop: Deep Learning »
Yoshua Bengio · Hugo Larochelle · Russ Salakhutdinov · Tomas Mikolov · Matthew D Zeiler · David Mcallester · Nando de Freitas · Josh Tenenbaum · Jian Zhou · Volodymyr Mnih -
2013 Session: Spotlight Session 10 »
Hugo Larochelle -
2013 Session: Spotlight Session 9 »
Hugo Larochelle -
2013 Session: Spotlight Session 8 »
Hugo Larochelle -
2013 Session: Spotlight Session 7 »
Hugo Larochelle -
2013 Session: Spotlight Session 6 »
Hugo Larochelle -
2013 Session: Spotlight Session 5 »
Hugo Larochelle -
2013 Session: Spotlight Session 4 »
Hugo Larochelle -
2013 Session: Spotlight Session 3 »
Hugo Larochelle -
2013 Session: Spotlight Session 2 »
Hugo Larochelle -
2013 Session: Spotlight Session 1 »
Hugo Larochelle -
2012 Poster: A Neural Autoregressive Topic Model »
Hugo Larochelle · Stanislas Lauly -
2012 Poster: Practical Bayesian Optimization of Machine Learning Algorithms »
Jasper Snoek · Hugo Larochelle · Ryan Adams -
2011 Poster: How biased are maximum entropy models? »
Jakob H Macke · Iain Murray · Peter E Latham -
2010 Workshop: Monte Carlo Methods for Bayesian Inference in Modern Day Applications »
Ryan Adams · Mark A Girolami · Iain Murray -
2010 Oral: Learning to combine foveal glimpses with a third-order Boltzmann machine »
Hugo Larochelle · Geoffrey E Hinton -
2010 Oral: Slice sampling covariance hyperparameters of latent Gaussian models »
Iain Murray · Ryan Adams -
2010 Poster: Learning to combine foveal glimpses with a third-order Boltzmann machine »
Hugo Larochelle · Geoffrey E Hinton -
2010 Poster: Slice sampling covariance hyperparameters of latent Gaussian models »
Iain Murray · Ryan Adams -
2010 Session: Spotlights Session 5 »
Iain Murray -
2010 Session: Oral Session 5 »
Iain Murray -
2008 Poster: Comparing model predictions of response bias and variance in cue combination »
Rama Natarajan · Iain Murray · Ladan Shams · Richard Zemel -
2008 Poster: The Gaussian Process Density Sampler »
Ryan Adams · Iain Murray · David MacKay -
2008 Spotlight: The Gaussian Process Density Sampler »
Ryan Adams · Iain Murray · David MacKay -
2008 Poster: Evaluating probabilities under high-dimensional latent variable models »
Iain Murray · Russ Salakhutdinov -
2008 Spotlight: Evaluating probabilities under high-dimensional latent variable models »
Iain Murray · Russ Salakhutdinov -
2006 Poster: Greedy Layer-Wise Training of Deep Networks »
Yoshua Bengio · Pascal Lamblin · Dan Popovici · Hugo Larochelle -
2006 Talk: Greedy Layer-Wise Training of Deep Networks »
Yoshua Bengio · Pascal Lamblin · Dan Popovici · Hugo Larochelle