Timezone: »
We present the first general purpose framework for marginal maximum a posteriori estimation of probabilistic program variables. By using a series of code transformations, the evidence of any probabilistic program, and therefore of any graphical model, can be optimized with respect to an arbitrary subset of its sampled variables. To carry out this optimization, we develop the first Bayesian optimization package to directly exploit the source code of its target, leading to innovations in problem-independent hyperpriors, unbounded optimization, and implicit constraint satisfaction; delivering significant performance improvements over prominent existing packages. We present applications of our method to a number of tasks including engineering design and parameter optimization.
Author Information
Thomas Rainforth (University of Oxford)
Tuan Anh Le (University of Oxford)
Jan-Willem van de Meent (University of Oxford)
Michael A Osborne (U Oxford)
Frank Wood (University of British Columbia)
Dr. Wood is an associate professor in the Department of Engineering Science at the University of Oxford. Before that he was an assistant professor of Statistics at Columbia University and a research scientist at the Columbia Center for Computational Learning Systems. He formerly was a postdoctoral fellow of the Gatsby Computational Neuroscience Unit of the University College London. He holds a PhD from Brown University (â07) and BS from Cornell University (â96), both in computer science. Dr. Wood is the original architect of both the Anglican and Probabilistic-C probabilistic programming systems. He conducts AI-driven research at the boundary of probabilistic programming, Bayesian modeling, and Monte Carlo methods. Dr. Wood holds 6 patents, has authored over 50 papers, received the AISTATS best paper award in 2009, and has been awarded faculty research awards from Xerox, Google and Amazon. Prior to his academic career he was a successful entrepreneur having run and sold the content-based image retrieval company ToFish! to AOL/Time Warner and served as CEO of Interfolio.
More from the Same Authors
-
2021 : A Closer Look at Gradient Estimators with Reinforcement Learning as Inference »
Jonathan Lavington · Michael Teng · Mark Schmidt · Frank Wood -
2022 : Understanding Optimization Challenges when Encoding to Geometric Structures »
Babak Esmaeili · Robin Walters · Heiko Zimmermann · Jan-Willem van de Meent -
2022 Poster: Bezier Gaussian Processes for Tall and Wide Data »
Martin Jørgensen · Michael A Osborne -
2022 Poster: Log-Linear-Time Gaussian Processes Using Binary Tree Kernels »
Michael K. Cohen · Samuel Daulton · Michael A Osborne -
2022 Poster: Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterization »
Samuel Daulton · Xingchen Wan · David Eriksson · Maximilian Balandat · Michael A Osborne · Eytan Bakshy -
2022 Poster: Fast Bayesian Inference with Batch Bayesian Quadrature via Kernel Recombination »
Masaki Adachi · Satoshi Hayakawa · Martin Jørgensen · Harald Oberhauser · Michael A Osborne -
2021 Poster: On Pathologies in KL-Regularized Reinforcement Learning from Expert Demonstrations »
Tim G. J. Rudner · Cong Lu · Michael A Osborne · Yarin Gal · Yee Teh -
2021 Poster: Nested Variational Inference »
Heiko Zimmermann · Hao Wu · Babak Esmaeili · Jan-Willem van de Meent -
2021 Poster: Adversarial Attacks on Graph Classifiers via Bayesian Optimisation »
Xingchen Wan · Henry Kenlay · Robin Ru · Arno Blaas · Michael A Osborne · Xiaowen Dong -
2020 Poster: Gaussian Process Bandit Optimization of the Thermodynamic Variational Objective »
Vu Nguyen · Vaden Masrani · Rob Brekelmans · Michael A Osborne · Frank Wood -
2020 Poster: Neural Topographic Factor Analysis for fMRI Data »
Eli Sennesh · Zulqarnain Khan · Yiyu Wang · J Benjamin Hutchinson · Ajay Satpute · Jennifer Dy · Jan-Willem van de Meent -
2020 Poster: Bayesian Optimization for Iterative Learning »
Vu Nguyen · Sebastian Schulze · Michael A Osborne -
2019 : Jan-Willem van de Meent - Compositional Methods for Learning and Inference in Deep Probabilistic Programs »
Jan-Willem van de Meent -
2019 : Poster Session »
Gergely Flamich · Shashanka Ubaru · Charles Zheng · Josip Djolonga · Kristoffer Wickstrøm · Diego Granziol · Konstantinos Pitas · Jun Li · Robert Williamson · Sangwoong Yoon · Kwot Sin Lee · Julian Zilly · Linda Petrini · Ian Fischer · Zhe Dong · Alexander Alemi · Bao-Ngoc Nguyen · Rob Brekelmans · Tailin Wu · Aditya Mahajan · Alexander Li · Kirankumar Shiragur · Yair Carmon · Linara Adilova · SHIYU LIU · Bang An · Sanjeeb Dash · Oktay Gunluk · Arya Mazumdar · Mehul Motani · Julia Rosenzweig · Michael Kamp · Marton Havasi · Leighton P Barnes · Zhengqing Zhou · Yi Hao · Dylan Foster · Yuval Benjamini · Nati Srebro · Michael Tschannen · Paul Rubenstein · Sylvain Gelly · John Duchi · Aaron Sidford · Robin Ru · Stefan Zohren · Murtaza Dalal · Michael A Osborne · Stephen J Roberts · Moses Charikar · Jayakumar Subramanian · Xiaodi Fan · Max Schwarzer · Nicholas Roberts · Simon Lacoste-Julien · Vinay Prabhu · Aram Galstyan · Greg Ver Steeg · Lalitha Sankar · Yung-Kyun Noh · Gautam Dasarathy · Frank Park · Ngai-Man (Man) Cheung · Ngoc-Trung Tran · Linxiao Yang · Ben Poole · Andrea Censi · Tristan Sylvain · R Devon Hjelm · Bangjie Liu · Jose Gallego-Posada · Tyler Sypherd · Kai Yang · Jan Nikolas Morshuis -
2018 : Panel on research process »
Zachary Lipton · Charles Sutton · Finale Doshi-Velez · Hanna Wallach · Suchi Saria · Rich Caruana · Thomas Rainforth -
2018 : Poster Session 1 »
Stefan Gadatsch · Danil Kuzin · Navneet Kumar · Patrick Dallaire · Tom Ryder · Remus-Petru Pop · Nathan Hunt · Adam Kortylewski · Sophie Burkhardt · Mahmoud Elnaggar · Dieterich Lawson · Yifeng Li · Jongha (Jon) Ryu · Juhan Bae · Micha Livne · Tim Pearce · Mariia Vladimirova · Jason Ramapuram · Jiaming Zeng · Xinyu Hu · Jiawei He · Danielle Maddix · Arunesh Mittal · Albert Shaw · Tuan Anh Le · Alexander Sagel · Lisha Chen · Victor Gallego · Mahdi Karami · Zihao Zhang · Tal Kachman · Noah Weber · Matt Benatan · Kumar K Sricharan · Vincent Cartillier · Ivan Ovinnikov · Buu Phan · Mahmoud Hossam · Liu Ziyin · Valerii Kharitonov · Eugene Golikov · Qiang Zhang · Jae Myung Kim · Sebastian Farquhar · Jishnu Mukhoti · Xu Hu · Gregory Gundersen · Lavanya Sita Tekumalla · Paris Perdikaris · Ershad Banijamali · Siddhartha Jain · Ge Liu · Martin Gottwald · Katy Blumer · Sukmin Yun · Ranganath Krishnan · Roman Novak · Yilun Du · Yu Gong · Beliz Gokkaya · Jessica Ai · Daniel Duckworth · Johannes von Oswald · Christian Henning · Louis-Philippe Morency · Ali Ghodsi · Mahesh Subedar · Jean-Pascal Pfister · Rémi Lebret · Chao Ma · Aleksander Wieczorek · Laurence Perreault Levasseur -
2018 : TBC 1 »
Frank Wood -
2018 Workshop: Critiquing and Correcting Trends in Machine Learning »
Thomas Rainforth · Matt Kusner · Benjamin Bloem-Reddy · Brooks Paige · Rich Caruana · Yee Whye Teh -
2018 Poster: Faithful Inversion of Generative Models for Effective Amortized Inference »
Stefan Webb · Adam Golinski · Rob Zinkov · Siddharth N · Thomas Rainforth · Yee Whye Teh · Frank Wood -
2017 : Poster Spotlights »
Francesco Locatello · Ari Pakman · Da Tang · Thomas Rainforth · Zalan Borsos · Marko Järvenpää · Eric Nalisnick · Gabriele Abbati · XIAOYU LU · Jonathan Huggins · Rachit Singh · Rui Luo -
2017 Workshop: Deep Learning for Physical Sciences »
Atilim Gunes Baydin · Mr. Prabhat · Kyle Cranmer · Frank Wood -
2017 Poster: Learning Disentangled Representations with Semi-Supervised Deep Generative Models »
Siddharth Narayanaswamy · Brooks Paige · Jan-Willem van de Meent · Alban Desmaison · Noah Goodman · Pushmeet Kohli · Frank Wood · Philip Torr -
2016 : Probabilistic structure discovery in time series data »
David Janz · Brooks Paige · Thomas Rainforth · Jan-Willem van de Meent -
2015 : Black Box Policy Search with Probabilistic Programs »
Jan-Willem van de Meent -
2015 Workshop: Black box learning and inference »
Josh Tenenbaum · Jan-Willem van de Meent · Tejas Kulkarni · S. M. Ali Eslami · Brooks Paige · Frank Wood · Zoubin Ghahramani -
2015 Workshop: Probabilistic Integration »
Michael A Osborne · Philipp Hennig -
2015 Symposium: Algorithms Among Us: the Societal Impacts of Machine Learning »
Michael A Osborne · Adrian Weller · Murray Shanahan -
2015 Poster: Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees »
François-Xavier Briol · Chris Oates · Mark Girolami · Michael A Osborne -
2015 Spotlight: Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees »
François-Xavier Briol · Chris Oates · Mark Girolami · Michael A Osborne -
2015 Tutorial: Probabilistic Programming »
Frank Wood -
2014 Workshop: 3rd NIPS Workshop on Probabilistic Programming »
Daniel Roy · Josh Tenenbaum · Thomas Dietterich · Stuart J Russell · YI WU · Ulrik R Beierholm · Alp Kucukelbir · Zenna Tavares · Yura Perov · Daniel Lee · Brian Ruttenberg · Sameer Singh · Michael Hughes · Marco Gaboardi · Alexey Radul · Vikash Mansinghka · Frank Wood · Sebastian Riedel · Prakash Panangaden -
2014 Poster: Asynchronous Anytime Sequential Monte Carlo »
Brooks Paige · Frank Wood · Arnaud Doucet · Yee Whye Teh -
2014 Poster: Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature »
Tom Gunter · Michael A Osborne · Roman Garnett · Philipp Hennig · Stephen J Roberts -
2014 Oral: Asynchronous Anytime Sequential Monte Carlo »
Brooks Paige · Frank Wood · Arnaud Doucet · Yee Whye Teh -
2013 Workshop: Bayesian Optimization in Theory and Practice »
Matthew Hoffman · Jasper Snoek · Nando de Freitas · Michael A Osborne · Ryan Adams · Sebastien Bubeck · Philipp Hennig · Remi Munos · Andreas Krause -
2012 Workshop: Probabilistic Numerics »
Philipp Hennig · John P Cunningham · Michael A Osborne -
2012 Poster: Active Learning of Model Evidence Using Bayesian Quadrature »
Michael A Osborne · David Duvenaud · Roman Garnett · Carl Edward Rasmussen · Stephen J Roberts · Zoubin Ghahramani -
2011 Workshop: Bayesian optimization, experimental design and bandits: Theory and applications »
Nando de Freitas · Roman Garnett · Frank R Hutter · Michael A Osborne -
2011 Poster: Hierarchically Supervised Latent Dirichlet Allocation »
Adler J Perotte · Frank Wood · Noemie Elhadad · Nicholas Bartlett -
2010 Spotlight: Probabilistic Deterministic Infinite Automata »
David Pfau · Nicholas Bartlett · Frank Wood -
2010 Poster: Probabilistic Deterministic Infinite Automata »
David Pfau · Nicholas Bartlett · Frank Wood -
2008 Poster: Characterizing neural dependencies with Poisson copula models »
Pietro Berkes · Frank Wood · Jonathan W Pillow -
2008 Spotlight: Characterizing neural dependencies with Poisson copula models »
Pietro Berkes · Frank Wood · Jonathan W Pillow -
2008 Poster: Dependent Dirichlet Process Spike Sorting »
Jan Gasthaus · Frank Wood · Dilan Gorur · Yee Whye Teh -
2006 Poster: Particle Filtering for Nonparametric Bayesian Matrix Factorization »
Frank Wood · Tom Griffiths