Bayesian optimization has recently emerged as a popular method for the sample-efficient optimization of expensive black-box functions. However, the application to high-dimensional problems with several thousand observations remains challenging, and on difficult problems Bayesian optimization is often not competitive with other paradigms. In this paper we take the view that this is due to the implicit homogeneity of the global probabilistic models and an overemphasized exploration that results from global acquisition. This motivates the design of a local probabilistic approach for global optimization of large-scale high-dimensional problems. We propose the TuRBO algorithm that fits a collection of local models and performs a principled global allocation of samples across these models via an implicit bandit approach. A comprehensive evaluation demonstrates that TuRBO outperforms state-of-the-art methods from machine learning and operations research on problems spanning reinforcement learning, robotics, and the natural sciences.
David Eriksson (Uber AI)
I'm interested in scalable Gaussian processes and Bayesian optimization.
Michael Pearce (Warwick University)
Jacob Gardner (Uber AI Labs)
Ryan Turner (Uber AI Labs)
Matthias Poloczek (Uber AI)
Matthias’ research interests lie at the intersection of machine learning and optimization, with a focus on Bayesian methods for "exotic" optimization problems arising in business applications and in the natural sciences. He is a Principled Scientist at Amazon. Previously, Matthias was a Senior Manager at Uber AI, where he founded Uber’s Bayesian optimization team and led the cross-org effort that built a company-wide service to tune ML models at scale. Matthias received his PhD in CS from Goethe University in Frankfurt in 2013 and then worked as a postdoc at Cornell with David Williamson and Peter Frazier from 2014 until 2017. He was an Assistant Professor in the Department of Systems and Industrial Engineering at the University of Arizona from 2017 until 2019.
Related Events (a corresponding poster, oral, or spotlight)
2019 Poster: Scalable Global Optimization via Local Bayesian Optimization »
Wed. Dec 11th 01:30 -- 03:30 AM Room East Exhibition Hall B + C #9
More from the Same Authors
2022 Poster: Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspaces »
Leonard Papenmeier · Luigi Nardi · Matthias Poloczek
2019 Poster: Exact Gaussian Processes on a Million Data Points »
Ke Alexander Wang · Geoff Pleiss · Jacob Gardner · Stephen Tyree · Kilian Weinberger · Andrew Gordon Wilson