Timezone: »
Contextual policies are used in many settings to customize system parameters and actions to the specifics of a particular setting. In some real-world settings, such as randomized controlled trials or A/B tests, it may not be possible to measure policy outcomes at the level of context—we observe only aggregate rewards across a distribution of contexts. This makes policy optimization much more difficult because we must solve a high-dimensional optimization problem over the entire space of contextual policies, for which existing optimization methods are not suitable. We develop effective models that leverage the structure of the search space to enable contextual policy optimization directly from the aggregate rewards using Bayesian optimization. We use a collection of simulation studies to characterize the performance and robustness of the models, and show that our approach of inferring a low-dimensional context embedding performs best. Finally, we show successful contextual policy optimization in a real-world video bitrate policy problem.
Author Information
Qing Feng (Facebook)
Ben Letham (Facebook)
Hongzi Mao (MIT)
Eytan Bakshy (Facebook)
Related Events (a corresponding poster, oral, or spotlight)
-
2020 Poster: High-Dimensional Contextual Policy Search with Unknown Context Rewards using Bayesian Optimization »
Wed. Dec 9th 05:00 -- 07:00 AM Room Poster Session 2 #593
More from the Same Authors
-
2021 : Practical Policy Optimization with PersonalizedExperimentation »
Mia Garrard · Hanson Wang · Ben Letham · Zehui Wang · Yin Huang · Yichun Hu · Chad Zhou · Norm Zhou · Eytan Bakshy -
2021 : Semiparametric approaches for decision making in high-dimensional sensory discrimination tasks »
Stephen Keeley · Ben Letham · Chase Tymms · Michael Shvartsman -
2021 : Optimizing High-Dimensional Physics Simulations via Composite Bayesian Optimization »
Wesley Maddox · Qing Feng · Maximilian Balandat -
2022 : Sparse Bayesian Optimization »
Sulin Liu · Qing Feng · David Eriksson · Ben Letham · Eytan Bakshy -
2022 : One-Shot Optimal Design for Gaussian Process Analysis of Randomized Experiments »
Jelena Markovic · Qing Feng · Eytan Bakshy -
2023 Poster: Unexpected Improvements to Expected Improvement for Bayesian Optimization »
Sebastian Ament · Samuel Daulton · David Eriksson · Maximilian Balandat · Eytan Bakshy -
2022 : Panel »
Roman Garnett · José Miguel Hernández-Lobato · Eytan Bakshy · Syrine Belakaria · Stefanie Jegelka -
2022 Poster: Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterization »
Samuel Daulton · Xingchen Wan · David Eriksson · Maximilian Balandat · Michael A Osborne · Eytan Bakshy -
2021 Poster: Multi-Step Budgeted Bayesian Optimization with Unknown Evaluation Costs »
Raul Astudillo · Daniel Jiang · Maximilian Balandat · Eytan Bakshy · Peter Frazier -
2021 Poster: Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvement »
Samuel Daulton · Maximilian Balandat · Eytan Bakshy -
2021 Poster: Bayesian Optimization with High-Dimensional Outputs »
Wesley Maddox · Maximilian Balandat · Andrew Wilson · Eytan Bakshy -
2020 Poster: Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimization »
Samuel Daulton · Maximilian Balandat · Eytan Bakshy -
2020 Poster: BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization »
Maximilian Balandat · Brian Karrer · Daniel Jiang · Samuel Daulton · Ben Letham · Andrew Wilson · Eytan Bakshy -
2020 Poster: Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization »
Ben Letham · Roberto Calandra · Akshara Rai · Eytan Bakshy -
2019 : Poster Session 1 »
Hongzi Mao · Vikram Nathan · Ioana Baldini · Viswanath Sivakumar · Haonan Wang · Vinoj Yasanga Jayasundara Magalle Hewa · Zhan Shi · Samuel Kaufman · Joyce Fang · Giulio Zhou · Jialin Ding · Hao He · Miles Lubin -
2019 : Invited Speaker: Eytan Bakshy »
Eytan Bakshy -
2019 Poster: Park: An Open Platform for Learning-Augmented Computer Systems »
Hongzi Mao · Parimarjan Negi · Akshay Narayan · Hanrui Wang · Jiacheng Yang · Haonan Wang · Ryan Marcus · Ravichandra Addanki · Mehrdad Khani Shirkoohi · Songtao He · Vikram Nathan · Frank Cangialosi · Shaileshh Venkatakrishnan · Wei-Hung Weng · Song Han · Tim Kraska · Dr.Mohammad Alizadeh -
2019 Poster: Learning Generalizable Device Placement Algorithms for Distributed Machine Learning »
Ravichandra Addanki · Shaileshh Bojja Venkatakrishnan · Shreyan Gupta · Hongzi Mao · Mohammad Alizadeh -
2018 : Poster Session (All Posters) »
Stephen Macke · Hongzi Mao · Caroline Lemieux · Saim Salman · Rishikesh Jha · Hanrui Wang · Shoumik P Palkar · Tianqi Chen · Thomas Pumir · Vaishnav Janardhan · adit bhardwaj · Ed Chi -
2018 : Software Panel »
Ben Letham · David Duvenaud · Dustin Tran · Aki Vehtari