Timezone: »

 
Sparse Bayesian Optimization
Sulin Liu · Qing Feng · David Eriksson · Ben Letham · Eytan Bakshy

@
in Gaussian Processes, Spatiotemporal Modeling, and Decision-making Systems »

Bayesian optimization (BO) is a powerful approach to sample-efficient optimization of black-box objective functions. However, the application of BO to areas such as recommendation systems often requires taking the interpretability and simplicity of the configurations into consideration, a setting that has not been previously studied in the BO literature. To make BO applicable in this setting, we present several regularization-based approaches that allow us to discover sparse and more interpretable configurations. We propose a novel differentiable relaxation based on homotopy continuation that makes it possible to target sparsity by working directly with regularization. We identify failure modes for regularized BO and develop a hyperparameter-free method, sparsity exploring Bayesian optimization (SEBO) that seeks to simultaneously maximize a target objective and sparsity. SEBO and methods based on fixed regularization are evaluated on synthetic and real-world problems, and we show that we are able to efficiently optimize for sparsity.

Author Information

Sulin Liu (Princeton University)
Qing Feng (Facebook)
David Eriksson (Meta)
Ben Letham (Facebook)
Eytan Bakshy (Meta)

More from the Same Authors