Timezone: »

Contrastive Embedding of Structured Space for Bayesian Optimization
Josh Tingey · Ciarán Lee · Zhenwen Dai
Event URL: https://openreview.net/forum?id=xFpkJUMS9te »

Bayesian optimisation (BO) has been used to search in structured spaces described by a context-free grammar, such as chemical molecules. Previous work has used a probabilistic generative model, such as a variational autoencoder, to map the structured representations into a compact continuous embedding within which BO can take advantage of local proximity and identify good search areas. However, the resultant embedding does not fully capture the structural proximity relations of the input space, which leads to inefficient search. In this paper, we propose to use contrastive learning to learn an alternative embedding. We outline how a subtree replacement strategy can generate structurally similar pairs of samples from the input space for use in contrastive learning. We demonstrate that the resulting embedding captures more of the structural proximity relationships of the input space and improves BO performance when applied to a synthetic arithmetic expression fitting task and a real-world molecule optimisation task.

Author Information

Josh Tingey (University College London)
Ciarán Lee (Babylon)
Zhenwen Dai (Spotify)

More from the Same Authors

  • 2021 : Efficient Automated Online Experimentation with Multi-Fidelity »
    Steven Kleinegesse · Zhenwen Dai · Andreas Damianou · Kamil Ciosek · Federico Tomasi
  • 2020 Poster: Model Selection for Production System via Automated Online Experiments »
    Zhenwen Dai · Praveen Chandar · Ghazal Fazelnia · Benjamin Carterette · Mounia Lalmas
  • 2019 : Poster Spotlights »
    Théophile Griveau-Billion · Rahul Singh · Zichen Zhang · Ciarán Lee · Jesse Krijthe · Grace Charles · Vira Semenova · Rahul Ladhania · Miruna Oprescu
  • 2019 : Coffee break, posters, and 1-on-1 discussions »
    Yangyi Lu · Daniel Chen · Hongseok Namkoong · Marie Charpignon · Maja Rudolph · Amanda Coston · Julius von Kügelgen · Niranjani Prasad · Paramveer Dhillon · Yunzong Xu · Yixin Wang · Alexander Markham · David Rohde · Rahul Singh · Zichen Zhang · Negar Hassanpour · Ankit Sharma · Ciarán Lee · Jean Pouget-Abadie · Jesse Krijthe · Divyat Mahajan · Nan Rosemary Ke · Peter Wirnsberger · Vira Semenova · Dmytro Mykhaylov · Dennis Shen · Kenta Takatsu · Liyang Sun · Jeremy Yang · Alexander Franks · Pak Kan Wong · Tauhid Zaman · Shira Mitchell · min kyoung kang · Qi Yang