Timezone: »

 
Transfer Learning for Bayesian HPO with End-to-End Landmark Meta-Features
Hadi Jomaa · Sebastian Pineda Arango · Lars Schmidt-Thieme · Josif Grabocka

@
in 5th Workshop on Meta-Learning »
Event URL: https://openreview.net/forum?id=wfD-fMD4StU »

Hyperparameter optimization (HPO) is a crucial component of deploying machine learning models, however, it remains an open problem due to the resource-constrained number of possible hyperparameter evaluations. As a result, prior work focus on exploring the direction of transfer learning for tackling the sample inefficiency of HPO. In contrast to existing approaches, we propose a novel Deep Kernel Gaussian Process surrogate with Landmark Meta-features (DKLM) that can be jointly meta-trained on a set of source tasks and then transferred efficiently on a new (unseen) target task. We design DKLM to capture the similarity between hyperparameter configurations with an end-to-end meta-feature network that embeds the set of evaluated configurations and their respective performance. As a result, our novel DKLM can learn contextualized dataset-specific similarity representations for hyperparameter configurations. We experimentally validate the performance of DKLM in a wide range of HPO meta-datasets from OpenML and demonstrate the empirical superiority of our method against a series of state-of-the-art baselines.

Author Information

Hadi Jomaa (University of Hildesheim)
Sebastian Pineda Arango (Albert-Ludwigs-Universität Freiburg)
Lars Schmidt-Thieme (University of Hildesheim)
Josif Grabocka (Universität Freiburg)

More from the Same Authors