Timezone: »

Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation
Jannik Kossen · Sebastian Farquhar · Yarin Gal · Thomas Rainforth

Tue Nov 29 09:00 AM -- 11:00 AM (PST) @ Hall J #114

We propose Active Surrogate Estimators (ASEs), a new method for label-efficient model evaluation. Evaluating model performance is a challenging and important problem when labels are expensive. ASEs address this active testing problem using a surrogate-based estimation approach that interpolates the errors of points with unknown labels, rather than forming a Monte Carlo estimator. ASEs actively learn the underlying surrogate, and we propose a novel acquisition strategy, XWED, that tailors this learning to the final estimation task. We find that ASEs offer greater label-efficiency than the current state-of-the-art when applied to challenging model evaluation problems for deep neural networks.

Author Information

Jannik Kossen (University of Oxford)
Sebastian Farquhar (DeepMind)
Yarin Gal (University of OXford)
Thomas Rainforth (University of Oxford)

More from the Same Authors