We propose Okapi, a simple, efficient, and general method for robust semi-supervised learning based on online statistical matching. Our method uses a nearest-neighbours-based matching procedure to generate cross-domain views for a consistency loss, while eliminating statistical outliers. In order to perform the online matching in a runtime- and memory-efficient way, we draw upon the self-supervised literature and combine a memory bank with a slow-moving momentum encoder. The consistency loss is applied within the feature space, rather than on the predictive distribution, making the method agnostic to both the modality and the task in question. We experiment on the WILDS 2.0 datasets Sagawa et al., which significantly expands the range of modalities, applications, and shifts available for studying and benchmarking real-world unsupervised adaptation. Contrary to Sagawa et al., we show that it is in fact possible to leverage additional unlabelled data to improve upon empirical risk minimisation (ERM) results with the right method. Our method outperforms the baseline methods in terms of out-of-distribution (OOD) generalisation on the iWildCam (a multi-class classification task) and PovertyMap (a regression task) image datasets as well as the CivilComments (a binary classification task) text dataset. Furthermore, from a qualitative perspective, we show the matches obtained from the learned encoder are strongly semantically related. Code for our paper is publicly available at https://github.com/wearepal/okapi/.
Myles Bartlett (University of Sussex)
Sara Romiti (University of Sussex)
I'm a PhD student in the Predictive Analytics Lab (PAL) at the University of Sussex (UK). My main research interests are causal inference and computer vision. Before joining the University of Sussex, I obtain my BSc in statistics and my MSc in data science at Sapienza University of Rome. Whilst doing my MSc, I have been an intern at the Tandon School of Engineering at New York University in the area of Visual Analytics, working on the relationship between perceptual and statistical data correlation.
Viktoriia Sharmanska (University of Sussex, Imperial College London)
Novi Quadrianto (University of Sussex, BCAM, and Monash Indonesia)
More from the Same Authors
2017 Poster: Recycling Privileged Learning and Distribution Matching for Fairness »
Novi Quadrianto · Viktoriia Sharmanska
2014 Poster: Mind the Nuisance: Gaussian Process Classification using Privileged Noise »
Daniel Hernández-lobato · Viktoriia Sharmanska · Kristian Kersting · Christoph Lampert · Novi Quadrianto