Timezone: »

 
Poster
GULP: a prediction-based metric between representations
Enric Boix-Adsera · Hannah Lawrence · George Stepaniants · Philippe Rigollet

Thu Dec 01 09:00 AM -- 11:00 AM (PST) @ Hall J #321
in Poster Session 5 »

Comparing the representations learned by different neural networks has recently emerged as a key tool to understand various architectures and ultimately optimize them. In this work, we introduce GULP, a family of distance measures between representations that is explicitly motivated by downstream predictive tasks. By construction, GULP provides uniform control over the difference in prediction performance between two representations, with respect to regularized linear prediction tasks. Moreover, it satisfies several desirable structural properties, such as the triangle inequality and invariance under orthogonal transformations, and thus lends itself to data embedding and visualization. We extensively evaluate GULP relative to other methods, and demonstrate that it correctly differentiates between architecture families, converges over the course of training, and captures generalization performance on downstream linear tasks.

Keywords: Neural Networks · transfer learning · CKA · representations · CCA
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Enric Boix-Adsera (MIT)
Hannah Lawrence (MIT)
George Stepaniants (Massachusetts Institute of Technology)
George Stepaniants

George Stepaniants is a fourth-year PhD student at the Massachusetts Institute of Technology (MIT) in the Department of Mathematics advised by Prof. Philippe Rigollet and Prof. Jörn Dunkel. He is a member of the Interdisciplinary Doctoral Program in Statistics through the Institute for Data, Systems, and Society (IDSS). He obtained his Bachelor of Science at the University of Washington (UW) in Mathematics and Computer Science in 2019 where he performed research in the Department of Applied Mathematics under Prof. Nathan Kutz. George’s research is on the intersection of statistics and physical applied mathematics where he studies how statistical and machine learning algorithms can be used to infer and predict systems governed by ordinary and partial differential equations such as physical, biological, and chemical processes. He is also interested in the applications of optimal transport methods for solving matching problems in genomics, metabolomics, and other fields.

Philippe Rigollet (MIT)

More from the Same Authors