Timezone: »

 
Poster
Breaking the Glass Ceiling for Embedding-Based Classifiers for Large Output Spaces
Chuan Guo · Ali Mousavi · Xiang Wu · Daniel Holtmann-Rice · Satyen Kale · Sashank Reddi · Sanjiv Kumar

Thu Dec 12 05:00 PM -- 07:00 PM (PST) @ East Exhibition Hall B + C #183
in Deep Learning -- Embedding Approaches »

In extreme classification settings, embedding-based neural network models are currently not competitive with sparse linear and tree-based methods in terms of accuracy. Most prior works attribute this poor performance to the low-dimensional bottleneck in embedding-based methods. In this paper, we demonstrate that theoretically there is no limitation to using low-dimensional embedding-based methods, and provide experimental evidence that overfitting is the root cause of the poor performance of embedding-based methods. These findings motivate us to investigate novel data augmentation and regularization techniques to mitigate overfitting. To this end, we propose GLaS, a new regularizer for embedding-based neural network approaches. It is a natural generalization from the graph Laplacian and spread-out regularizers, and empirically it addresses the drawback of each regularizer alone when applied to the extreme classification setup. With the proposed techniques, we attain or improve upon the state-of-the-art on most widely tested public extreme classification datasets with hundreds of thousands of labels.

Keywords: Representation Learning · Embedding Approaches · Algorithms · Deep Learning
[ Paper]  [ Poster] 

Author Information

Chuan Guo (Cornell University)
Ali Mousavi (Google Brain)
Xiang Wu (ByteDance)
Daniel Holtmann-Rice (Google Inc)
Satyen Kale (Google)
Sashank Reddi (Google)
Sanjiv Kumar (Google Research)

More from the Same Authors