Timezone: »

 
On the Role of Pre-training for Meta Few-Shot Learning
Chia-You Chen · Hsuan-Tien Lin · Masashi Sugiyama · Gang Niu
Event URL: https://openreview.net/forum?id=rMb5uMu1vuj »

Few-shot learning aims to classify unknown classes of examples with a few new examples per class. There are two key routes for few-shot learning. One is to (pre-)train a classifier with examples from known classes, and then transfer the pre-trained classifier to unknown classes using the new examples. The other, called meta few-shot learning, is to couple pre-training with episodic training, which contains episodes of few-shot learning tasks simulated from the known classes. Pre-training is known to play a crucial role for the transfer route, but the role of pre-training for the episodic route is less clear. In this work, we study the role of pre-training for the episodic route. We find that pre-training serves as major role of disentangling representations of known classes, which makes the resulting learning tasks easier for episodic training. The finding allows us to shift the huge simulation burden of episodic training to a simpler pre-training stage. We justify such a benefit of shift by designing a new disentanglement-based pre-training model, which helps episodic training achieve competitive performance more efficiently.

Author Information

Chia-You Chen (Department of computer science and informational engineering, National Taiwan University)
Hsuan-Tien Lin (National Taiwan University)
Masashi Sugiyama (RIKEN / University of Tokyo)
Gang Niu (RIKEN)

More from the Same Authors