Timezone: »

 
Poster
Population Matching Discrepancy and Applications in Deep Learning
Jianfei Chen · Chongxuan LI · Yizhong Ru · Jun Zhu

Mon Dec 04 06:30 PM -- 10:30 PM (PST) @ Pacific Ballroom #141
in Posters Mon »

A differentiable estimation of the distance between two distributions based on samples is important for many deep learning tasks. One such estimation is maximum mean discrepancy (MMD). However, MMD suffers from its sensitive kernel bandwidth hyper-parameter, weak gradients, and large mini-batch size when used as a training objective. In this paper, we propose population matching discrepancy (PMD) for estimating the distribution distance based on samples, as well as an algorithm to learn the parameters of the distributions using PMD as an objective. PMD is defined as the minimum weight matching of sample populations from each distribution, and we prove that PMD is a strongly consistent estimator of the first Wasserstein metric. We apply PMD to two deep learning tasks, domain adaptation and generative modeling. Empirical results demonstrate that PMD overcomes the aforementioned drawbacks of MMD, and outperforms MMD on both tasks in terms of the performance as well as the convergence speed.

Keywords: Deep Learning · Unsupervised Learning · Multitask and Transfer Learning · Generative Models
[ Code]  [ Paper] 

Author Information

Jianfei Chen (RealAI)
Chongxuan LI (Tsinghua University)
Yizhong Ru (Tsinghua University)
Jun Zhu (Tsinghua University)

More from the Same Authors