Skip to yearly menu bar Skip to main content


Poster

Population Matching Discrepancy and Applications in Deep Learning

Jianfei Chen · Chongxuan LI · Yizhong Ru · Jun Zhu

Pacific Ballroom #141

Keywords: [ Deep Learning ] [ Unsupervised Learning ] [ Multitask and Transfer Learning ] [ Generative Models ]


Abstract:

A differentiable estimation of the distance between two distributions based on samples is important for many deep learning tasks. One such estimation is maximum mean discrepancy (MMD). However, MMD suffers from its sensitive kernel bandwidth hyper-parameter, weak gradients, and large mini-batch size when used as a training objective. In this paper, we propose population matching discrepancy (PMD) for estimating the distribution distance based on samples, as well as an algorithm to learn the parameters of the distributions using PMD as an objective. PMD is defined as the minimum weight matching of sample populations from each distribution, and we prove that PMD is a strongly consistent estimator of the first Wasserstein metric. We apply PMD to two deep learning tasks, domain adaptation and generative modeling. Empirical results demonstrate that PMD overcomes the aforementioned drawbacks of MMD, and outperforms MMD on both tasks in terms of the performance as well as the convergence speed.

Live content is unavailable. Log in and register to view live content