Timezone: »

On Fenchel Mini-Max Learning
Chenyang Tao · Liqun Chen · Shuyang Dai · Junya Chen · Ke Bai · Dong Wang · Jianfeng Feng · Wenlian Lu · Georgiy Bobashev · Lawrence Carin

Tue Dec 10 10:45 AM -- 12:45 PM (PST) @ East Exhibition Hall B + C #50

Inference, estimation, sampling and likelihood evaluation are four primary goals of probabilistic modeling. Practical considerations often force modeling approaches to make compromises between these objectives. We present a novel probabilistic learning framework, called Fenchel Mini-Max Learning (FML), that accommodates all four desiderata in a flexible and scalable manner. Our derivation is rooted in classical maximum likelihood estimation, and it overcomes a longstanding challenge that prevents unbiased estimation of unnormalized statistical models. By reformulating MLE as a mini-max game, FML enjoys an unbiased training objective that (i) does not explicitly involve the intractable normalizing constant and (ii) is directly amendable to stochastic gradient descent optimization. To demonstrate the utility of the proposed approach, we consider learning unnormalized statistical models, nonparametric density estimation and training generative models, with encouraging empirical results presented.

Author Information

Chenyang Tao (Duke University)
Liqun Chen (Duke University)
Shuyang Dai (Duke University)
Junya Chen (Duke U)
Ke Bai (Duke University)
Dong Wang (Duke University)
Jianfeng Feng (Fudan University)
Wenlian Lu (Fudan University)
Georgiy Bobashev (RTI International)
Lawrence Carin (Duke University)

More from the Same Authors