Timezone: »
In this paper, we analyze the numerics of common algorithms for training Generative Adversarial Networks (GANs). Using the formalism of smooth two-player games we analyze the associated gradient vector field of GAN training objectives. Our findings suggest that the convergence of current algorithms suffers due to two factors: i) presence of eigenvalues of the Jacobian of the gradient vector field with zero real-part, and ii) eigenvalues with big imaginary part. Using these findings, we design a new algorithm that overcomes some of these limitations and has better convergence properties. Experimentally, we demonstrate its superiority on training common GAN architectures and show convergence on GAN architectures that are known to be notoriously hard to train.
Author Information
Lars Mescheder (Max-Planck Institute Tuebingen)
Sebastian Nowozin (Microsoft Research Cambridge)
Andreas Geiger (MPI Tübingen)
Related Events (a corresponding poster, oral, or spotlight)
-
2017 Poster: The Numerics of GANs »
Thu. Dec 7th 02:30 -- 06:30 AM Room Pacific Ballroom #101
More from the Same Authors
-
2021 : STEP: Segmenting and Tracking Every Pixel »
Mark Weber · Jun Xie · Maxwell Collins · Yukun Zhu · Paul Voigtlaender · Hartwig Adam · Bradley Green · Andreas Geiger · Bastian Leibe · Daniel Cremers · Aljosa Osep · Laura Leal-Taixé · Liang-Chieh Chen -
2021 Poster: On the Frequency Bias of Generative Models »
Katja Schwarz · Yiyi Liao · Andreas Geiger -
2021 Oral: Shape As Points: A Differentiable Poisson Solver »
Songyou Peng · Chiyu Jiang · Yiyi Liao · Michael Niemeyer · Marc Pollefeys · Andreas Geiger -
2021 Poster: ATISS: Autoregressive Transformers for Indoor Scene Synthesis »
Despoina Paschalidou · Amlan Kar · Maria Shugrina · Karsten Kreis · Andreas Geiger · Sanja Fidler -
2021 Poster: Shape As Points: A Differentiable Poisson Solver »
Songyou Peng · Chiyu Jiang · Yiyi Liao · Michael Niemeyer · Marc Pollefeys · Andreas Geiger -
2021 Poster: Projected GANs Converge Faster »
Axel Sauer · Kashyap Chitta · Jens Müller · Andreas Geiger -
2021 Poster: MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Images »
Shaofei Wang · Marko Mihajlovic · Qianli Ma · Andreas Geiger · Siyu Tang -
2018 Workshop: Smooth Games Optimization and Machine Learning »
Simon Lacoste-Julien · Ioannis Mitliagkas · Gauthier Gidel · Vasilis Syrgkanis · Eva Tardos · Leon Bottou · Sebastian Nowozin -
2017 Poster: Stabilizing Training of Generative Adversarial Networks through Regularization »
Kevin Roth · Aurelien Lucchi · Sebastian Nowozin · Thomas Hofmann -
2016 : Discussion panel »
Ian Goodfellow · Soumith Chintala · Arthur Gretton · Sebastian Nowozin · Aaron Courville · Yann LeCun · Emily Denton -
2016 : Training Generative Neural Samplers using Variational Divergence »
Sebastian Nowozin