Gauss-Newton drifting extends natural gradient descent

Drifting methods have recently been introduced as a promising new paradigm for training generative models, and have attracted a lot of attention so far. In this work, we focus on how the optimization procedure of the drift loss shapes their convergence properties, shedding light on the underlying machinery that may explain their empirical success. More specifically, we show that using a Gauss--Newton scheme with a drift field given by a Wasserstein gradient amounts to performing a natural gradient descent in parameter space. This connection allows us to provide global convergence guarantees under suitable convexity assumptions.