Timezone: »

 
Poster
Large-Scale Wasserstein Gradient Flows
Petr Mokrov · Alexander Korotin · Lingxiao Li · Aude Genevay · Justin Solomon · Evgeny Burnaev

Tue Dec 07 08:30 AM -- 10:00 AM (PST) @ Virtual
in Poster Session 1 »

Wasserstein gradient flows provide a powerful means of understanding and solving many diffusion equations. Specifically, Fokker-Planck equations, which model the diffusion of probability measures, can be understood as gradient descent over entropy functionals in Wasserstein space. This equivalence, introduced by Jordan, Kinderlehrer and Otto, inspired the so-called JKO scheme to approximate these diffusion processes via an implicit discretization of the gradient flow in Wasserstein space. Solving the optimization problem associated with each JKO step, however, presents serious computational challenges. We introduce a scalable method to approximate Wasserstein gradient flows, targeted to machine learning applications. Our approach relies on input-convex neural networks (ICNNs) to discretize the JKO steps, which can be optimized by stochastic gradient descent. Contrarily to previous work, our method does not require domain discretization or particle simulation. As a result, we can sample from the measure at each time step of the diffusion and compute its probability density. We demonstrate the performance of our algorithm by computing diffusions following the Fokker-Planck equation and apply it to unnormalized density sampling as well as nonlinear filtering.

Keywords: Deep Learning · Optimization · Machine Learning · Optimal Transport

Author Information

Petr Mokrov (Skolkovo Institute of Science and Technology, Moscow Institute of Physics and Technology)
Alexander Korotin (Skolkovo Institute of Science and Technology)
Lingxiao Li (MIT)
Aude Genevay (MIT)
Justin Solomon (MIT)
Evgeny Burnaev (Skoltech)

Evgeny is an experienced scientist working at the interface between machine learning and applied engineering problems. He obtained his Master’s degree in Applied Physics and Mathematics from the Moscow Institute of Physics and Technology in 2006. After successfully defending his PhD thesis in Foundations of Computer Science at the Institute for Information Transmission Problem RAS (IITP RAS) in 2008, Evgeny stayed with the Institute as a head of IITP Data Analysis and Modeling group. Today, Evgeny’s research interests encompass the areas of regression based on Gaussian Processes, bootstrap, confidence sets and conformal predictors, volatility modeling and nonparametric estimation, statistical decisions and rapid detection of anomalies in complex multicomponent systems. Evgeny always demonstrated a deep fundamental knowledge and engineer-like thinking that enabled him to effectively use methods of statistics, machine learning and predictive modeling to deal with practical tasks in hi-tech industries, primarily aerospace, medicine and life sciences. He carried out a number of successful industrial projects with Airbus, Eurocopter and Sahara Force India Formula 1 team among others. The corresponding data analysis algorithms, developed by Evgeny and his group at IITP, formed a core of the algorithmic software library for surrogate modeling and optimization. Thanks to the developed functionality, engineers can construct fast mathematical approximations to long running computer codes (realizing physical models) based on available data and perform design space exploration for trade-off studies. The software library passed the final Technology Readiness Level 6 certification in Airbus. According to Airbus experts, application of the library “provides the reduction of up to 10% of lead time and cost in several areas of the aircraft design process”. Nowadays several dozens of Airbus departments use it. Later a spin-off company developed a Software platform for Design Space Exploration with GUI based on this algorithmic core. Evgeny has also a considerable teaching experience both in Russian and English. He has developed and taught various undergraduate and graduate courses in applied mathematics at MIPT, IITP, Yandex School of Data Analysis and the Humboldt University of Berlin, as well as mini courses on application of machine learning in engineering multidisciplinary modeling and optimization for technological companies such as Astrium, Safran, SAFT, CNES, etc. Before joining Skoltech, Evgeny was a Lecturer at Yandex School of Data Analysis, Associate Professor and Vice Chairman of Information Transmission Problems and Data Analysis Chair at MIPT, data analysis expert at DATADVANCE llc., and head of IITP Data Analysis and Predictive Modeling Lab. At Skoltech, Evgeny is actively engaged in the development of CDISE educational and research programs, and continues his research in the areas of development of theoretical tools for estimation of change-point algorithms’ performance, effective algorithms for anomaly detection and failures prediction, analysis of their properties, and development of a core library for anomaly detection and failures prediction.

More from the Same Authors