Timezone: »

 
Poster
Do Neural Optimal Transport Solvers Work? A Continuous Wasserstein-2 Benchmark
Alexander Korotin · Lingxiao Li · Aude Genevay · Justin Solomon · Alexander Filippov · Evgeny Burnaev

Tue Dec 07 08:30 AM -- 10:00 AM (PST) @
in Poster Session 1 »

Despite the recent popularity of neural network-based solvers for optimal transport (OT), there is no standard quantitative way to evaluate their performance. In this paper, we address this issue for quadratic-cost transport---specifically, computation of the Wasserstein-2 distance, a commonly-used formulation of optimal transport in machine learning. To overcome the challenge of computing ground truth transport maps between continuous measures needed to assess these solvers, we use input-convex neural networks (ICNN) to construct pairs of measures whose ground truth OT maps can be obtained analytically. This strategy yields pairs of continuous benchmark measures in high-dimensional spaces such as spaces of images. We thoroughly evaluate existing optimal transport solvers using these benchmark measures. Even though these solvers perform well in downstream tasks, many do not faithfully recover optimal transport maps. To investigate the cause of this discrepancy, we further test the solvers in a setting of image generation. Our study reveals crucial limitations of existing solvers and shows that increased OT accuracy does not necessarily correlate to better results downstream.

Keywords: Deep Learning · Machine Learning · Generative Model · Optimal Transport

Author Information

Alexander Korotin (Skolkovo Institute of Science and Technology)
Lingxiao Li (MIT)
Aude Genevay (MIT)
Justin Solomon (MIT)
Alexander Filippov (Huawei Noah's Ark Lab)
Evgeny Burnaev (Skoltech)

Evgeny is an experienced scientist working at the interface between machine learning and applied engineering problems. He obtained his Master’s degree in Applied Physics and Mathematics from the Moscow Institute of Physics and Technology in 2006. After successfully defending his PhD thesis in Foundations of Computer Science at the Institute for Information Transmission Problem RAS (IITP RAS) in 2008, Evgeny stayed with the Institute as a head of IITP Data Analysis and Modeling group. Today, Evgeny’s research interests encompass the areas of regression based on Gaussian Processes, bootstrap, confidence sets and conformal predictors, volatility modeling and nonparametric estimation, statistical decisions and rapid detection of anomalies in complex multicomponent systems. Evgeny always demonstrated a deep fundamental knowledge and engineer-like thinking that enabled him to effectively use methods of statistics, machine learning and predictive modeling to deal with practical tasks in hi-tech industries, primarily aerospace, medicine and life sciences. He carried out a number of successful industrial projects with Airbus, Eurocopter and Sahara Force India Formula 1 team among others. The corresponding data analysis algorithms, developed by Evgeny and his group at IITP, formed a core of the algorithmic software library for surrogate modeling and optimization. Thanks to the developed functionality, engineers can construct fast mathematical approximations to long running computer codes (realizing physical models) based on available data and perform design space exploration for trade-off studies. The software library passed the final Technology Readiness Level 6 certification in Airbus. According to Airbus experts, application of the library “provides the reduction of up to 10% of lead time and cost in several areas of the aircraft design process”. Nowadays several dozens of Airbus departments use it. Later a spin-off company developed a Software platform for Design Space Exploration with GUI based on this algorithmic core. Evgeny has also a considerable teaching experience both in Russian and English. He has developed and taught various undergraduate and graduate courses in applied mathematics at MIPT, IITP, Yandex School of Data Analysis and the Humboldt University of Berlin, as well as mini courses on application of machine learning in engineering multidisciplinary modeling and optimization for technological companies such as Astrium, Safran, SAFT, CNES, etc. Before joining Skoltech, Evgeny was a Lecturer at Yandex School of Data Analysis, Associate Professor and Vice Chairman of Information Transmission Problems and Data Analysis Chair at MIPT, data analysis expert at DATADVANCE llc., and head of IITP Data Analysis and Predictive Modeling Lab. At Skoltech, Evgeny is actively engaged in the development of CDISE educational and research programs, and continues his research in the areas of development of theoretical tools for estimation of change-point algorithms’ performance, effective algorithms for anomaly detection and failures prediction, analysis of their properties, and development of a core library for anomaly detection and failures prediction.

More from the Same Authors