Timezone: »
Poster
Asymptotic Guarantees for Generative Modeling Based on the Smooth Wasserstein Distance
Ziv Goldfeld · Kristjan Greenewald · Kengo Kato
Minimum distance estimation (MDE) gained recent attention as a formulation of (implicit) generative modeling. It considers minimizing, over model parameters, a statistical distance between the empirical data distribution and the model. This formulation lends itself well to theoretical analysis, but typical results are hindered by the curse of dimensionality. To overcome this and devise a scalable finite-sample statistical MDE theory, we adopt the framework of smooth 1-Wasserstein distance (SWD) $\mathsf{W}_1^{(\sigma)}$. The SWD was recently shown to preserve the metric and topological structure of classic Wasserstein distances, while enjoying dimension-free empirical convergence rates. In this work, we conduct a thorough statistical study of the minimum smooth Wasserstein estimators (MSWEs), first proving the estimator's measurability and asymptotic consistency. We then characterize the limit distribution of the optimal model parameters and their associated minimal SWD. These results imply an $O(n^{-1/2})$ generalization bound for generative modeling based on MSWE, which holds in arbitrary dimension. Our main technical tool is a novel high-dimensional limit distribution result for empirical $\mathsf{W}_1^{(\sigma)}$. The characterization of a nondegenerate limit stands in sharp contrast with the classic empirical 1-Wasserstein distance, for which a similar result is known only in the one-dimensional case. The validity of our theory is supported by empirical results, posing the SWD as a potent tool for learning and inference in high dimensions.
Author Information
Ziv Goldfeld (Cornell University)
Kristjan Greenewald (IBM Research)
Kengo Kato (Cornell University)
More from the Same Authors
-
2021 Spotlight: Sliced Mutual Information: A Scalable Measure of Statistical Dependence »
Ziv Goldfeld · Kristjan Greenewald -
2022 Poster: $k$-Sliced Mutual Information: A Quantitative Study of Scalability with Dimension »
Ziv Goldfeld · Kristjan Greenewald · Theshani Nuradha · Galen Reeves -
2022 Poster: Statistical, Robustness, and Computational Guarantees for Sliced Wasserstein Distances »
Sloan Nietert · Ziv Goldfeld · Ritwik Sadhu · Kengo Kato -
2021 Poster: Sliced Mutual Information: A Scalable Measure of Statistical Dependence »
Ziv Goldfeld · Kristjan Greenewald -
2020 Poster: Active Structure Learning of Causal DAGs via Directed Clique Trees »
Chandler Squires · Sara Magliacane · Kristjan Greenewald · Dmitriy Katz · Murat Kocaoglu · Karthikeyan Shanmugam -
2020 Poster: Entropic Causal Inference: Identifiability and Finite Sample Results »
Spencer Compton · Murat Kocaoglu · Kristjan Greenewald · Dmitriy Katz -
2019 Poster: Statistical Model Aggregation via Parameter Matching »
Mikhail Yurochkin · Mayank Agarwal · Soumya Ghosh · Kristjan Greenewald · Nghia Hoang -
2019 Poster: Sample Efficient Active Learning of Causal Trees »
Kristjan Greenewald · Dmitriy Katz · Karthikeyan Shanmugam · Sara Magliacane · Murat Kocaoglu · Enric Boix-Adsera · Guy Bresler