Practical Measure-to-Measure Transformers: Efficient Learning with Universality Guarantees

This paper proposes transformer-based architectures for learning with empirical measures. Our architectures alleviate the computational burden of existing transformer layers for this task, which either require all-to-all particle interactions or rely on expensive optimal transport subprocedures. For measures on $\mathbb{R}$, we encode each empirical measure as a finite-dimensional token derived from order statistics and establish universal approximation guarantees for continuous maps on compact classes. For measures on $\mathbb{R}^d$, we extend both the construction and the universality result to compact classes of elliptical distributions using finitely many one-dimensional projections. Preliminary synthetic experiments validate our approach in terms of both learning performance and computational efficiency.