Distance Marching for Generative Modeling

Modern generative models based on diffusion or flow matching transport noise toward the data manifold, without representing the data manifold explicitly. We propose \dm that explores an alternative design of generative modeling by explicitly modeling a distance field of the data manifold. Designing loss functions to learn distance fields in high-dimensional space from point data can be challenging. This is because the data do not provide reliable signals for closest point projection onto the manifold. To mitigate this problem, we design new loss functions that weight training data by their proximity to each sampling position. This distance-field perspective naturally motivates two principled deterministic samplers, showing fast convergence in inference. Our model is related to the recent time-unconditional generative models since it also does not require time input, and our analysis reveals that our method helps time-unconditional generative modeling to disambiguate the denoising problem and to avoid biasing towards the data mean. As a result, our model is the only time-unconditional model that remains competitive to time-conditional ones across datasets and architectures. Moreover, our distance prediction is also helpful for early stopping during sampling and for OOD detection. We hope distance-field modeling can open a promising direction for exploring generative modeling from a geometric perspective.