Skip to yearly menu bar Skip to main content


Poster

Simulation-Free Training of Neural ODEs on Paired Data

Semin Kim · Jaehoon Yoo · Jinwoo Kim · Yeonwoo Cha · Saehoon Kim · Seunghoon Hong

[ ]
Thu 12 Dec 11 a.m. PST — 2 p.m. PST

Abstract:

In this work, we investigate a method for simulation-free training of Neural Ordinary Differential Equations (NODEs) for learning a deterministic mapping between paired data. Despite the analogy of NODEs as continuous-depth residual networks, their application in typical regression tasks has not been popular, mainly due to the large number of function evaluations of solvers and numerical instability in optimization. To alleviate this problem, we employ the flow matching framework for simulation-free training of NODEs, which directly regresses the parameterized dynamics function to the predefined target velocity field. Contrary to generative tasks, however, we show that applying flow matching directly between paired data can often lead to an ill-defined flow that breaks the coupling of the data pairs (\emph{e.g.}, due to crossing trajectories). We propose a simple extension that applies the flow matching in the embedding space of the data pairs, where such embeddings are learned jointly with the dynamic function to ensure the validity of the flow that is also easier to learn. We demonstrate the effectiveness of our method on both regression and classification tasks, where our method outperforms existing NODEs with significantly lower number of function evaluations.

Live content is unavailable. Log in and register to view live content