Timezone: »
Universal Approximation Property of Neural Ordinary Differential Equations
Takeshi Teshima · Koichi Tojo · Masahiro Ikeda · Isao Ishikawa · Kenta Oono
Neural ordinary differential equations (NODEs) is an invertible neural network architecture promising for its free-form Jacobian and the availability of a tractable Jacobian determinant estimator. Recently, the representation power of NODEs has been partly uncovered: they form an $L^p$-universal approximator for continuous maps under certain conditions. However, the $L^p$-universality may fail to guarantee an approximation for the entire input domain as it may still hold even if the approximator largely differs from the target function on a small region of the input space. To further uncover the potential of NODEs, we show their stronger approximation property, namely the $\sup$-universality for approximating a large class of diffeomorphisms. It is shown by leveraging a structure theorem of the diffeomorphism group, and the result complements the existing literature by establishing a fairly large set of mappings that NODEs can approximate with a stronger guarantee.
Author Information
Takeshi Teshima (The University of Tokyo)
Koichi Tojo (RIKEN AIP)
Masahiro Ikeda (RIKEN AIP)
Isao Ishikawa (Ehime University)
Kenta Oono (The University of Tokyo, Preferred Networks Inc.)
More from the Same Authors
-
2022 Poster: Universality of Group Convolutional Neural Networks Based on Ridgelet Analysis on Groups »
Sho Sonoda · Isao Ishikawa · Masahiro Ikeda -
2020 Poster: Optimization and Generalization Analysis of Transduction through Gradient Boosting and Application to Multi-scale Graph Neural Networks »
Kenta Oono · Taiji Suzuki -
2020 Poster: Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators »
Takeshi Teshima · Isao Ishikawa · Koichi Tojo · Kenta Oono · Masahiro Ikeda · Masashi Sugiyama -
2020 Oral: Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators »
Takeshi Teshima · Isao Ishikawa · Koichi Tojo · Kenta Oono · Masahiro Ikeda · Masashi Sugiyama -
2018 Poster: Metric on Nonlinear Dynamical Systems with Perron-Frobenius Operators »
Isao Ishikawa · Keisuke Fujii · Masahiro Ikeda · Yuka Hashimoto · Yoshinobu Kawahara