Timezone: »

 
Poster
Differentiable Multiple Shooting Layers
Stefano Massaroli · Michael Poli · Sho Sonoda · Taiji Suzuki · Jinkyoo Park · Atsushi Yamashita · Hajime Asama

Thu Dec 09 08:30 AM -- 10:00 AM (PST) @
in Poster Session 6 »

We detail a novel class of implicit neural models. Leveraging time-parallel methods for differential equations, Multiple Shooting Layers (MSLs) seek solutions of initial value problems via parallelizable root-finding algorithms. MSLs broadly serve as drop-in replacements for neural ordinary differential equations (Neural ODEs) with improved efficiency in number of function evaluations (NFEs) and wall-clock inference time. We develop the algorithmic framework of MSLs, analyzing the different choices of solution methods from a theoretical and computational perspective. MSLs are showcased in long horizon optimal control of ODEs and PDEs and as latent models for sequence generation. Finally, we investigate the speedups obtained through application of MSL inference in neural controlled differential equations (Neural CDEs) for time series classification of medical data.

Keywords: Deep Learning · Machine Learning

Author Information

Stefano Massaroli (The University of Tokyo)
Michael Poli (Stanford University)

My work spans topics in deep learning, dynamical systems, variational inference and numerical methods. I am broadly interested in ensuring the successes achieved by deep learning methods in computer vision and natural language are extended to other engineering domains.

Sho Sonoda (RIKEN AIP)
Taiji Suzuki (The University of Tokyo/RIKEN-AIP)
Jinkyoo Park (KAIST)
Atsushi Yamashita (The University of Tokyo)
Hajime Asama (The University of Tokyo)

More from the Same Authors