Timezone: »

 
Poster
On Second Order Behaviour in Augmented Neural ODEs
Alexander Norcliffe · Cristian Bodnar · Ben Day · Nikola Simidjievski · Pietro Lió

Thu Dec 10 09:00 AM -- 11:00 AM (PST) @ Poster Session 5 #1403
in Poster Session 6 »

Neural Ordinary Differential Equations (NODEs) are a new class of models that transform data continuously through infinite-depth architectures. The continuous nature of NODEs has made them particularly suitable for learning the dynamics of complex physical systems. While previous work has mostly been focused on first order ODEs, the dynamics of many systems, especially in classical physics, are governed by second order laws. In this work, we consider Second Order Neural ODEs (SONODEs). We show how the adjoint sensitivity method can be extended to SONODEs and prove that the optimisation of a first order coupled ODE is equivalent and computationally more efficient. Furthermore, we extend the theoretical understanding of the broader class of Augmented NODEs (ANODEs) by showing they can also learn higher order dynamics with a minimal number of augmented dimensions, but at the cost of interpretability. This indicates that the advantages of ANODEs go beyond the extra space offered by the augmented dimensions, as originally thought. Finally, we compare SONODEs and ANODEs on synthetic and real dynamical systems and demonstrate that the inductive biases of the former generally result in faster training and better performance.

Author Information

Alexander Norcliffe (University College London)

I'm a PhD student in Machine Learning for Medicine. I am co-supervised by Mihaela Van der Schaar and Pietro Lio in the Cambridge Centre for AI in Medicine.

Cristian Bodnar (University of Cambridge)
Ben Day (University of Cambridge)
Nikola Simidjievski (University of Cambridge)
Pietro Lió (University of Cambridge)

More from the Same Authors