Skip to yearly menu bar Skip to main content


Poster

Neural Networks Trained to Solve Differential Equations Learn General Representations

Martin Magill · Faisal Qureshi · Hendrick de Haan

Room 517 AB #166

Keywords: [ Visualization or Exposition Techniques for Deep Networks ] [ Components Analysis (e.g., CCA, ICA, LDA, PCA) ] [ Representation Learning ] [ Multitask and Transfer Learning ] [ Supervised Deep Networks ]


Abstract:

We introduce a technique based on the singular vector canonical correlation analysis (SVCCA) for measuring the generality of neural network layers across a continuously-parametrized set of tasks. We illustrate this method by studying generality in neural networks trained to solve parametrized boundary value problems based on the Poisson partial differential equation. We find that the first hidden layers are general, and that they learn generalized coordinates over the input domain. Deeper layers are successively more specific. Next, we validate our method against an existing technique that measures layer generality using transfer learning experiments. We find excellent agreement between the two methods, and note that our method is much faster, particularly for continuously-parametrized problems. Finally, we also apply our method to networks trained on MNIST, and show it is consistent with, and complimentary to, another study of intrinsic dimensionality.

Live content is unavailable. Log in and register to view live content