Timezone: »

A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning
Marco Fraccaro · Simon Kamronn · Ulrich Paquet · Ole Winther

Wed Dec 06 06:30 PM -- 10:30 PM (PST) @ Pacific Ballroom #176

This paper takes a step towards temporal reasoning in a dynamically changing video, not in the pixel space that constitutes its frames, but in a latent space that describes the non-linear dynamics of the objects in its world. We introduce the Kalman variational auto-encoder, a framework for unsupervised learning of sequential data that disentangles two latent representations: an object's representation, coming from a recognition model, and a latent state describing its dynamics. As a result, the evolution of the world can be imagined and missing data imputed, both without the need to generate high dimensional frames at each time step. The model is trained end-to-end on videos of a variety of simulated physical systems, and outperforms competing methods in generative and missing data imputation tasks.

Author Information

Marco Fraccaro (Technical University of Denmark (DTU))
Simon Kamronn (Technical University of Denmark)

Deep learning and Bayesian statistical modelling of time series from a long-term intervention study.

Ulrich Paquet
Ole Winther (Technical University of Denmark)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors