Timezone: »
Motivated by recent progress in natural image statistics, we use newly available datasets with ground truth optical flow to learn the local statistics of optical flow and rigorously compare the learned model to prior models assumed by computer vision optical flow algorithms. We find that a Gaussian mixture model with 64 components provides a significantly better model for local flow statistics when compared to commonly used models. We investigate the source of the GMMs success and show it is related to an explicit representation of flow boundaries. We also learn a model that jointly models the local intensity pattern and the local optical flow. In accordance with the assumptions often made in computer vision, the model learns that flow boundaries are more likely at intensity boundaries. However, when evaluated on a large dataset, this dependency is very weak and the benefit of conditioning flow estimation on the local intensity pattern is marginal.
Author Information
Dan Rosenbaum (The Hebrew University)
Daniel Zoran (DeepMind)
Yair Weiss (Hebrew University)
Yair Weiss is an Associate Professor at the Hebrew University School of Computer Science and Engineering. He received his Ph.D. from MIT working with Ted Adelson on motion analysis and did postdoctoral work at UC Berkeley. Since 2005 he has been a fellow of the Canadian Institute for Advanced Research. With his students and colleagues he has co-authored award winning papers in NIPS (2002),ECCV (2006), UAI (2008) and CVPR (2009).
More from the Same Authors
-
2023 Poster: Combining Behaviors with the Successor Features Keyboard »
Wilka Carvalho Carvalho · Andre Saraiva · Angelos Filos · Andrew Lampinen · Loic Matthey · Richard L Lewis · Honglak Lee · Satinder Singh · Danilo Jimenez Rezende · Daniel Zoran -
2021 Poster: When Is Unsupervised Disentanglement Possible? »
Daniella Horan · Eitan Richardson · Yair Weiss -
2021 Poster: SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decomposition »
Rishabh Kabra · Daniel Zoran · Goker Erdogan · Loic Matthey · Antonia Creswell · Matt Botvinick · Alexander Lerchner · Chris Burgess -
2019 Poster: Towards Interpretable Reinforcement Learning Using Attention Augmented Agents »
Alexander Mott · Daniel Zoran · Mike Chrzanowski · Daan Wierstra · Danilo Jimenez Rezende -
2018 Poster: On GANs and GMMs »
Eitan Richardson · Yair Weiss -
2018 Spotlight: On GANs and GMMs »
Eitan Richardson · Yair Weiss -
2017 Poster: Variational Memory Addressing in Generative Models »
Jörg Bornschein · Andriy Mnih · Daniel Zoran · Danilo Jimenez Rezende -
2017 Poster: Visual Interaction Networks: Learning a Physics Simulator from Video »
Nicholas Watters · Daniel Zoran · Theophane Weber · Peter Battaglia · Razvan Pascanu · Andrea Tacchetti -
2015 Poster: The Return of the Gating Network: Combining Generative Models and Discriminative Training in Natural Image Priors »
Dan Rosenbaum · Yair Weiss -
2015 Spotlight: The Return of the Gating Network: Combining Generative Models and Discriminative Training in Natural Image Priors »
Dan Rosenbaum · Yair Weiss -
2014 Poster: Shape and Illumination from Shading using the Generic Viewpoint Assumption »
Daniel Zoran · Dilip Krishnan · José Bento · Bill Freeman -
2012 Poster: Natural Images, Gaussian Mixtures and Dead Leaves »
Daniel Zoran · Yair Weiss -
2012 Poster: Learning about Canonical Views from Internet Image Collections »
Elad Mezuman · Yair Weiss -
2009 Invited Talk: Learning and Inference in Low-Level Vision »
Yair Weiss -
2009 Poster: Semi-Supervised Learning in Gigantic Image Collections »
Rob Fergus · Yair Weiss · Antonio Torralba -
2009 Oral: Semi-Supervised Learning in Gigantic Image Collections »
Rob Fergus · Yair Weiss · Antonio Torralba -
2009 Poster: The "tree-dependent components" of natural scenes are edge filters »
Daniel Zoran · Yair Weiss -
2008 Poster: Spectral Hashing »
Yair Weiss · Antonio Torralba · Rob Fergus