Timezone: »
Advances in unsupervised learning of object-representations have culminated in the development of a broad range of methods for unsupervised object segmentation and interpretable object-centric scene generation. These methods, however, are limited to simulated and real-world datasets with limited visual complexity. Moreover, object representations are often inferred using RNNs which do not scale well to large images or iterative refinement which avoids imposing an unnatural ordering on objects in an image but requires the a priori initialisation of a fixed number of object representations. In contrast to established paradigms, this work proposes an embedding-based approach in which embeddings of pixels are clustered in a differentiable fashion using a stochastic stick-breaking process. Similar to iterative refinement, this clustering procedure also leads to randomly ordered object representations, but without the need of initialising a fixed number of clusters a priori. This is used to develop a new model, GENESIS-v2, which can infer a variable number of object representations without using RNNs or iterative refinement. We show that GENESIS-v2 performs strongly in comparison to recent baselines in terms of unsupervised image segmentation and object-centric scene generation on established synthetic datasets as well as more complex real-world datasets.
Author Information
Martin Engelcke (DeepMind (prev. Uni. of Oxford))
Oiwi Parker Jones (University of Oxford)
Ingmar Posner (Oxford University)
More from the Same Authors
-
2022 : Causal Discovery for Modular World Models »
Anson Lei · Bernhard Schölkopf · Ingmar Posner -
2023 Poster: Neural Latent Geometry Search: Product Manifold Inference via Gromov-Hausdorff-Informed Bayesian Optimization »
Haitz Sáez de Ocáriz Borde · Alvaro Arroyo · Ismael Morales · Ingmar Posner · Xiaowen Dong -
2021 : Panel Discussion 1 »
Megan Peters · Jürgen Schmidhuber · Simona Ghetti · Nick Roy · Oiwi Parker Jones · Ingmar Posner -
2021 Workshop: Metacognition in the Age of AI: Challenges and Opportunities »
Ingmar Posner · Francesca Rossi · Lior Horesh · Steve Fleming · Oiwi Parker Jones · Rohan Paul · Biplav Srivastava · Andrea Loreggia · Marianna Ganapini -
2021 : Introduction to the Workshop on Metacognition in the Age of AI: Challenges and Opportunities »
Ingmar Posner · Steve Fleming · Francesca Rossi -
2021 Poster: E(n) Equivariant Normalizing Flows »
Victor Garcia Satorras · Emiel Hoogeboom · Fabian Fuchs · Ingmar Posner · Max Welling -
2021 Oral: E(n) Equivariant Normalizing Flows »
Victor Garcia Satorras · Emiel Hoogeboom · Fabian Fuchs · Ingmar Posner · Max Welling -
2020 Poster: RELATE: Physically Plausible Multi-Object Scene Synthesis Using Structured Latent Spaces »
Sebastien Ehrhardt · Oliver Groth · Aron Monszpart · Martin Engelcke · Ingmar Posner · Niloy Mitra · Andrea Vedaldi -
2019 : Coffee Break & Poster Session 1 »
Yan Zhang · Jonathon Hare · Adam Prugel-Bennett · Po Leung · Patrick Flaherty · Pitchaya Wiratchotisatian · Alessandro Epasto · Silvio Lattanzi · Sergei Vassilvitskii · Morteza Zadimoghaddam · Theja Tulabandhula · Fabian Fuchs · Adam Kosiorek · Ingmar Posner · William Hang · Anna Goldie · Sujith Ravi · Azalia Mirhoseini · Yuwen Xiong · Mengye Ren · Renjie Liao · Raquel Urtasun · Haici Zhang · Michele Borassi · Shengda Luo · Andrew Trapp · Geoffroy Dubourg-Felonneau · Yasmeen Kussad · Christopher Bender · Manzil Zaheer · Junier Oliva · Michał Stypułkowski · Maciej Zieba · Austin Dill · Chun-Liang Li · Songwei Ge · Eunsu Kang · Oiwi Parker Jones · Kelvin Ka Wing Wong · Joshua Payne · Yang Li · Azade Nazi · Erkut Erdem · Aykut Erdem · Kevin O'Connor · Juan J Garcia · Maciej Zamorski · Jan Chorowski · Deeksha Sinha · Harry Clifford · John W Cassidy -
2018 : Invited Talk: Ingmar Posner, Oxford and Oxbotica »
Ingmar Posner -
2018 : Ingmar Posner »
Ingmar Posner -
2018 Poster: Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects »
Adam Kosiorek · Hyunjik Kim · Yee Whye Teh · Ingmar Posner -
2018 Spotlight: Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects »
Adam Kosiorek · Hyunjik Kim · Yee Whye Teh · Ingmar Posner -
2017 Workshop: Acting and Interacting in the Real World: Challenges in Robot Learning »
Ingmar Posner · Raia Hadsell · Martin Riedmiller · Markus Wulfmeier · Rohan Paul -
2017 Poster: Hierarchical Attentive Recurrent Tracking »
Adam Kosiorek · Alex Bewley · Ingmar Posner