Timezone: »
We propose a self-supervised capsule architecture for 3D point clouds. We compute capsule decompositions of objects through permutation-equivariant attention, and self-supervise the process by training with pairs of randomly rotated objects. Our key idea is to aggregate the attention masks into semantic keypoints, and use these to supervise a decomposition that satisfies the capsule invariance/equivariance properties. This not only enables the training of a semantically consistent decomposition, but also allows us to learn a canonicalization operation that enables object-centric reasoning. To train our neural network we require neither classification labels nor manually-aligned training datasets. Yet, by learning an object-centric representation in a self-supervised manner, our method outperforms the state-of-the-art on 3D point cloud reconstruction, canonicalization, and unsupervised classification.
Author Information
Weiwei Sun (University of British Columbia)
Andrea Tagliasacchi (Google Research, Brain)
Boyang Deng (Waymo LLC)
Sara Sabour (Google)
Soroosh Yazdani (Google Canada)
Geoffrey Hinton (Google)
Kwang Moo Yi (The University of British Columbia)
More from the Same Authors
-
2021 Spotlight: Neural Additive Models: Interpretable Machine Learning with Neural Nets »
Rishabh Agarwal · Levi Melnick · Nicholas Frosst · Xuezhou Zhang · Ben Lengerich · Rich Caruana · Geoffrey Hinton -
2022 Poster: D^2NeRF: Self-Supervised Decoupling of Dynamic and Static Objects from a Monocular Video »
Tianhao Wu · Fangcheng Zhong · Andrea Tagliasacchi · Forrester Cole · Cengiz Oztireli -
2023 Poster: Unsupervised Semantic Correspondence Using Stable Diffusion »
Eric Hedlin · Gopal Sharma · Shweta Mahajan · Hossam Isack · Abhishek Kar · Andrea Tagliasacchi · Kwang Moo Yi -
2023 Poster: Neural Fields with Hard Constraints of Arbitrary Differential Order »
Fangcheng Zhong · Kyle Fogarty · Param Hanji · Tianhao Wu · Alejandro Sztrajman · Andrew Spielberg · Andrea Tagliasacchi · Petra Bosilj · Cengiz Oztireli -
2022 Poster: TUSK: Task-Agnostic Unsupervised Keypoints »
Yuhe Jin · Weiwei Sun · Jan Hosang · Eduard Trulls · Kwang Moo Yi -
2022 Poster: Unsupervised Learning of Shape Programs with Repeatable Implicit Parts »
Boyang Deng · Sumith Kulal · Zhengyang Dong · Congyue Deng · Yonglong Tian · Jiajun Wu -
2022 Poster: A Unified Sequence Interface for Vision Tasks »
Ting Chen · Saurabh Saxena · Lala Li · Tsung-Yi Lin · David Fleet · Geoffrey Hinton -
2021 Poster: Neural Additive Models: Interpretable Machine Learning with Neural Nets »
Rishabh Agarwal · Levi Melnick · Nicholas Frosst · Xuezhou Zhang · Ben Lengerich · Rich Caruana · Geoffrey Hinton -
2021 Poster: Revisiting 3D Object Detection From an Egocentric Perspective »
Boyang Deng · Charles R Qi · Mahyar Najibi · Thomas Funkhouser · Yin Zhou · Dragomir Anguelov -
2020 : Andrea Tagliasacchi »
Andrea Tagliasacchi -
2020 Poster: ShapeFlow: Learnable Deformation Flows Among 3D Shapes »
Chiyu Jiang · Jingwei Huang · Andrea Tagliasacchi · Leonidas Guibas -
2020 Spotlight: ShapeFlow: Learnable Deformation Flows Among 3D Shapes »
Chiyu Jiang · Jingwei Huang · Andrea Tagliasacchi · Leonidas Guibas -
2020 Poster: CoSE: Compositional Stroke Embeddings »
Emre Aksan · Thomas Deselaers · Andrea Tagliasacchi · Otmar Hilliges -
2020 Poster: PIE-NET: Parametric Inference of Point Cloud Edges »
Xiaogang Wang · Yuelang Xu · Kai Xu · Andrea Tagliasacchi · Bin Zhou · Ali Mahdavi-Amiri · Hao Zhang -
2019 : Posters »
Colin Graber · Yuan-Ting Hu · Tiantian Fang · Jessica Hamrick · Giorgio Giannone · John Co-Reyes · Boyang Deng · Eric Crawford · Andrea Dittadi · Peter Karkus · Matthew Dirks · Rakshit Trivedi · Sunny Raj · Javier Felip Leon · Harris Chan · Jan Chorowski · Jeff Orchard · Aleksandar Stanić · Adam Kortylewski · Ben Zinberg · Chenghui Zhou · Wei Sun · Vikash Mansinghka · Chun-Liang Li · Marco Cusumano-Towner -
2019 : Spotlights 2 »
Boyang Deng · Eric Crawford -
2019 Poster: Stacked Capsule Autoencoders »
Adam Kosiorek · Sara Sabour · Yee Whye Teh · Geoffrey E Hinton -
2018 Poster: LF-Net: Learning Local Features from Images »
Yuki Ono · Eduard Trulls · Pascal Fua · Kwang Moo Yi -
2017 Poster: Dynamic Routing Between Capsules »
Sara Sabour · Nicholas Frosst · Geoffrey E Hinton -
2017 Spotlight: Dynamic Routing Between Capsules »
Sara Sabour · Nicholas Frosst · Geoffrey E Hinton -
2015 Poster: Grammar as a Foreign Language »
Oriol Vinyals · Łukasz Kaiser · Terry Koo · Slav Petrov · Ilya Sutskever · Geoffrey Hinton