Timezone: »
Shape programs encode shape structures by representing object parts as subroutines and constructing the overall shape by composing these subroutines. This usually involves the reuse of subroutines for repeatable parts, enabling the modeling of correlations among shape elements such as geometric similarity. However, existing learning-based shape programs suffer from limited representation capacity, because they use coarse geometry representations such as geometric primitives and low-resolution voxel grids. Further, their training requires manually annotated ground-truth programs, which are expensive to attain. We address these limitations by proposing Shape Programs with Repeatable Implicit Parts (ProGRIP). Using implicit functions to represent parts, ProGRIP greatly boosts the representation capacity of shape programs while preserving the higher-level structure of repetitions and symmetry. Meanwhile, we free ProGRIP from any inaccessible supervised training via devising a matching-based unsupervised training objective. Our empirical studies show that ProGRIP outperforms existing structured representations in both shape reconstruction fidelity and segmentation accuracy of semantic parts.
Author Information
Boyang Deng (Stanford University)
Sumith Kulal (Stanford University)
Zhengyang Dong (Stanford University)
Congyue Deng (Stanford University)
Yonglong Tian (Massachusetts Institute of Technology)
Jiajun Wu (Stanford University)
More from the Same Authors
-
2022 : What Makes Certain Pre-Trained Visual Representations Better for Robotic Learning? »
Kyle Hsu · Tyler Lum · Ruohan Gao · Shixiang (Shane) Gu · Jiajun Wu · Chelsea Finn -
2022 : A Control-Centric Benchmark for Video Prediction »
Stephen Tian · Chelsea Finn · Jiajun Wu -
2022 : Breaking the Symmetry: Resolving Symmetry Ambiguities in Equivariant Neural Networks »
Sidhika Balachandar · Adrien Poulenard · Congyue Deng · Leonidas Guibas -
2022 : What Makes Certain Pre-Trained Visual Representations Better for Robotic Learning? »
Kyle Hsu · Tyler Lum · Ruohan Gao · Shixiang (Shane) Gu · Jiajun Wu · Chelsea Finn -
2022 : Giving Robots a Hand: Broadening Generalization via Hand-Centric Human Video Demonstrations »
Moo J Kim · Jiajun Wu · Chelsea Finn -
2023 Poster: Model-Based Control with Sparse Neural Dynamics »
Ziang Liu · Jeff He · Genggeng Zhou · Tobia Marcucci · Fei-Fei Li · Jiajun Wu · Yunzhu Li -
2023 Poster: Siamese Masked Autoencoders »
Agrim Gupta · Jiajun Wu · Jia Deng · Fei-Fei Li -
2023 Poster: StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners »
Yonglong Tian · Lijie Fan · Phillip Isola · Huiwen Chang · Dilip Krishnan -
2023 Poster: Restart Sampling for Improving Generative Processes »
Yilun Xu · Mingyang Deng · Xiang Cheng · Yonglong Tian · Ziming Liu · Tommi Jaakkola -
2023 Poster: What’s Left: Concept Grounding with Large Language Models »
Joy Hsu · Jiayuan Mao · Josh Tenenbaum · Jiajun Wu -
2023 Poster: Inferring Hybrid Neural Fluid Fields from Videos »
Hong-Xing Yu · Yang Zheng · Yuan Gao · Yitong Deng · Bo Zhu · Jiajun Wu -
2023 Poster: 3D Copy-Paste: Physical Plausible Indoor Object Insertion for Monocular 3D Object Detection »
Yunhao Ge · Hong-Xing Yu · Cheng Zhao · Yuliang Guo · Xinyu Huang · Liu Ren · Laurent Itti · Jiajun Wu -
2023 Poster: NAP: Neural 3D Articulation Prior »
Jiahui Lei · Congyue Deng · William B Shen · Leonidas Guibas · Kostas Daniilidis -
2023 Poster: Improving CLIP Training with Language Rewrites »
Lijie Fan · Dilip Krishnan · Phillip Isola · Dina Katabi · Yonglong Tian -
2023 Poster: Disentanglement via Latent Quantization »
Kyle Hsu · William Dorrell · James Whittington · Chelsea Finn · Jiajun Wu -
2023 Poster: Banana: Banach Fixed-Point Network for Pointcloud Segmentation with Inter-Part Equivariance »
Congyue Deng · Jiahui Lei · William B Shen · Kostas Daniilidis · Leonidas Guibas -
2023 Poster: Real-World 3D Object Inverse Rendering Benchmark »
Zhengfei Kuang · Yunzhi Zhang · Hong-Xing Yu · Samir Agarwala · Shangzhe Wu · Jiajun Wu -
2023 Poster: Holistic Evaluation of Text-to-Image Models »
Tony Lee · Michihiro Yasunaga · Chenlin Meng · Yifan Mai · Joon Sung Park · Agrim Gupta · Yunzhi Zhang · Deepak Narayanan · Hannah Teufel · Marco Bellagente · Minguk Kang · Taesung Park · Jure Leskovec · Jun-Yan Zhu · Fei-Fei Li · Jiajun Wu · Stefano Ermon · Percy Liang -
2023 Poster: Platonic Distance: Intrinsic Object-Centric Image Similarity »
Klemen Kotar · Stephen Tian · Hong-Xing Yu · Dan Yamins · Jiajun Wu -
2023 Poster: SoundCam: A Dataset for Tasks in Tracking and Identifying Humans from Real Room Acoustics »
Mason Wang · Samuel Clarke · Jui-Hsien Wang · Ruohan Gao · Jiajun Wu -
2023 Oral: Siamese Masked Autoencoders »
Agrim Gupta · Jiajun Wu · Jia Deng · Fei-Fei Li -
2022 Spotlight: Lightning Talks 5A-3 »
Minting Pan · Xiang Chen · Wenhan Huang · Can Chang · Zhecheng Yuan · Jianzhun Shao · Yushi Cao · Peihao Chen · Ke Xue · Zhengrong Xue · Zhiqiang Lou · Xiangming Zhu · Lei Li · Zhiming Li · Kai Li · Jiacheng Xu · Dongyu Ji · Ni Mu · Kun Shao · Tianpei Yang · Kunyang Lin · Ningyu Zhang · Yunbo Wang · Lei Yuan · Bo Yuan · Hongchang Zhang · Jiajun Wu · Tianze Zhou · Xueqian Wang · Ling Pan · Yuhang Jiang · Xiaokang Yang · Xiaozhuan Liang · Hao Zhang · Weiwen Hu · Miqing Li · YAN ZHENG · Matthew Taylor · Huazhe Xu · Shumin Deng · Chao Qian · YI WU · Shuncheng He · Wenbing Huang · Chuanqi Tan · Zongzhang Zhang · Yang Gao · Jun Luo · Yi Li · Xiangyang Ji · Thomas Li · Mingkui Tan · Fei Huang · Yang Yu · Huazhe Xu · Dongge Wang · Jianye Hao · Chuang Gan · Yang Liu · Luo Si · Hangyu Mao · Huajun Chen · Jianye Hao · Jun Wang · Xiaotie Deng -
2022 Spotlight: E-MAPP: Efficient Multi-Agent Reinforcement Learning with Parallel Program Guidance »
Can Chang · Ni Mu · Jiajun Wu · Ling Pan · Huazhe Xu -
2022 Poster: E-MAPP: Efficient Multi-Agent Reinforcement Learning with Parallel Program Guidance »
Can Chang · Ni Mu · Jiajun Wu · Ling Pan · Huazhe Xu -
2022 Poster: CLEVRER-Humans: Describing Physical and Causal Events the Human Way »
Jiayuan Mao · Xuelin Yang · Xikun Zhang · Noah Goodman · Jiajun Wu -
2022 Poster: Interaction Modeling with Multiplex Attention »
Fan-Yun Sun · Isaac Kauvar · Ruohan Zhang · Jiachen Li · Mykel J Kochenderfer · Jiajun Wu · Nick Haber -
2022 Poster: Geoclidean: Few-Shot Generalization in Euclidean Geometry »
Joy Hsu · Jiajun Wu · Noah Goodman -
2022 Poster: MOMA-LRG: Language-Refined Graphs for Multi-Object Multi-Actor Activity Parsing »
Zelun Luo · Zane Durante · Linden Li · Wanze Xie · Ruochen Liu · Emily Jin · Zhuoyi Huang · Lun Yu Li · Jiajun Wu · Juan Carlos Niebles · Ehsan Adeli · Fei-Fei Li -
2022 Poster: IKEA-Manual: Seeing Shape Assembly Step by Step »
Ruocheng Wang · Yunzhi Zhang · Jiayuan Mao · Ran Zhang · Chin-Yi Cheng · Jiajun Wu -
2021 Poster: Grammar-Based Grounded Lexicon Learning »
Jiayuan Mao · Freda Shi · Jiajun Wu · Roger Levy · Josh Tenenbaum -
2021 Poster: Canonical Capsules: Self-Supervised Capsules in Canonical Pose »
Weiwei Sun · Andrea Tagliasacchi · Boyang Deng · Sara Sabour · Soroosh Yazdani · Geoffrey Hinton · Kwang Moo Yi -
2021 Poster: Revisiting 3D Object Detection From an Egocentric Perspective »
Boyang Deng · Charles R Qi · Mahyar Najibi · Thomas Funkhouser · Yin Zhou · Dragomir Anguelov -
2020 : Panel #2 »
Oren Etzioni · Heng Ji · Subbarao Kambhampati · Victoria Lin · Jiajun Wu -
2020 : Q&A #2 »
Heng Ji · Jure Leskovec · Jiajun Wu -
2020 : Invited Talk #6 »
Jiajun Wu -
2020 Poster: Multi-Plane Program Induction with 3D Box Priors »
Yikai Li · Jiayuan Mao · Xiuming Zhang · Bill Freeman · Josh Tenenbaum · Noah Snavely · Jiajun Wu -
2020 Poster: Learning Physical Graph Representations from Visual Scenes »
Daniel Bear · Chaofei Fan · Damian Mrowca · Yunzhu Li · Seth Alter · Aran Nayebi · Jeremy Schwartz · Li Fei-Fei · Jiajun Wu · Josh Tenenbaum · Daniel Yamins -
2020 Oral: Learning Physical Graph Representations from Visual Scenes »
Daniel Bear · Chaofei Fan · Damian Mrowca · Yunzhu Li · Seth Alter · Aran Nayebi · Jeremy Schwartz · Li Fei-Fei · Jiajun Wu · Josh Tenenbaum · Daniel Yamins -
2020 : Neuro-Symbolic Visual Concept Learning »
Jiajun Wu -
2019 : Posters »
Colin Graber · Yuan-Ting Hu · Tiantian Fang · Jessica Hamrick · Giorgio Giannone · John Co-Reyes · Boyang Deng · Eric Crawford · Andrea Dittadi · Peter Karkus · Matthew Dirks · Rakshit Trivedi · Sunny Raj · Javier Felip Leon · Harris Chan · Jan Chorowski · Jeff Orchard · Aleksandar Stanić · Adam Kortylewski · Ben Zinberg · Chenghui Zhou · Wei Sun · Vikash Mansinghka · Chun-Liang Li · Marco Cusumano-Towner -
2019 : Poster Session »
Ethan Harris · Tom White · Oh Hyeon Choung · Takashi Shinozaki · Dipan Pal · Katherine L. Hermann · Judy Borowski · Camilo Fosco · Chaz Firestone · Vijay Veerabadran · Benjamin Lahner · Chaitanya Ryali · Fenil Doshi · Pulkit Singh · Sharon Zhou · Michel Besserve · Michael Chang · Anelise Newman · Mahesan Niranjan · Jonathon Hare · Daniela Mihai · Marios Savvides · Simon Kornblith · Christina M Funke · Aude Oliva · Virginia de Sa · Dmitry Krotov · Colin Conwell · George Alvarez · Alex Kolchinski · Shengjia Zhao · Mitchell Gordon · Michael Bernstein · Stefano Ermon · Arash Mehrjou · Bernhard Schölkopf · John Co-Reyes · Michael Janner · Jiajun Wu · Josh Tenenbaum · Sergey Levine · Yalda Mohsenzadeh · Zhenglong Zhou -
2019 : Spotlights 2 »
Boyang Deng · Eric Crawford -
2019 Poster: SPoC: Search-based Pseudocode to Code »
Sumith Kulal · Panupong Pasupat · Kartik Chandra · Mina Lee · Oded Padon · Alex Aiken · Percy Liang -
2019 Poster: Modeling Expectation Violation in Intuitive Physics with Coarse Probabilistic Object Representations »
Kevin Smith · Lingjie Mei · Shunyu Yao · Jiajun Wu · Elizabeth Spelke · Josh Tenenbaum · Tomer Ullman -
2019 Poster: Visual Concept-Metaconcept Learning »
Chi Han · Jiayuan Mao · Chuang Gan · Josh Tenenbaum · Jiajun Wu -
2018 Workshop: Modeling the Physical World: Learning, Perception, and Control »
Jiajun Wu · Kelsey Allen · Kevin Smith · Jessica Hamrick · Emmanuel Dupoux · Marc Toussaint · Josh Tenenbaum -
2018 Poster: Learning to Reconstruct Shapes from Unseen Classes »
Xiuming Zhang · Zhoutong Zhang · Chengkai Zhang · Josh Tenenbaum · Bill Freeman · Jiajun Wu -
2018 Oral: Learning to Reconstruct Shapes from Unseen Classes »
Xiuming Zhang · Zhoutong Zhang · Chengkai Zhang · Josh Tenenbaum · Bill Freeman · Jiajun Wu -
2018 Poster: Visual Object Networks: Image Generation with Disentangled 3D Representations »
Jun-Yan Zhu · Zhoutong Zhang · Chengkai Zhang · Jiajun Wu · Antonio Torralba · Josh Tenenbaum · Bill Freeman -
2018 Poster: Learning to Exploit Stability for 3D Scene Parsing »
Yilun Du · Zhijian Liu · Hector Basevi · Ales Leonardis · Bill Freeman · Josh Tenenbaum · Jiajun Wu -
2018 Poster: Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding »
Kexin Yi · Jiajun Wu · Chuang Gan · Antonio Torralba · Pushmeet Kohli · Josh Tenenbaum -
2018 Poster: 3D-Aware Scene Manipulation via Inverse Graphics »
Shunyu Yao · Tzu Ming Hsu · Jun-Yan Zhu · Jiajun Wu · Antonio Torralba · Bill Freeman · Josh Tenenbaum -
2018 Spotlight: Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding »
Kexin Yi · Jiajun Wu · Chuang Gan · Antonio Torralba · Pushmeet Kohli · Josh Tenenbaum -
2017 Spotlight: Shape and Material from Sound »
Zhoutong Zhang · Qiujia Li · Zhengjia Huang · Jiajun Wu · Josh Tenenbaum · Bill Freeman -
2017 Spotlight: Scene Physics Acquisition via Visual De-animation »
Jiajun Wu · Erika Lu · Pushmeet Kohli · Bill Freeman · Josh Tenenbaum -
2017 Poster: Learning to See Physics via Visual De-animation »
Jiajun Wu · Erika Lu · Pushmeet Kohli · Bill Freeman · Josh Tenenbaum -
2017 Poster: Shape and Material from Sound »
Zhoutong Zhang · Qiujia Li · Zhengjia Huang · Jiajun Wu · Josh Tenenbaum · Bill Freeman -
2017 Poster: MarrNet: 3D Shape Reconstruction via 2.5D Sketches »
Jiajun Wu · Yifan Wang · Tianfan Xue · Xingyuan Sun · Bill Freeman · Josh Tenenbaum -
2017 Poster: Self-Supervised Intrinsic Image Decomposition »
Michael Janner · Jiajun Wu · Tejas Kulkarni · Ilker Yildirim · Josh Tenenbaum -
2016 Workshop: Intuitive Physics »
Adam Lerer · Jiajun Wu · Josh Tenenbaum · Emmanuel Dupoux · Rob Fergus -
2016 Poster: Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling »
Jiajun Wu · Chengkai Zhang · Tianfan Xue · Bill Freeman · Josh Tenenbaum -
2016 Poster: Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks »
Tianfan Xue · Jiajun Wu · Katherine Bouman · Bill Freeman -
2016 Oral: Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks »
Tianfan Xue · Jiajun Wu · Katherine Bouman · Bill Freeman -
2015 Poster: Galileo: Perceiving Physical Object Properties by Integrating a Physics Engine with Deep Learning »
Jiajun Wu · Ilker Yildirim · Joseph Lim · Bill Freeman · Josh Tenenbaum