Timezone: »

 
Poster
GAUDI: A Neural Architect for Immersive 3D Scene Generation
Miguel Angel Bautista · Pengsheng Guo · Samira Abnar · Walter Talbott · Alexander Toshev · Zhuoyuan Chen · Laurent Dinh · Shuangfei Zhai · Hanlin Goh · Daniel Ulbricht · Afshin Dehghan · Joshua Susskind

Thu Dec 01 09:00 AM -- 11:00 AM (PST) @ Hall J #427
in Poster Session 5 »

We introduce GAUDI, a generative model capable of capturing the distribution of complex and realistic 3D scenes that can be rendered immersively from a moving camera. We tackle this challenging problem with a scalable yet powerful approach, where we first optimize a latent representation that disentangles radiance fields and camera poses. This latent representation is then used to learn a generative model that enables both unconditional and conditional generation of 3D scenes. Our model generalizes previous works that focus on single objects by removing the assumption that the camera pose distribution can be shared across samples. We show that GAUDI obtains state-of-the-art performance in the unconditional generative setting across multiple datasets and allows for conditional generation of 3D scenes given conditioning variables like sparse image observations or text that describes the scene.

Keywords: generative modeling · 3D · radiance fields
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Miguel Angel Bautista (Apple)
Pengsheng Guo (Apple)
Samira Abnar (University of Amsterdam)
Walter Talbott (Apple)
Alexander Toshev (Google)
Zhuoyuan Chen (Baidu Research)

I'm a senior researcher in Baidu IDL, supervised by Wei Xu and Andrew Ng. My research focus on deep learning, one-shot learning, computer vision and meta-learning.

Laurent Dinh (Apple)
Laurent Dinh

Laurent Dinh is a machine learning researcher. He has been mainly working on deep generative models, variational models and most notably flow-based models, and (mis)understanding of machine learning methods. He also follows work in connected areas like probabilistic modelling, approximate inference, optimization, learning of discrete structures, and sociotechnical studies of machine learning. He obtained his PhD in deep learning at Mila (Montréal, Canada), under the supervision of Yoshua Bengio. Prior to that he studied at École Centrale de Paris (Paris, France) in applied mathematics and at ÉNS Paris-Saclay (Paris, France) in machine learning and computer vision. He worked in the machine learning group led by Nando de Freitas both at UBC (Vancouver, Canada) and DeepMind (London, United Kingdom), and also at Google Brain (Mountain View, US), under the supervision of Samy Bengio.

Shuangfei Zhai (Apple)
Hanlin Goh (Apple Inc.)
Daniel Ulbricht (Apple Inc.)
Afshin Dehghan (Apple)

PhD in Computer Science Deep Learning Enthusiast Machine Learning Manager @Apple

Joshua Susskind (Apple Inc.)

I was an undergraduate in Cognitive Science at UCSD from 1995-2003 (with some breaks). Then I earned a PhD from UofT in machine learning and cognitive neuroscience, with Dr. Geoff Hinton and Dr. Adam Anderson. Following grad school I moved to UCSD for a post-doctoral position. Before coming to Apple I co-founded Emotient in 2012 and led the deep learning effort for facial expression and demographics recognition. Since joining Apple, I led the Face ID neural network team responsible for face recognition, and then started a machine learning research group within the hardware organization focused on fundamental ML technology.

More from the Same Authors