Timezone: »
A recent trend in generative modeling is building 3D-aware generators from 2D image collections. To induce the 3D bias, such models typically rely on volumetric rendering, which is expensive to employ at high resolutions. Over the past months, more than ten works have addressed this scaling issue by training a separate 2D decoder to upsample a low-resolution image (or a feature tensor) produced from a pure 3D generator. But this solution comes at a cost: not only does it break multi-view consistency (i.e., shape and texture change when the camera moves), but it also learns geometry in low fidelity. In this work, we show that obtaining a high-resolution 3D generator with SotA image quality is possible by following a completely different route of simply training the model patch-wise. We revisit and improve this optimization scheme in two ways. First, we design a location- and scale-aware discriminator to work on patches of different proportions and spatial positions. Second, we modify the patch sampling strategy based on an annealed beta distribution to stabilize training and accelerate the convergence. The resulting model, named EpiGRAF, is an efficient, high-resolution, pure 3D generator, and we test it on four datasets (two introduced in this work) at (256^2) and (512^2) resolutions. It obtains state-of-the-art image quality, high-fidelity geometry and trains ({\approx})2.5 faster than the upsampler-based counterparts. Code/data/visualizations: https://universome.github.io/epigraf.
Author Information
Ivan Skorokhodov (KAUST)
Sergey Tulyakov (Snap Inc)
Yiqun Wang (King Abdullah University of Science and Technology)
Peter Wonka (KAUST)
Related Events (a corresponding poster, oral, or spotlight)
-
2022 Poster: EpiGRAF: Rethinking training of 3D GANs »
Thu. Dec 1st through Fri the 2nd Room Hall J #641
More from the Same Authors
-
2023 Poster: Text-to-Image Diffusion Model on Mobile Devices within Two Seconds »
Yanyu Li · Huan Wang · Qing Jin · Ju Hu · Pavlo Chemerys · Yun Fu · Yanzhi Wang · Sergey Tulyakov · Jian Ren -
2023 Poster: LightSpeed: Lighter and Faster Neural Light Field on Mobile Devices »
Aarush Gupta · Junli Cao · Chaoyang Wang · Jian Ren · Sergey Tulyakov · Ju Hu · László Jeni -
2023 Poster: Autodecoding Latent 3D Diffusion Models »
Evangelos Ntavelis · Aliaksandr Siarohin · Kyle Olszewski · Chaoyang Wang · Luc V Gool · Sergey Tulyakov -
2023 Poster: SLIBO-Net: Floorplan Reconstruction via Slicing Box Representation with Local Geometry Regularization »
Jheng-Wei Su · Kuei-Yu Tung · Chi-Han Peng · Peter Wonka · Hung-Kuo Chu -
2022 Spotlight: HF-NeuS: Improved Surface Reconstruction Using High-Frequency Details »
Yiqun Wang · Ivan Skorokhodov · Peter Wonka -
2022 Spotlight: Lightning Talks 5B-1 »
Devansh Arpit · Xiaojun Xu · Zifan Shi · Ivan Skorokhodov · Shayan Shekarforoush · Zhan Tong · Yiqun Wang · Shichong Peng · Linyi Li · Ivan Skorokhodov · Huan Wang · Yibing Song · David Lindell · Yinghao Xu · Seyed Alireza Moazenipourasil · Sergey Tulyakov · Peter Wonka · Yiqun Wang · Ke Li · David Fleet · Yujun Shen · Yingbo Zhou · Bo Li · Jue Wang · Peter Wonka · Marcus Brubaker · Caiming Xiong · Limin Wang · Deli Zhao · Qifeng Chen · Dit-Yan Yeung -
2022 Spotlight: 3DILG: Irregular Latent Grids for 3D Generative Modeling »
Biao Zhang · Matthias Niessner · Peter Wonka -
2022 Poster: 3DILG: Irregular Latent Grids for 3D Generative Modeling »
Biao Zhang · Matthias Niessner · Peter Wonka -
2022 Poster: HF-NeuS: Improved Surface Reconstruction Using High-Frequency Details »
Yiqun Wang · Ivan Skorokhodov · Peter Wonka -
2022 Poster: Layer Freezing & Data Sieving: Missing Pieces of a Generic Framework for Sparse Training »
Geng Yuan · Yanyu Li · Sheng Li · Zhenglun Kong · Sergey Tulyakov · Xulong Tang · Yanzhi Wang · Jian Ren -
2022 Poster: EfficientFormer: Vision Transformers at MobileNet Speed »
Yanyu Li · Geng Yuan · Yang Wen · Ju Hu · Georgios Evangelidis · Sergey Tulyakov · Yanzhi Wang · Jian Ren -
2019 : Poster Session »
Eduard Gorbunov · Alexandre d'Aspremont · Lingxiao Wang · Liwei Wang · Boris Ginsburg · Alessio Quaglino · Camille Castera · Saurabh Adya · Diego Granziol · Rudrajit Das · Raghu Bollapragada · Fabian Pedregosa · Martin Takac · Majid Jahani · Sai Praneeth Karimireddy · Hilal Asi · Balint Daroczy · Leonard Adolphs · Aditya Rawal · Nicolas Brandt · Minhan Li · Giuseppe Ughi · Orlando Romero · Ivan Skorokhodov · Damien Scieur · Kiwook Bae · Konstantin Mishchenko · Rohan Anil · Vatsal Sharan · Aditya Balu · Chao Chen · Zhewei Yao · Tolga Ergen · Paul Grigas · Chris Junchi Li · Jimmy Ba · Stephen J Roberts · Sharan Vaswani · Armin Eftekhari · Chhavi Sharma -
2019 Poster: First Order Motion Model for Image Animation »
Aliaksandr Siarohin · Stéphane Lathuilière · Sergey Tulyakov · Elisa Ricci · Nicu Sebe