Timezone: »

 
Poster
PerspectiveNet: A Scene-consistent Image Generator for New View Synthesis in Real Indoor Environments
David Novotny · Ben Graham · Jeremy Reizenstein

Thu Dec 12 05:00 PM -- 07:00 PM (PST) @ East Exhibition Hall B + C #94
in Applications -- Computer Vision »

Given a set of a reference RGBD views of an indoor environment, and a new viewpoint, our goal is to predict the view from that location. Prior work on new-view generation has predominantly focused on significantly constrained scenarios, typically involving artificially rendered views of isolated CAD models. Here we tackle a much more challenging version of the problem. We devise an approach that exploits known geometric properties of the scene (per-frame camera extrinsics and depth) in order to warp reference views into the new ones. The defects in the generated views are handled by a novel RGBD inpainting network, PerspectiveNet, that is fine-tuned for a given scene in order to obtain images that are geometrically consistent with all the views in the scene camera system. Experiments conducted on the ScanNet and SceneNet datasets reveal performance superior to strong baselines.

Keywords: Deep Learning · Applications -> Computer Vision; Applications -> Denoising; Deep Learning -> Deep Autoencoders; Deep Learning · Predictive Mod
[ Paper]  [ Slides] 

Author Information

David Novotny (Facebook AI Research)
Ben Graham (Facebook Research)
Jeremy Reizenstein (Facebook AI Research)

More from the Same Authors