3D Deep Learning
Fisher Yu · Joseph Lim · Matthew D Fisher · Qixing Huang · Jianxiong Xiao

Fri Dec 9th 08:00 AM -- 06:30 PM @ Room 115
Event URL: »

Deep learning is proven to be a powerful tool to build models for language (one-dimensional) and image (two-dimensional) understanding. Tremendous efforts have been devoted into these areas, however, it is still at the early stage to apply deep learning to 3D data, despite their great research values and broad real-world applications. In particular, existing methods poorly serve the three-dimensional data that drives a broad range of critical applications such as augmented reality, autonomous driving, graphics, robotics, medical imaging, neuroscience, and scientific simulations. These problems have drawn attention of researchers in different fields such as neuroscience, computer vision and graphics.

Different from text or images that can be naturally represented as 1D or 2D arrays, 3D data have multiple representation candidates, such as volumes, polygonal meshes, multi-views renderings, depth maps, and point clouds. Coupled with these representations are the myriad 3D learning problems, such as object recognition, scene layout estimation, compositional structure parsing, novel view synthesis, model completion and hallucination, etc. 3D data opens new and vast research space, which naturally calls for interdisciplinary expertise ranging from Computer Vision, Computer Graphics, to Machine Learning.

The goal of this workshop is to foster interdisciplinary communication of researchers working on 3D data (Computer Vision and Computer Graphics), so that more attention of broader community can be drawn to 3D deep learning problems. Through those studies, new ideas and discoveries are expected to emerge, which can inspire advances in related fields.

This workshop is composed of invited talks, oral presentations of outstanding submissions and a poster session to showcase the state-of-the-art results in the topic. In particular, a panel discussion among leading researchers in the field is planned, so as to provide a common playground for inspiring discussions and stimulating debates.

We aim to build a venue for publishing original research results in 3D deep learning, as well as exhibiting the latest trends and ideas. To be specific, we are interested in the following topics using 3D deep learning methods:

3D object detection from depth images and videos
3D scene understanding
3D spatial understanding from 2D images
3D shape classification and segmentation
3D mapping and reconstruction
Learning 3D geometrical properties and representations
Analysis of 3D medical and biological imaging data

We accept two tracks of submissions to the workshop on those topics: paper (6 - 9 pages) and extended abstract (4 pages). We are inviting researchers of related fields to join the workshop program committee to review the submissions. All the submissions will follow NIPS main conference paper style. The paper will be reviewed in double-blind form from three researchers in the workshop program committee. High quality papers will be selected for oral presentation. The abstracts will be reviewed by the workshop committee in single-blind fashion. Accepted submissions will either be presented as posters or talks at the workshop. We encourage submissions of works that has been previously published or is to be presented in the main conference.

08:30 AM Oral Presentation (Introduction)
08:45 AM Learning 3D representations, disparity estimation, and structure from motion (Talk) Thomas Brox
10:30 AM Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling (Oral Presentation)
11:00 AM FusionNet: 3D Object Classification Using Multiple Data Representations (Oral Presentation)
02:30 PM Invited Talk by Abhinav Gupta (Invited Talk)
03:00 PM Invited Talk by Michael Bronstein (Invited Talk)
04:00 PM Invited Talk (Talk) Thomas Funkhouser
04:30 PM Generative and Discriminative Voxel Modeling with Convolutional Neural Networks (Oral Presentation)
05:00 PM Sparse 3D Convolutional Neural Networks for Large-Scale Shape Retrieval (Oral Presentation)

Author Information

Fisher Yu (Princeton University)
Joseph Lim (MIT)
Matt D Fisher (Stanford University)
Qixing Huang (Toyota Technological Institute at Chicago)
Jianxiong Xiao (Princeton University)

More from the Same Authors