Workshop: Differentiable computer vision, graphics, and physics in machine learning

Krishna Jatavallabhula, Kelsey Allen, Victoria Dean, Johanna Hansen, Shuran Song, Florian Shkurti, Liam Paull, Derek Nowrouzezahrai, Josh Tenenbaum

2020-12-11T06:45:00-08:00 - 2020-12-11T14:30:00-08:00
Abstract: “Differentiable programs” are parameterized programs that allow themselves to be rewritten by gradient-based optimization. They are ubiquitous in modern-day machine learning. Recently, explicitly encoding our knowledge of the rules of the world in the form of differentiable programs has become more popular. In particular, differentiable realizations of well-studied processes such as physics, rendering, projective geometry, optimization to name a few, have enabled the design of several novel learning techniques. For example, many approaches have been proposed for unsupervised learning of depth estimation from unlabeled videos. Differentiable 3D reconstruction pipelines have demonstrated the potential for task-driven representation learning. A number of differentiable rendering approaches have been shown to enable single-view 3D reconstruction and other inverse graphics tasks (without requiring any form of 3D supervision). Differentiable physics simulators are being built to perform physical parameter estimation from video or for model-predictive control. While these advances have largely occurred in isolation, recent efforts have attempted to bridge the gap between the aforementioned areas. Narrowing the gaps between these otherwise isolated disciplines holds tremendous potential to yield new research directions and solve long-standing problems, particularly in understanding and reasoning about the 3D world.

Hence, we propose the “first workshop on differentiable computer vision, graphics, and physics in machine learning” with the aim of:
1. Narrowing the gap and fostering synergies between the computer vision, graphics, physics, and machine learning communities
2. Debating the promise and perils of differentiable methods, and identifying challenges that need to be overcome
3. Raising awareness about these techniques to the larger ML community
4. Discussing the broader impact of such techniques, and any ethical implications thereof.

Video

Chat

Chat is not available.

Schedule

2020-12-11T06:45:00-08:00 - 2020-12-11T07:00:00-08:00
Opening remarks
Krishna Jatavallabhula, Kelsey Allen, Johanna Hansen, Victoria Dean
2020-12-11T07:00:00-08:00 - 2020-12-11T07:30:00-08:00
Sanja Fidler
Sanja Fidler, Krishna Jatavallabhula, Kelsey Allen
2020-12-11T07:30:00-08:00 - 2020-12-11T08:00:00-08:00
Andrea Tagliasacchi
Andrea Tagliasacchi, Krishna Jatavallabhula, Kelsey Allen
2020-12-11T08:02:00-08:00 - 2020-12-11T08:31:00-08:00
Peter Battaglia
Peter Battaglia, Kelsey Allen, Krishna Jatavallabhula
2020-12-11T08:32:00-08:00 - 2020-12-11T08:37:00-08:00
Peter Battaglia - Q&A
2020-12-11T08:38:00-08:00 - 2020-12-11T08:53:00-08:00
Camillo Jose Taylor
CJ Taylor, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
2020-12-11T08:54:00-08:00 - 2020-12-11T08:59:00-08:00
Camillo Jose Taylor - Q&A
2020-12-11T09:00:00-08:00 - 2020-12-11T09:13:00-08:00
Oral 01: phiflow - A differentiable PDE solving framework for deep learning via physical simulations
Nils Thuerey, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
2020-12-11T09:13:00-08:00 - 2020-12-11T09:23:00-08:00
Oral 02: Differentiable HDR image synthesis using multi-exposure images
Jung Hee Kim, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
2020-12-11T09:23:00-08:00 - 2020-12-11T09:35:00-08:00
Oral 03: DELUCA - Differentiable control library - environments, methods, and benchmarking
Paula Gradu, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
2020-12-11T09:35:00-08:00 - 2020-12-11T09:44:00-08:00
Oral 04: Blendshape-augmented facial action units detection
Zijun Cui, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
2020-12-11T09:44:00-08:00 - 2020-12-11T09:57:00-08:00
Oral 05: Inverse articulated-body dynamics from video via variational sequential Monte-Carlo
Dan Biderman, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
2020-12-11T09:58:00-08:00 - 2020-12-11T10:08:00-08:00
Contributed Talk - Q&A
2020-12-11T10:10:00-08:00 - 2020-12-11T10:35:00-08:00
Bethany Lusch
Bethany Lusch, Krishna Jatavallabhula, Kelsey Allen
2020-12-11T10:36:00-08:00 - 2020-12-11T10:41:00-08:00
Bethany Lusch - Q&A
2020-12-11T10:42:00-08:00 - 2020-12-11T11:13:00-08:00
Yuanming Hu
Yuanming Hu, Kelsey Allen, Krishna Jatavallabhula
2020-12-11T11:14:00-08:00 - 2020-12-11T11:19:00-08:00
Yuanming Hu - Q&A
2020-12-11T11:20:00-08:00 - 2020-12-11T11:39:00-08:00
Georgia Gkioxari
Georgia Gkioxari, Kelsey Allen, Krishna Jatavallabhula
2020-12-11T11:40:00-08:00 - 2020-12-11T11:45:00-08:00
Georgia Gkioxari - Q&A
2020-12-11T11:46:00-08:00 - 2020-12-11T12:16:00-08:00
Ming Lin
Ming Lin, Krishna Jatavallabhula, Kelsey Allen
2020-12-11T12:16:00-08:00 - 2020-12-11T13:15:00-08:00
Panel Discussion
2020-12-11T13:15:00-08:00 - 2020-12-11T14:30:00-08:00
Poster session (gather.town)
Poster 14: MSR-Net: Multi-scale relighting network for one-to-one relighting
Nisarg Shah, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
Poster 08: Solving physics puzzles by reasoning about paths
Augustin Harter, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
Poster 06: Instance-wise depth and motion learning from monocular videos
Seokju Lee, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
Poster 13: End-to-end differentiable 6DoF object pose estimation with local and global constraints
Anshul Gupta, Joydeep Medhi, Aratrik Chattopadhyay, Vikram Gupta, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
Poster 10: Tractable loss function and color image generation of multinary restricted Boltzmann machine
Juno Hwang, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
Poster 15: Towards end-to-end training of proposal-based 3D human pose estimation
Sirdaniel Ajisafe, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
Poster 01: Using differentiable physics for self-supervised assimilation of chaotic dynamical systems
Michael McCabe, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
Poster 05: Inverse graphics GAN
Sebastian Lunz, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
Poster 11: Differentiable path tracing by regularizing discontinuities
Peter Quinn, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
Poster 03: Differentiable data augmentation with Kornia
Jian Shi, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
Poster 02: Learned equivariant rendering without transformation supervision
Cinjon Resnick, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
Poster 07: System level differentiable simulation of radio access networks
Dmitriy Rivkin, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
Poster 12: Spring-Rod system identification via differentiable physics engine
Kun Wang, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
Poster 09: Sparse-input neural network augmentations for differentiable simulators
Eric Heiden, David Millard, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean
Poster 04: Semantic adversarial robustness with differentiable ray-tracing
Rahul Venkatesh, Krishna Jatavallabhula, Kelsey Allen, Victoria Dean