Unsupervised monocular depth estimation techniques have demonstrated encouraging results but typically assume that the scene is static. These techniques suffer when trained on dynamical scenes, where apparent object motion can equally be explained by hypothesizing the object's independent motion, or by altering its depth. This ambiguity causes depth estimators to predict erroneous depth for moving objects. To resolve this issue, we introduce Dynamo-Depth, an unifying approach that disambiguates dynamical motion by jointly learning monocular depth, 3D independent flow field, and motion segmentation from unlabeled monocular videos. Specifically, we offer our key insight that a good initial estimation of motion segmentation is sufficient for jointly learning depth and independent motion despite the fundamental underlying ambiguity. Our proposed method achieves state-of-the-art performance on monocular depth estimation on Waymo Open and nuScenes Dataset with significant improvement in the depth of moving objects. Code and additional results are available at https://dynamo-depth.github.io.
Yihong Sun (Cornell University)
I am a CS PhD Student at Cornell University, advised by Prof. Bharath Hariharan. Previously, I obtained my Bachelor’s degree from Johns Hopkins University where I worked with Bloomberg Distinguished Prof. Alan Yuille and Dr. Adam Kortylewski. My work is supported by the NSF GRFP and my research interests are computer vision and machine learning, especially in building vision algorithms that can learn from little supervision and generalize to unseen domains.
Bharath Hariharan (Cornell University)
More from the Same Authors
2023 Poster: Emergent Correspondence from Image Diffusion »
Luming Tang · Menglin Jia · Qianqian Wang · Cheng Perng Phoo · Bharath Hariharan
2023 Poster: Reward Finetuning for Faster and More Accurate Unsupervised Object Discovery »
Katie Luo · Zhenzhen Liu · Xiangyu Chen · Yurong You · Sagie Benaim · Cheng Perng Phoo · Mark Campbell · Wen Sun · Bharath Hariharan · Kilian Weinberger
2022 Poster: Unsupervised Adaptation from Repeated Traversals for Autonomous Driving »
Yurong You · Cheng Perng Phoo · Katie Luo · Travis Zhang · Wei-Lun (Harry) Chao · Bharath Hariharan · Mark Campbell · Kilian Weinberger
2022 Poster: Change Event Dataset for Discovery from Spatio-temporal Remote Sensing Imagery »
Utkarsh Mall · Bharath Hariharan · Kavita Bala
2022 Poster: Polynomial Neural Fields for Subband Decomposition and Manipulation »
Guandao Yang · Sagie Benaim · Varun Jampani · Kyle Genova · Jonathan Barron · Thomas Funkhouser · Bharath Hariharan · Serge Belongie