ViHOPE: Visuotactile In-Hand Object 6D Pose Estimation with Shape Completion

In this paper, we present ViHOPE, a framework for estimating the 6D pose of an in-hand object using visuotactile perception. In our framework, we employ a conditional Generative Adversarial Network to complete the shape of an in-hand object based on volumetric representation. This completed shape is then utilized to estimate the 6D pose, demonstrating that our approach outperforms prior methods. We assess the effectiveness of our model by training and testing on a synthetic dataset. In both the visuotactile shape completion task and the visuotactile pose estimation task, our approach outperforms the state-of-the-art by a significant margin. We present our pivotal lesson learned: the value of explicitly completing object shapes. Furthermore, we ablate our framework to confirm gains from explicit shape completion and demonstrate that our framework produces models that are robust to sim-to-real transfer on a real-world robot platform.