Towards Scalable Egocentric HOI for Humanoids: Benchmarking Whole-Body Dexterous Interaction with Tactile Prediction
Abstract
Humanoid whole-body loco-manipulation requires scalable and physically reliable demonstrations, yet existing human-object interaction datasets lack direct contact supervision and egocentric observability, often resulting in physically inconsistent interactions. We present DeXHOI, a multimodal egocentric HOI benchmark for whole-body dexterous interaction, capturing synchronized body motion, object trajectories, tactile sensing, and egocentric RGB-D across diverse spatial layouts and long-horizon activities. To improve physical consistency, we introduce a tactile-guided refinement pipeline that leverages measured pressure to correct hand-object interactions while preserving global motion fidelity. We further establish an Egocentric HOI Recovery Benchmark for contact-aware reconstruction from first-person observations, along with EgoTacti, a tactile prediction model. Together, EgoTacti provides a scalable and physically grounded foundation for learning humanoid whole-body dexterous interaction, pointing toward low-cost egocentric data collection for embodied intelligence.