Inverting Retargeting: Humanoid Datasets Remember Their Operators
Abstract
Humanoid teleoperation datasets are growing rapidly, with operator demonstrations retargeted onto a shared robot skeleton to train whole-body controllers and foundation models. We report a surprising property of this pipeline: retargeting normalizes body proportions but preserves operator movement dynamics (joint velocity profiles, ranges of motion, and coordination patterns shaped by the operator's physiology). On BONES-SEED (522 operators, 142K sequences), retargeted Unitree G1 trajectories support gender classification at 96.0% and operator re-identification at 97.2% Top-1; on operators never seen during training, gender holds at 83.4% and age and height regress within ±4.2 yr and ±5.7 cm. Partial correlation analysis reveals emergent, biomechanically interpretable structure: the signals are task-invariant across activity categories and hold across retargeting implementations. We introduce UNVEIL, a skeleton-aware spatiotemporal graph network to measure and interpret this effect, take initial steps toward operator anonymization, and ask the community: as teleoperation datasets scale, what should our data practices be? Code, models, and anonymized trajectories are at our project website https://project-unveil.github.io.