Exploring the Limits of Compositional Generalization in Vision-Language-Action Manipulation
Abstract
Vision Language Action (VLA) models can execute isolated atomic skills, but practical long horizon manipulation requires recombining these skills into new compositions without demonstrations for every skill combination. The direct full instruction route treats skill chaining as long horizon imitation. In our controlled study, full instruction training on limited skill combination sequences reaches 40.0% ordered success on covered combinations but only 7.7% on recombination, suggesting memorization of sequence templates rather than robust skill recombination. We therefore introduce Hierarchical Subtask Execution (HSE), a diagnostic execution framework that exposes skill recombination through a shared modular interface over the planner, trigger, and policy, enabling long horizon failures to be decomposed into separable modes. To make this diagnosis measurable, we introduce RoboCombine, a benchmark for skill recombination without combined skill training trajectories, with metrics for ordered success and transition conversion across subtask boundaries. RoboCombine reveals that atomic competence alone does not ensure ordered execution. Failures concentrate at transitions where downstream subtasks must launch from handoff states produced by preceding subtasks rather than canonical atomic starts. We identify this mismatch as the start state gap. To address it, we introduce Pose Robust Task Initialization (PRTI), a lightweight atomic task post training strategy that broadens start state coverage beyond canonical starts. Under oracle timed HSE on RoboCombine, adding PRTI raises ordered success from 2.11% to 26.32% and transition conversion from 2.63% to 60.53%, without combined skill training trajectories. Together, these results show that combinatorial generalization requires handoff robust atomic skills that convert first stage progress into reliable downstream execution, not merely decomposition into known subtasks. We will release RoboCombine and HSE code to enable reproducible diagnosis of atomic skill recombination failures.