Timezone: »

 
Spotlight
Habitat 2.0: Training Home Assistants to Rearrange their Habitat
Andrew Szot · Alexander Clegg · Eric Undersander · Erik Wijmans · Yili Zhao · John Turner · Noah Maestre · Mustafa Mukadam · Devendra Singh Chaplot · Oleksandr Maksymets · Aaron Gokaslan · Vladimír Vondruš · Sameer Dharur · Franziska Meier · Wojciech Galuba · Angel Chang · Zsolt Kira · Vladlen Koltun · Jitendra Malik · Manolis Savva · Dhruv Batra

@

We introduce Habitat 2.0 (H2.0), a simulation platform for training virtual robots in interactive 3D environments and complex physics-enabled scenarios. We make comprehensive contributions to all levels of the embodied AI stack – data, simulation, and benchmark tasks. Specifically, we present: (i) ReplicaCAD: an artist-authored, annotated, reconfigurable 3D dataset of apartments (matching real spaces) with articulated objects (e.g. cabinets and drawers that can open/close); (ii) H2.0: a high-performance physics-enabled 3D simulator with speeds exceeding 25,000 simulation steps per second (850x real-time) on an 8-GPU node, representing 100x speed-ups over prior work; and, (iii) Home Assistant Benchmark (HAB): a suite of common tasks for assistive robots (tidy the house, stock groceries, set the table) that test a range of mobile manipulation capabilities. These large-scale engineering contributions allow us to systematically compare deep reinforcement learning (RL) at scale and classical sense-plan-act (SPA) pipelines in long-horizon structured tasks, with an emphasis on generalization to new objects, receptacles, and layouts. We find that (1) flat RL policies struggle on HAB compared to hierarchical ones; (2) a hierarchy with independent skills suffers from ‘hand-off problems’, and (3) SPA pipelines are more brittle than RL policies.

Author Information

Andrew Szot (Georgia Institute of Technology)
Alexander Clegg (Facebook (FAIR Labs))
Eric Undersander (Facebook)
Erik Wijmans (Georgia Institute of Technology)
Yili Zhao (Facebook AI Research)
John Turner (Facebook)
Noah Maestre (Facebook)
Mustafa Mukadam (Facebook AI Research)
Devendra Singh Chaplot (Carnegie Mellon University)
Oleksandr Maksymets (Facebook AI Research)
Aaron Gokaslan (Cornell University)
Vladimír Vondruš (Magnum Engine)
Sameer Dharur (Georgia Tech)
Franziska Meier (Facebook AI Research)
Wojciech Galuba (Meta AI)
Angel Chang (Simon Fraser University)
Zsolt Kira (Georgia Institute of Techology)
Vladlen Koltun (Apple)
Jitendra Malik (UC Berkeley)
Manolis Savva (Simon Fraser University)
Dhruv Batra (FAIR (Meta) / Georgia Tech)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors