Timezone: »
We propose the Habitat Rearrangement Challenge. Specifically, a virtual robot (Fetch mobile manipulator) is spawned in a previously unseen simulation environment and asked to rearrange objects from initial to desired positions -- picking/placing objects from receptacles (counter, sink, sofa, table), opening/closing containers (drawers, fridges) as necessary. The robot operates entirely from onboard sensing -- head- and arm-mounted RGB-D cameras, proprioceptive joint-position sensors (for the arm), and egomotion sensors (for the mobile base) -- and may not access any privileged state information (no prebuilt maps, no 3D models of rooms or objects, no physically-implausible sensors providing knowledge of mass, friction, articulation of containers). This is a challenging embodied AI task involving embodied perception, mobile manipulation, sequential decision making in long-horizon tasks, and (potentially) deep reinforcement and imitation learning. Developing such embodied intelligent systems is a goal of deep scientific and societal value, including practical applications in home assistant robots.
Thu 1:00 p.m. - 1:15 p.m.
|
Challenge Overview
(
Presentation
)
|
Andrew Szot 🔗 |
Thu 1:15 p.m. - 1:30 p.m.
|
Rearrangement Challenge Team “Multi-skill Mobile Manipulation”
(
Presentation
)
|
Jiayuan Gu 🔗 |
Thu 1:30 p.m. - 1:30 p.m.
|
Rearrangement Challenge Team “UESTC-529”
(
Presentation
)
|
Xuan Rao 🔗 |
Thu 1:40 p.m. - 1:50 p.m.
|
Rearrangement Challenge Team “srye”
(
Presentation
)
|
Sriram Yenamandra 🔗 |
Thu 1:50 p.m. - 2:20 p.m.
|
Speaker Franziska Meier
(
Presentation
)
|
Franziska Meier 🔗 |
Thu 2:20 p.m. - 2:30 p.m.
|
Break
|
🔗 |
Thu 2:30 p.m. - 2:50 p.m.
|
Speaker Naoki Yokoyama
(
Presentation
)
|
Naoki Yokoyama 🔗 |
Thu 2:50 p.m. - 3:20 p.m.
|
Speaker Andy Zeng
(
Presentation
)
|
Andy Zeng 🔗 |
Thu 3:20 p.m. - 3:50 p.m.
|
ObjectNav Challenge Overview
(
Presentation
)
|
Karmesh Yadav 🔗 |
Thu 3:50 p.m. - 4:00 p.m.
|
Closing Remarks
|
🔗 |
-
|
Fifteen-minute Competition Overview Video
(
Overview
)
SlidesLive Video » |
Dhruv Batra · Manolis Savva · Zsolt Kira · Vincent-Pierre Berges · Karmesh Yadav · Angel Chang · Andrew Szot · Alexander Clegg · Aaron Gokaslan 🔗 |
Author Information
Andrew Szot (Georgia Institute of Technology)
Karmesh Yadav (Carnegie)
Alexander Clegg (Facebook (FAIR Labs))
Vincent-Pierre Berges (Facebook AI Research)
Aaron Gokaslan (Cornell)
Angel Chang (Simon Fraser University)
Manolis Savva (Simon Fraser University)
Zsolt Kira (Georgia Institute of Techology)
Dhruv Batra (FAIR (Meta) / Georgia Tech)
More from the Same Authors
-
2021 Spotlight: Habitat 2.0: Training Home Assistants to Rearrange their Habitat »
Andrew Szot · Alexander Clegg · Eric Undersander · Erik Wijmans · Yili Zhao · John Turner · Noah Maestre · Mustafa Mukadam · Devendra Singh Chaplot · Oleksandr Maksymets · Aaron Gokaslan · Vladimír Vondruš · Sameer Dharur · Franziska Meier · Wojciech Galuba · Angel Chang · Zsolt Kira · Vladlen Koltun · Jitendra Malik · Manolis Savva · Dhruv Batra -
2021 Spotlight: A Geometric Perspective towards Neural Calibration via Sensitivity Decomposition »
Junjiao Tian · Dylan Yung · Yen-Chang Hsu · Zsolt Kira -
2021 : Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI »
Santhosh Kumar Ramakrishnan · Aaron Gokaslan · Erik Wijmans · Oleksandr Maksymets · Alexander Clegg · John Turner · Eric Undersander · Wojciech Galuba · Andrew Westbury · Angel Chang · Manolis Savva · Yili Zhao · Dhruv Batra -
2021 : Exploring Covariate and Concept Shift for Out-of-Distribution Detection »
Junjiao Tian · Yen-Chang Hsu · Yilin Shen · Hongxia Jin · Zsolt Kira -
2022 : Fifteen-minute Competition Overview Video »
Dhruv Batra · Manolis Savva · Zsolt Kira · Vincent-Pierre Berges · Karmesh Yadav · Angel Chang · Andrew Szot · Alexander Clegg · Aaron Gokaslan -
2023 Poster: DAMEX: Dataset-aware Mixture-of-Experts for visual understanding of mixture-of-datasets »
Yash Jain · Harkirat Behl · Zsolt Kira · Vibhav Vineet -
2023 Poster: Fast Trainable Projection for Robust Fine-tuning »
Junjiao Tian · Yen-Cheng Liu · James S Smith · Zsolt Kira -
2023 Poster: Training Energy-Based Normalizing Flow with Score-Matching Objectives »
Chen-Hao Chao · Wei-Fang Sun · Yen-Chang Hsu · Zsolt Kira · Chun-Yi Lee -
2023 Poster: Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence? »
Arjun Majumdar · Karmesh Yadav · Sergio Arnaud · Jason Yecheng Ma · Claire Chen · Sneha Silwal · Aryan Jain · Vincent-Pierre Berges · Tingfan Wu · Jay Vakil · Pieter Abbeel · Jitendra Malik · Dhruv Batra · Yixin Lin · Oleksandr Maksymets · Aravind Rajeswaran · Franziska Meier -
2023 Competition: The HomeRobot Open Vocabulary Mobile Manipulation Challenge »
Sriram Yenamandra · Arun Ramachandran · Mukul Khanna · Karmesh Yadav · Devendra Singh Chaplot · Gunjan Chhablani · Alexander Clegg · Theophile Gervet · Vidhi Jain · Ruslan Partsey · Ram Ramrakhya · Andrew Szot · Austin Wang · Tsung-Yen Yang · Aaron Edsinger · Charles Kemp · Binit Shah · Zsolt Kira · Dhruv Batra · Roozbeh Mottaghi · Yonatan Bisk · Chris Paxton -
2022 : On the Surprising Effectiveness of Transformers in Low-Labeled Video Recognition »
Farrukh Rahman · Ömer Mubarek · Zsolt Kira -
2022 Spotlight: Lightning Talks 6A-4 »
Xiu-Shen Wei · Konstantina Dritsa · Guillaume Huguet · ABHRA CHAUDHURI · Zhenbin Wang · Kevin Qinghong Lin · Yutong Chen · Jianan Zhou · Yongsen Mao · Junwei Liang · Jinpeng Wang · Mao Ye · Yiming Zhang · Aikaterini Thoma · H.-Y. Xu · Daniel Sumner Magruder · Enwei Zhang · Jianing Zhu · Ronglai Zuo · Massimiliano Mancini · Hanxiao Jiang · Jun Zhang · Fangyun Wei · Faen Zhang · Ioannis Pavlopoulos · Zeynep Akata · Xiatian Zhu · Jingfeng ZHANG · Alexander Tong · Mattia Soldan · Chunhua Shen · Yuxin Peng · Liuhan Peng · Michael Wray · Tongliang Liu · Anjan Dutta · Yu Wu · Oluwadamilola Fasina · Panos Louridas · Angel Chang · Manik Kuchroo · Manolis Savva · Shujie LIU · Wei Zhou · Rui Yan · Gang Niu · Liang Tian · Bo Han · Eric Z. XU · Guy Wolf · Yingying Zhu · Brian Mak · Difei Gao · Masashi Sugiyama · Smita Krishnaswamy · Rong-Cheng Tu · Wenzhe Zhao · Weijie Kong · Chengfei Cai · WANG HongFa · Dima Damen · Bernard Ghanem · Wei Liu · Mike Zheng Shou -
2022 Spotlight: MultiScan: Scalable RGBD scanning for 3D environments with articulated objects »
Yongsen Mao · Yiming Zhang · Hanxiao Jiang · Angel Chang · Manolis Savva -
2022 : ObjectNav Challenge Overview »
Karmesh Yadav -
2022 : Challenge Overview »
Andrew Szot -
2022 : Panel »
Tyler Hayes · Tinne Tuytelaars · Subutai Ahmad · João Sacramento · Zsolt Kira · Hava Siegelmann · Christopher Summerfield -
2022 Poster: Polyhistor: Parameter-Efficient Multi-Task Adaptation for Dense Vision Tasks »
Yen-Cheng Liu · CHIH-YAO MA · Junjiao Tian · Zijian He · Zsolt Kira -
2022 Poster: VER: Scaling On-Policy RL Leads to the Emergence of Navigation in Embodied Rearrangement »
Erik Wijmans · Irfan Essa · Dhruv Batra -
2022 Poster: SoundSpaces 2.0: A Simulation Platform for Visual-Acoustic Learning »
Changan Chen · Carl Schissler · Sanchit Garg · Philip Kobernik · Alexander Clegg · Paul Calamia · Dhruv Batra · Philip Robinson · Kristen Grauman -
2022 Poster: MultiScan: Scalable RGBD scanning for 3D environments with articulated objects »
Yongsen Mao · Yiming Zhang · Hanxiao Jiang · Angel Chang · Manolis Savva -
2022 Poster: ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddings »
Arjun Majumdar · Gunjan Aggarwal · Bhavika Devnani · Judy Hoffman · Dhruv Batra -
2021 : Habitat 2.0: Training Home Assistants to Rearrange their Habitat »
Andrew Szot · Alexander Clegg · Eric Undersander · Erik Wijmans · Yili Zhao · Noah Maestre · Mustafa Mukadam · Oleksandr Maksymets · Aaron Gokaslan · Sameer Dharur · Franziska Meier · Wojciech Galuba · Angel Chang · Zsolt Kira · Vladlen Koltun · Jitendra Malik · Manolis Savva · Dhruv Batra -
2021 : Habitat 2.0: Training Home Assistants to Rearrange their Habitat »
Andrew Szot · Alexander Clegg · Eric Undersander · Erik Wijmans · Yili Zhao · Noah Maestre · Mustafa Mukadam · Oleksandr Maksymets · Aaron Gokaslan · Sameer Dharur · Franziska Meier · Wojciech Galuba · Angel Chang · Zsolt Kira · Vladlen Koltun · Jitendra Malik · Manolis Savva · Dhruv Batra -
2021 Poster: SOAT: A Scene- and Object-Aware Transformer for Vision-and-Language Navigation »
Abhinav Moudgil · Arjun Majumdar · Harsh Agrawal · Stefan Lee · Dhruv Batra -
2021 Poster: A Geometric Perspective towards Neural Calibration via Sensitivity Decomposition »
Junjiao Tian · Dylan Yung · Yen-Chang Hsu · Zsolt Kira -
2021 Poster: Habitat 2.0: Training Home Assistants to Rearrange their Habitat »
Andrew Szot · Alexander Clegg · Eric Undersander · Erik Wijmans · Yili Zhao · John Turner · Noah Maestre · Mustafa Mukadam · Devendra Singh Chaplot · Oleksandr Maksymets · Aaron Gokaslan · Vladimír Vondruš · Sameer Dharur · Franziska Meier · Wojciech Galuba · Angel Chang · Zsolt Kira · Vladlen Koltun · Jitendra Malik · Manolis Savva · Dhruv Batra -
2021 Poster: Generalizable Imitation Learning from Observation via Inferring Goal Proximity »
Youngwoon Lee · Andrew Szot · Shao-Hua Sun · Joseph Lim -
2020 Poster: Posterior Re-calibration for Imbalanced Datasets »
Junjiao Tian · Yen-Cheng Liu · Nathaniel Glaser · Yen-Chang Hsu · Zsolt Kira -
2020 Poster: Look-ahead Meta Learning for Continual Learning »
Gunshi Gupta · Karmesh Yadav · Liam Paull -
2020 Oral: Look-ahead Meta Learning for Continual Learning »
Gunshi Gupta · Karmesh Yadav · Liam Paull -
2020 Poster: MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigation »
Saim Wani · Shivansh Patel · Unnat Jain · Angel Chang · Manolis Savva -
2018 : Lunch & Posters »
Haytham Fayek · German Parisi · Brian Xu · Pramod Kaushik Mudrakarta · Sophie Cerf · Sarah Wassermann · Davit Soselia · Rahaf Aljundi · Mohamed Elhoseiny · Frantzeska Lavda · Kevin J Liang · Arslan Chaudhry · Sanmit Narvekar · Vincenzo Lomonaco · Wesley Chung · Michael Chang · Ying Zhao · Zsolt Kira · Pouya Bashivan · Banafsheh Rafiee · Oleksiy Ostapenko · Andrew Jones · Christos Kaplanis · Sinan Kalkan · Dan Teng · Xu He · Vincent Liu · Somjit Nath · Sungsoo Ahn · Ting Chen · Shenyang Huang · Yash Chandak · Nathan Sprague · Martin Schrimpf · Tony Kendall · Jonathan Richard Schwarz · Michael Li · Yunshu Du · Yen-Chang Hsu · Samira Abnar · Bo Wang -
2018 Workshop: Visually grounded interaction and language »
Florian Strub · Harm de Vries · Erik Wijmans · Samyak Datta · Ethan Perez · Mateusz Malinowski · Stefan Lee · Peter Anderson · Aaron Courville · Jeremie MARY · Dhruv Batra · Devi Parikh · Olivier Pietquin · Chiori HORI · Tim Marks · Anoop Cherian -
2017 : Morning panel discussion »
Jürgen Schmidhuber · Noah Goodman · Anca Dragan · Pushmeet Kohli · Dhruv Batra -
2017 : Invited Talk 2 »
Dhruv Batra -
2017 Poster: Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model »
Jiasen Lu · Anitha Kannan · Jianwei Yang · Devi Parikh · Dhruv Batra