Timezone: »
It is a long-standing problem to find effective representations for training reinforcement learning (RL) agents. This paper demonstrates that learning state representations with supervision from Neural Radiance Fields (NeRFs) can improve the performance of RL compared to other learned representations or even low-dimensional, hand-engineered state information. Specifically, we propose to train an encoder that maps multiple image observations to a latent space describing the objects in the scene. The decoder built from a latent-conditioned NeRF serves as the supervision signal to learn the latent space. An RL algorithm then operates on the learned latent space as its state representation. We call this NeRF-RL. Our experiments indicate that NeRF as supervision leads to a latent space better suited for the downstream RL tasks involving robotic object manipulations like hanging mugs on hooks, pushing objects, or opening doors.Video: https://dannydriess.github.io/nerf-rl
Author Information
Danny Driess (TU Berlin)
Ingmar Schubert (Technische Universität Berlin / Learning and Intelligent Systems Group)
Pete Florence (Google)
Yunzhu Li (Stanford University)
Yunzhu Li is an Assistant Professor of Computer Science at the University of Illinois Urbana-Champaign (UIUC). Before joining UIUC, he collaborated with Fei-Fei Li and Jiajun Wu during his Postdoc at Stanford. Yunzhu earned his PhD from MIT under the guidance of Antonio Torralba and Russ Tedrake. His work stands at the intersection of robotics, computer vision, and machine learning, with the goal of helping robots perceive and interact with the physical world as dexterously and effectively as humans do. Yunzhu received the Adobe Research Fellowship and was selected as the First Place Recipient of the Ernst A. Guillemin Master's Thesis Award in Artificial Intelligence and Decision Making at MIT. His research has been published in top journals and conferences, including Nature, NeurIPS, CVPR, and RSS, and featured by major media outlets, including CNN, BBC, The Wall Street Journal, Forbes, The Economist, and MIT Technology Review. He received his bachelor's degree in Computer Science from Peking University and has also spent time at the NVIDIA Robotics Research Lab.
Marc Toussaint (TU Berlin)
More from the Same Authors
-
2021 : 3D Neural Scene Representations for Visuomotor Control »
Yunzhu Li · Shuang Li · Vincent Sitzmann · Pulkit Agrawal · Antonio Torralba -
2021 : 3D Neural Scene Representations for Visuomotor Control »
Yunzhu Li · Shuang Li · Vincent Sitzmann · Pulkit Agrawal · Antonio Torralba -
2023 : Open X-Embodiment: Robotic Learning Datasets and RT-X Models »
Quan Vuong · Ajinkya Jain · Alex Bewley · Alexander Irpan · Alexander Khazatsky · Anant Rai · Anikait Singh · Antonin Raffin · Ayzaan Wahid · Beomjoon Kim · Bernhard Schölkopf · brian ichter · Cewu Lu · Charles Xu · Chelsea Finn · Chenfeng Xu · Cheng Chi · Chenguang Huang · Chuer Pan · Chuyuan Fu · Coline Devin · Danny Driess · Deepak Pathak · Dhruv Shah · Dieter Büchler · Dmitry Kalashnikov · Dorsa Sadigh · Edward Johns · Federico Ceola · Fei Xia · Freek Stulp · Gaoyue Zhou · Gaurav Sukhatme · Gautam Salhotra · Ge Yan · Giulio Schiavi · Hao Su · Hao-Shu Fang · Haochen Shi · Heni Ben Amor · Henrik Christensen · Hiroki Furuta · Homer Walke · Hongjie Fang · Igor Mordatch · Ilija Radosavovic · Isabel Leal · Jacky Liang · Jaehyung Kim · Jan Schneider · Jasmine Hsu · Jeannette Bohg · Jiajun Wu · Jialin Wu · Jianlan Luo · Jiayuan Gu · Jie Tan · Jitendra Malik · Jonathan Tompson · Jonathan Yang · Joseph Lim · João Silvério · Junhyek Han · Kanishka Rao · Karl Pertsch · Karol Hausman · Keegan Go · Keerthana Gopalakrishnan · Ken Goldberg · Kevin Zhang · Keyvan Majd · Krishan Rana · Krishnan Srinivasan · Lawrence Yunliang Chen · Lerrel Pinto · Liam Tan · Lionel Ott · Lisa Lee · Masayoshi TOMIZUKA · Michael Ahn · Mingyu Ding · Mohan Kumar Srirama · Mohit Sharma · Moo J Kim · Nicklas Hansen · Nicolas Heess · Nikhil Joshi · Niko Suenderhauf · Norman Di Palo · Nur Muhammad Shafiullah · Oier Mees · Oliver Kroemer · Pannag Sanketi · Paul Wohlhart · Peng Xu · Pierre Sermanet · Priya Sundaresan · Rafael Rafailov · Ran Tian · Ria Doshi · Roberto Martín-Martín · Russell Mendonca · Rutav Shah · Ryan Hoque · Ryan Julian · Samuel Bustamante · Sean Kirmani · Sergey Levine · Sherry Q Moore · Shikhar Bahl · Shivin Dass · Shuran Song · Sichun Xu · Siddhant Haldar · Simeon Adebola · Simon Guist · Soroush Nasiriany · Stefan Schaal · Stefan Welker · Stephen Tian · Sudeep Dasari · Suneel Belkhale · Takayuki Osa · Tatsuya Harada · Tatsuya Matsushima · Ted Xiao · Tianhe Yu · Tianli Ding · Todor Davchev · Tony Zhao · Trevor Darrell · Vidhi Jain · Vincent Vanhoucke · Wei Zhan · Wenxuan Zhou · Wolfram Burgard · Xi Chen · Xiaolong Wang · Xinghao Zhu · Xuanlin Li · Yao Lu · Yevgen Chebotar · Yifan Zhou · Yifeng Zhu · Yonatan Bisk · Yoonyoung Cho · Youngwoon Lee · Yuchen Cui · Yueh-Hua Wu · Yujin Tang · Yuke Zhu · Yunzhu Li · Yusuke Iwasawa · Yutaka Matsuo · Zhuo Xu · Zichen Cui · Alexander Herzog · Abhishek Padalkar · Acorn Pooley · Anthony Brohan · Ben Burgess-Limerick · Christine Chan · Jeffrey Bingham · Jihoon Oh · Kendra Byrne · Kenneth Oslund · Kento Kawaharazuka · Maximilian Du · Mingtong Zhang · Naoaki Kanazawa · Travis Armstrong · Ying Xu · Yixuan Wang · Jan Peters -
2023 : D$^3$Fields: Dynamic 3D Descriptor Fields for Zero-Shot Generalizable Robotic Manipulation »
Yixuan Wang · Zhuoran Li · Mingtong Zhang · Katherine Driggs-Campbell · Jiajun Wu · Fei-Fei Li · Yunzhu Li -
2023 : D$^3$Fields: Dynamic 3D Descriptor Fields for Zero-Shot Generalizable Robotic Manipulation »
Yixuan Wang · Zhuoran Li · Mingtong Zhang · Katherine Driggs-Campbell · Jiajun Wu · Fei-Fei Li · Yunzhu Li -
2023 : D$^3$Fields: Dynamic 3D Descriptor Fields for Zero-Shot Generalizable Robotic Manipulation »
Yixuan Wang · Zhuoran Li · Mingtong Zhang · Katherine Driggs-Campbell · Jiajun Wu · Fei-Fei Li · Yunzhu Li -
2023 : RoboVQA: Multimodal Long-Horizon Reasoningfor Robotics »
Pierre Sermanet · Tianli Ding · Jeffrey Zhao · Fei Xia · Debidatta Dwibedi · Keerthana Gopalakrishnan · Gabriel Dulac-Arnold · sharath maddineni · Nikhil Joshi · Pete Florence · Wei Han · Robert Baruch · Yao Lu · Suvir Mirchandani · Peng Xu · Pannag Sanketi · Karol Hausman · Izhak Shafran · brian ichter · Yuan Cao -
2023 : RoboVQA: Multimodal Long-Horizon Reasoningfor Robotics »
Pierre Sermanet · Tianli Ding · Jeffrey Zhao · Fei Xia · Debidatta Dwibedi · Keerthana Gopalakrishnan · Gabriel Dulac-Arnold · sharath maddineni · Nikhil Joshi · Pete Florence · Wei Han · Robert Baruch · Yao Lu · Suvir Mirchandani · Peng Xu · Pannag Sanketi · Karol Hausman · Izhak Shafran · brian ichter · Yuan Cao -
2023 : VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models »
Wenlong Huang · Chen Wang · Ruohan Zhang · Yunzhu Li · Jiajun Wu · Fei-Fei Li -
2023 : VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models »
Wenlong Huang · Chen Wang · Ruohan Zhang · Yunzhu Li · Jiajun Wu · Fei-Fei Li -
2023 Poster: 3D-IntPhys: Towards More Generalized 3D-grounded Visual Intuitive Physics under Challenging Scenes »
Haotian Xue · Antonio Torralba · Josh Tenenbaum · Dan Yamins · Yunzhu Li · Hsiao-Yu Tung -
2023 Poster: Model-Based Control with Sparse Neural Dynamics »
Ziang Liu · Genggeng Zhou · Jeff He · Tobia Marcucci · Fei-Fei Li · Jiajun Wu · Yunzhu Li -
2023 Poster: Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents »
Wenlong Huang · Fei Xia · Dhruv Shah · Danny Driess · Andy Zeng · Yao Lu · Pete Florence · Igor Mordatch · Sergey Levine · Karol Hausman · brian ichter -
2022 Poster: ActionSense: A Multimodal Dataset and Recording Framework for Human Activities Using Wearable Sensors in a Kitchen Environment »
Joseph DelPreto · Chao Liu · Yiyue Luo · Michael Foshey · Yunzhu Li · Antonio Torralba · Wojciech Matusik · Daniela Rus -
2021 : 3D Neural Scene Representations for Visuomotor Control »
Yunzhu Li · Shuang Li · Vincent Sitzmann · Pulkit Agrawal · Antonio Torralba -
2021 : Spotlights »
Hager Radi · Krishan Rana · Yunzhu Li · Shuang Li · Gal Leibovich · Guy Jacob · Ruihan Yang -
2021 Poster: Learning to Execute: Efficient Learning of Universal Plan-Conditioned Policies in Robotics »
Ingmar Schubert · Danny Driess · Ozgur S. Oguz · Marc Toussaint -
2020 Poster: Causal Discovery in Physical Systems from Videos »
Yunzhu Li · Antonio Torralba · Anima Anandkumar · Dieter Fox · Animesh Garg -
2020 Poster: Learning Physical Graph Representations from Visual Scenes »
Daniel Bear · Chaofei Fan · Damian Mrowca · Yunzhu Li · Seth Alter · Aran Nayebi · Jeremy Schwartz · Li Fei-Fei · Jiajun Wu · Josh Tenenbaum · Daniel Yamins -
2020 Oral: Learning Physical Graph Representations from Visual Scenes »
Daniel Bear · Chaofei Fan · Damian Mrowca · Yunzhu Li · Seth Alter · Aran Nayebi · Jeremy Schwartz · Li Fei-Fei · Jiajun Wu · Josh Tenenbaum · Daniel Yamins -
2019 : Poster Session #2 »
Yunzhu Li · Peter Meltzer · Jianing Sun · Guillaume SALHA · Marin Vlastelica Pogančić · Chia-Cheng Liu · Fabrizio Frasca · Marc-Alexandre Côté · Vikas Verma · Abdulkadir Celikkanat · Pierluca D'Oro · Priyesh Vijayan · Maria Schuld · Petar Veličković · Kshitij Tayal · Yulong Pei · Hao Xu · Lei Chen · Pengyu Cheng · Ines Chami · Dongkwan Kim · Guilherme Gomes · Lukasz Maziarka · Jessica Hoffmann · Ron Levie · Antonia Gogoglou · Shunwang Gong · Federico Monti · Wenlin Wang · Yan Leng · Salvatore Vivona · Daniel Flam-Shepherd · Chester Holtz · Li Zhang · MAHMOUD KHADEMI · I-Chung Hsieh · Aleksandar Stanić · Ziqiao Meng · Yuhang Jiao -
2017 Poster: InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations »
Yunzhu Li · Jiaming Song · Stefano Ermon