Timezone: »
We propose learning graph representations from 2D feature maps for visual recognition. Our method draws inspiration from region based recognition, and learns to transform a 2D image into a graph structure. The vertices of the graph define clusters of pixels ("regions"), and the edges measure the similarity between these clusters in a feature space. Our method further learns to propagate information across all vertices on the graph, and is able to project the learned graph representation back into 2D grids. Our graph representation facilitates reasoning beyond regular grids and can capture long range dependencies among regions. We demonstrate that our model can be trained from end-to-end, and is easily integrated into existing networks. Finally, we evaluate our method on three challenging recognition tasks: semantic segmentation, object detection and object instance segmentation. For all tasks, our method outperforms state-of-the-art methods.
Author Information
Yin Li (University of Wisconsin-Madison)
Abhinav Gupta (Facebook AI Research/CMU)
More from the Same Authors
-
2021 : RB2: Robotic Manipulation Benchmarking with a Twist »
Sudeep Dasari · Jianren Wang · Joyce Hong · Shikhar Bahl · Yixin Lin · Austin Wang · Abitha Thankaraj · Karanbir Chahal · Berk Calli · Saurabh Gupta · David Held · Lerrel Pinto · Deepak Pathak · Vikash Kumar · Abhinav Gupta -
2021 : KitchenShift: Evaluating Zero-Shot Generalization of Imitation-Based Policy Learning Under Domain Shifts »
Eliot Xing · Abhinav Gupta · Samantha Powers · Victoria Dean -
2022 : Train Offline, Test Online: A Real Robot Learning Benchmark »
Gaoyue Zhou · Victoria Dean · Mohan Kumar Srirama · Aravind Rajeswaran · Jyothish Pari · Kyle Hatch · Aryan Jain · Tianhe Yu · Pieter Abbeel · Lerrel Pinto · Chelsea Finn · Abhinav Gupta -
2022 : Offline Reinforcement Learning on Real Robot with Realistic Data Sources »
Gaoyue Zhou · Liyiming Ke · Siddhartha Srinivasa · Abhinav Gupta · Aravind Rajeswaran · Vikash Kumar -
2022 : Train Offline, Test Online: A Real Robot Learning Benchmark »
Gaoyue Zhou · Victoria Dean · Mohan Kumar Srirama · Aravind Rajeswaran · Jyothish Pari · Kyle Hatch · Aryan Jain · Tianhe Yu · Pieter Abbeel · Lerrel Pinto · Chelsea Finn · Abhinav Gupta -
2022 : Train Offline, Test Online: A Real Robot Learning Benchmark »
Gaoyue Zhou · Victoria Dean · Mohan Kumar Srirama · Aravind Rajeswaran · Jyothish Pari · Kyle Hatch · Aryan Jain · Tianhe Yu · Pieter Abbeel · Lerrel Pinto · Chelsea Finn · Abhinav Gupta -
2022 : Offline Reinforcement Learning on Real Robot with Realistic Data Sources »
Gaoyue Zhou · Liyiming Ke · Siddhartha Srinivasa · Abhinav Gupta · Aravind Rajeswaran · Vikash Kumar -
2022 : Train Offline, Test Online: A Real Robot Learning Benchmark »
Gaoyue Zhou · Victoria Dean · Mohan Kumar Srirama · Aravind Rajeswaran · Jyothish Pari · Kyle Hatch · Aryan Jain · Tianhe Yu · Pieter Abbeel · Lerrel Pinto · Chelsea Finn · Abhinav Gupta -
2022 : Train Offline, Test Online: A Real Robot Learning Benchmark »
Gaoyue Zhou · Victoria Dean · Mohan Kumar Srirama · Aravind Rajeswaran · Jyothish Pari · Kyle Hatch · Aryan Jain · Tianhe Yu · Pieter Abbeel · Lerrel Pinto · Chelsea Finn · Abhinav Gupta -
2022 : Multi-modal 3D Human Pose Estimation using mmWave, RGB-D, and Inertial Sensors »
Sizhe An · Yin Li · Umit Ogras -
2022 Poster: mRI: Multi-modal 3D Human Pose Estimation Dataset using mmWave, RGB-D, and Inertial Sensors »
Sizhe An · Yin Li · Umit Ogras -
2022 Poster: Learning State-Aware Visual Representations from Audible Interactions »
Himangi Mittal · Pedro Morgado · Unnat Jain · Abhinav Gupta -
2021 Oral: Interesting Object, Curious Agent: Learning Task-Agnostic Exploration »
Simone Parisi · Victoria Dean · Deepak Pathak · Abhinav Gupta -
2021 Poster: No RL, No Simulation: Learning to Navigate without Navigating »
Meera Hahn · Devendra Singh Chaplot · Shubham Tulsiani · Mustafa Mukadam · James Rehg · Abhinav Gupta -
2021 Poster: Interesting Object, Curious Agent: Learning Task-Agnostic Exploration »
Simone Parisi · Victoria Dean · Deepak Pathak · Abhinav Gupta -
2020 : QA: Abhinav Gupta »
Abhinav Gupta -
2020 : Invited Talk: Abhinav Gupta »
Abhinav Gupta -
2020 Poster: Neural Dynamic Policies for End-to-End Sensorimotor Learning »
Shikhar Bahl · Mustafa Mukadam · Abhinav Gupta · Deepak Pathak -
2020 Poster: Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases »
Senthil Purushwalkam · Abhinav Gupta -
2020 Spotlight: Neural Dynamic Policies for End-to-End Sensorimotor Learning »
Shikhar Bahl · Mustafa Mukadam · Abhinav Gupta · Deepak Pathak -
2020 Poster: See, Hear, Explore: Curiosity via Audio-Visual Association »
Victoria Dean · Shubham Tulsiani · Abhinav Gupta -
2020 Poster: Object Goal Navigation using Goal-Oriented Semantic Exploration »
Devendra Singh Chaplot · Dhiraj Prakashchand Gandhi · Abhinav Gupta · Russ Salakhutdinov -
2019 Poster: Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller »
Pratyusha Sharma · Deepak Pathak · Abhinav Gupta -
2018 Poster: Hardware Conditioned Policies for Multi-Robot Transfer Learning »
Tao Chen · Adithyavairavan Murali · Abhinav Gupta -
2018 Poster: Robot Learning in Homes: Improving Generalization and Reducing Dataset Bias »
Abhinav Gupta · Adithyavairavan Murali · Dhiraj Prakashchand Gandhi · Lerrel Pinto -
2016 : Invited Talk - Self Supervised Learning of Visual Representations »
Abhinav Gupta -
2016 : Abhinav Gupta »
Abhinav Gupta -
2016 : Abhinav Gupta »
Abhinav Gupta -
2013 Poster: Mid-level Visual Element Discovery as Discriminative Mode Seeking »
Carl Doersch · Abhinav Gupta · Alexei A Efros -
2010 Poster: Estimating Spatial Layout of Rooms using Volumetric Reasoning about Objects and Surfaces »
David C Lee · Abhinav Gupta · Martial Hebert · Takeo Kanade -
2008 Poster: A "Shape Aware" Model for semi-supervised Learning of Objects and its Context »
Abhinav Gupta · Jianbo Shi · Larry Davis -
2008 Spotlight: A "Shape Aware'' Model for semi-supervised Learning of Objects and its Context »
Abhinav Gupta · Jianbo Shi · Larry Davis