Timezone: »
We present Theseus, an efficient application-agnostic open source library for differentiable nonlinear least squares (DNLS) optimization built on PyTorch, providing a common framework for end-to-end structured learning in robotics and vision. Existing DNLS implementations are application specific and do not always incorporate many ingredients important for efficiency. Theseus is application-agnostic, as we illustrate with several example applications that are built using the same underlying differentiable components, such as second-order optimizers, standard costs functions, and Lie groups. For efficiency, Theseus incorporates support for sparse solvers, automatic vectorization, batching, GPU acceleration, and gradient computation with implicit differentiation and direct loss minimization. We do extensive performance evaluation in a set of applications, demonstrating significant efficiency gains and better scalability when these features are incorporated. Project page: https://sites.google.com/view/theseus-ai/
Author Information
Luis Pineda (Facebook AI Research)
Taosha Fan (Northwestern University, Northwestern University)
Maurizio Monge
Shobha Venkataraman (Facebook)
Paloma Sodhi (ASAPP, Inc.)
Ricky T. Q. Chen (FAIR Labs, Meta AI)
Joseph Ortiz (Imperial College London)
Daniel DeTone (Meta)
Austin Wang (CMU, Carnegie Mellon University)
Stuart Anderson (Facebook)
Jing Dong (Georgia Institute of Technology)
Brandon Amos (Facebook AI Research)
Mustafa Mukadam (Meta AI / FAIR)
More from the Same Authors
-
2021 : RB2: Robotic Manipulation Benchmarking with a Twist »
Sudeep Dasari · Jianren Wang · Joyce Hong · Shikhar Bahl · Yixin Lin · Austin Wang · Abitha Thankaraj · Karanbir Chahal · Berk Calli · Saurabh Gupta · David Held · Lerrel Pinto · Deepak Pathak · Vikash Kumar · Abhinav Gupta -
2022 : Meta Optimal Transport »
Brandon Amos · Samuel Cohen · Giulia Luise · Ievgen Redko -
2022 Poster: Semi-Discrete Normalizing Flows through Differentiable Tessellation »
Ricky T. Q. Chen · Brandon Amos · Maximilian Nickel -
2022 Poster: Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real world »
Eugene Vinitsky · Nathan LichtlĂ© · Xiaomeng Yang · Brandon Amos · Jakob Foerster -
2022 Poster: Neural Conservation Laws: A Divergence-Free Perspective »
Jack Richter-Powell · Yaron Lipman · Ricky T. Q. Chen -
2021 : Efficient and Interpretable Robot Manipulation with Graph Neural Networks »
Yixin Lin · Austin Wang · Eric Undersander · Akshara Rai -
2021 Poster: Active 3D Shape Reconstruction from Vision and Touch »
Edward Smith · David Meger · Luis Pineda · Roberto Calandra · Jitendra Malik · Adriana Romero Soriano · Michal Drozdzal -
2021 Poster: K-level Reasoning for Zero-Shot Coordination in Hanabi »
Brandon Cui · Hengyuan Hu · Luis Pineda · Jakob Foerster -
2021 Poster: CrypTen: Secure Multi-Party Computation Meets Machine Learning »
Brian Knott · Shobha Venkataraman · Awni Hannun · Shubho Sengupta · Mark Ibrahim · Laurens van der Maaten -
2020 Workshop: Learning Meets Combinatorial Algorithms »
Marin Vlastelica · Jialin Song · Aaron Ferber · Brandon Amos · Georg Martius · Bistra Dilkina · Yisong Yue -
2020 Poster: Neural Dynamic Policies for End-to-End Sensorimotor Learning »
Shikhar Bahl · Mustafa Mukadam · Abhinav Gupta · Deepak Pathak -
2020 Spotlight: Neural Dynamic Policies for End-to-End Sensorimotor Learning »
Shikhar Bahl · Mustafa Mukadam · Abhinav Gupta · Deepak Pathak -
2020 : Learning rich observation models in factor graphs »
Paloma Sodhi -
2019 Poster: Differentiable Convex Optimization Layers »
Akshay Agrawal · Brandon Amos · Shane Barratt · Stephen Boyd · Steven Diamond · J. Zico Kolter -
2018 : Introduction »
Mustafa Mukadam · Sanjiban Choudhury · Siddhartha Srinivasa -
2018 Workshop: Imitation Learning and its Challenges in Robotics »
Mustafa Mukadam · Sanjiban Choudhury · Siddhartha Srinivasa -
2017 : 6 Spotlight Talks (3 min each) »
Mennatullah Siam · Mohit Prabhushankar · Priyam Parashar · Mustafa Mukadam · hengshuai yao · Ransalu Senanayake