Timezone: »
We present a method for training recurrent neural networks to act as near-optimal feedback controllers. It is able to generate stable and realistic behaviors for a range of dynamical systems and tasks -- swimming, flying, biped and quadruped walking with different body morphologies. It does not require motion capture or task-specific features or state machines. The controller is a neural network, having a large number of feed-forward units that learn elaborate state-action mappings, and a small number of recurrent units that implement memory states beyond the physical system state. The action generated by the network is defined as velocity. Thus the network is not learning a control policy, but rather the dynamics under an implicit policy. Essential features of the method include interleaving supervised learning with trajectory optimization, injecting noise during training, training for unexpected changes in the task specification, and using the trajectory optimizer to obtain optimal feedback gains in addition to optimal actions.
Author Information
Igor Mordatch (University of Washington)
Kendall Lowrey (University of Washington)
Galen Andrew (University of Washington, Seattle)
Zoran Popovic (University of Washington)
Emanuel Todorov (University of Washington)
More from the Same Authors
-
2021 : The Neural MMO Platform for Massively Multiagent Research »
Joseph Suarez · Yilun Du · Clare Zhu · Igor Mordatch · Phillip Isola -
2021 : Brax - A Differentiable Physics Engine for Large Scale Rigid Body Simulation »
Daniel Freeman · Erik Frey · Anton Raichuk · Sertan Girgin · Igor Mordatch · Olivier Bachem -
2022 : Panel Discussion »
Cynthia Rudin · Dan Bohus · Brenna Argall · Alison Gopnik · Igor Mordatch · Samuel Kaski -
2022 : Language models and interactive decision-making »
Igor Mordatch -
2022 Poster: Subgroup Robustness Grows On Trees: An Empirical Baseline Investigation »
Josh Gardner · Zoran Popovic · Ludwig Schmidt -
2021 Poster: Unsupervised Learning of Compositional Energy Concepts »
Yilun Du · Shuang Li · Yash Sharma · Josh Tenenbaum · Igor Mordatch -
2021 Poster: Decision Transformer: Reinforcement Learning via Sequence Modeling »
Lili Chen · Kevin Lu · Aravind Rajeswaran · Kimin Lee · Aditya Grover · Misha Laskin · Pieter Abbeel · Aravind Srinivas · Igor Mordatch -
2018 Workshop: Infer to Control: Probabilistic Reinforcement Learning and Structured Control »
Leslie Kaelbling · Martin Riedmiller · Marc Toussaint · Igor Mordatch · Roy Fox · Tuomas Haarnoja -
2018 : Talk 2: Emo Todorov - Physics-Based Control »
Emanuel Todorov -
2017 Poster: Towards Generalization and Simplicity in Continuous Control »
Aravind Rajeswaran · Kendall Lowrey · Emanuel Todorov · Sham Kakade -
2016 : Emo Todorov (University of Washington) »
Emanuel Todorov -
2015 Oral: Interactive Control of Diverse Complex Characters with Neural Networks »
Igor Mordatch · Kendall Lowrey · Galen Andrew · Zoran Popovic · Emanuel Todorov -
2010 Poster: Feature Construction for Inverse Reinforcement Learning »
Sergey Levine · Zoran Popovic · Vladlen Koltun -
2010 Poster: Policy gradients in linearly-solvable MDPs »
Emanuel Todorov -
2009 Poster: Compositionality of optimal control laws »
Emanuel Todorov