Timezone: »

 
Poster
Low-Rank Modular Reinforcement Learning via Muscle Synergy
Heng Dong · Tonghan Wang · Jiayuan Liu · Chongjie Zhang

Wed Nov 30 09:00 AM -- 11:00 AM (PST) @ Hall J #800
in Poster Session 3 »

Modular Reinforcement Learning (RL) decentralizes the control of multi-joint robots by learning policies for each actuator. Previous work on modular RL has proven its ability to control morphologically different agents with a shared actuator policy. However, with the increase in the Degree of Freedom (DoF) of robots, training a morphology-generalizable modular controller becomes exponentially difficult. Motivated by the way the human central nervous system controls numerous muscles, we propose a Synergy-Oriented LeARning (SOLAR) framework that exploits the redundant nature of DoF in robot control. Actuators are grouped into synergies by an unsupervised learning method, and a synergy action is learned to control multiple actuators in synchrony. In this way, we achieve a low-rank control at the synergy level. We extensively evaluate our method on a variety of robot morphologies, and the results show its superior efficiency and generalizability, especially on robots with a large DoF like Humanoids++ and UNIMALs.

Keywords: Reinforcement Learning · Low-Rank · Muscle Synergy
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Heng Dong (Tsinghua University)
Tonghan Wang (Tsinghua University)

Tonghan Wang is currently a Master student working with Prof. Chongjie Zhang at Institute for Interdisciplinary Information Sciences, Tsinghua University, headed by Prof. Andrew Yao. His primary research goal is to develop innovative models and methods to enable effective multi-agent cooperation, allowing a group of individuals to explore, communicate, and accomplish tasks of higher complexity. His research interests include multi-agent learning, reasoning under uncertainty, reinforcement learning, and representation learning in multi-agent systems.

Jiayuan Liu (Tsinghua University)
Chongjie Zhang (Tsinghua University)

More from the Same Authors