Timezone: »

 
Poster
Celebrating Diversity in Shared Multi-Agent Reinforcement Learning
Chenghao Li · Tonghan Wang · Chengjie Wu · Qianchuan Zhao · Jun Yang · Chongjie Zhang

Wed Dec 08 12:30 AM -- 02:00 AM (PST) @
in Poster Session 3 »

Recently, deep multi-agent reinforcement learning (MARL) has shown the promise to solve complex cooperative tasks. Its success is partly because of parameter sharing among agents. However, such sharing may lead agents to behave similarly and limit their coordination capacity. In this paper, we aim to introduce diversity in both optimization and representation of shared multi-agent reinforcement learning. Specifically, we propose an information-theoretical regularization to maximize the mutual information between agents' identities and their trajectories, encouraging extensive exploration and diverse individualized behaviors. In representation, we incorporate agent-specific modules in the shared neural network architecture, which are regularized by L1-norm to promote learning sharing among agents while keeping necessary diversity. Empirical results show that our method achieves state-of-the-art performance on Google Research Football and super hard StarCraft II micromanagement tasks.

Keywords: Deep Learning · Optimization · Reinforcement Learning and Planning

Author Information

Chenghao Li (Tsinghua University)
Tonghan Wang (Tsinghua University)

Tonghan Wang is currently a Master student working with Prof. Chongjie Zhang at Institute for Interdisciplinary Information Sciences, Tsinghua University, headed by Prof. Andrew Yao. His primary research goal is to develop innovative models and methods to enable effective multi-agent cooperation, allowing a group of individuals to explore, communicate, and accomplish tasks of higher complexity. His research interests include multi-agent learning, reasoning under uncertainty, reinforcement learning, and representation learning in multi-agent systems.

Chengjie Wu (Tsinghua University)
Qianchuan Zhao (Tsinghua University, Tsinghua University)
Jun Yang (Tsinghua University, Tsinghua University)
Chongjie Zhang (Tsinghua University)

More from the Same Authors