Timezone: »
Independent reinforcement learning algorithms have no theoretical guarantees for finding the best policy in multi-agent settings. However, in practice, prior works have reported good performance with independent algorithms in some domains and bad performance in others. Moreover, a comprehensive study of the strengths and weaknesses of independent algorithms is lacking in the literature. In this paper, we carry out an empirical comparison of the performance of independent algorithms on four PettingZoo environments that span the three main categories of multi-agent environments, i.e., cooperative, competitive, and mixed. We show that in fully-observable environments, independent algorithms can perform on par with multi-agent algorithms in cooperative and competitive settings. For the mixed environments, we show that agents trained via independent algorithms learn to perform well individually, but fail to learn to cooperate with allies and compete with enemies. We also show that adding recurrence improves the learning of independent algorithms in cooperative partially observable environments.
Author Information
Ken Ming Lee (University of Waterloo)
Hi, I'm Ken, a Computer Engineering undergraduate from University of Waterloo. My current interests lie in the field of reinforcement learning (RL). For more information about myself, please head to [kenminglee.github.io](https://kenminglee.github.io).
Sriram Ganapathi (University of Waterloo)
Mark Crowley (University of Waterloo)
Prof. Mark Crowley runs the UWECEML lab and is an Associate Professor at the University of Waterloo in the ECE department. His research explores how to augment human decision making in complex domains in dependable and transparent ways by investigating the theoretical and practical challenges that arise from the presence of spatial structure, large scale streaming data, uncertainty, or unknown causal structure, or interaction of multiple decision makers. His focus is on developing new algorithms, methodologies, simulations, and datasets within the fields of Reinforcement Learning (RL), Deep Learning, Manifold Learning and Ensemble Methods.
More from the Same Authors
-
2020 : Contributed Talk: Maximum Reward Formulation In Reinforcement Learning »
Vijaya Sai Krishna Gottipati · Yashaswi Pathak · Rohan Nuttall · Sahir . · Raviteja Chunduru · Ahmed Touati · Sriram Ganapathi · Matthew Taylor · Sarath Chandar -
2018 Workshop: AI for social good »
Margaux Luck · Tristan Sylvain · Joseph Paul Cohen · Arsene Fansi Tchango · Valentine Goddard · Aurelie Helouis · Yoshua Bengio · Sam Greydanus · Cody Wild · Taras Kucherenko · Arya Farahi · Jonathan Penn · Sean McGregor · Mark Crowley · Abhishek Gupta · Kenny Chen · Myriam Côté · Rediet Abebe