Timezone: »

Calibration of Shared Equilibria in General Sum Partially Observable Markov Games
Nelson Vadori · Sumitra Ganesh · Prashant Reddy · Manuela Veloso

Thu Dec 10 09:00 AM -- 11:00 AM (PST) @ Poster Session 5 #1506

Training multi-agent systems (MAS) to achieve realistic equilibria gives us a useful tool to understand and model real-world systems. We consider a general sum partially observable Markov game where agents of different types share a single policy network, conditioned on agent-specific information. This paper aims at i) formally understanding equilibria reached by such agents, and ii) matching emergent phenomena of such equilibria to real-world targets. Parameter sharing with decentralized execution has been introduced as an efficient way to train multiple agents using a single policy network. However, the nature of resulting equilibria reached by such agents has not been yet studied: we introduce the novel concept of Shared equilibrium as a symmetric pure Nash equilibrium of a certain Functional Form Game (FFG) and prove convergence to the latter for a certain class of games using self-play. In addition, it is important that such equilibria satisfy certain constraints so that MAS are calibrated to real world data for practical use: we solve this problem by introducing a novel dual-Reinforcement Learning based approach that fits emergent behaviors of agents in a Shared equilibrium to externally-specified targets, and apply our methods to a n-player market example. We do so by calibrating parameters governing distributions of agent types rather than individual agents, which allows both behavior differentiation among agents and coherent scaling of the shared policy network to multiple agents.

Author Information

Nelson Vadori (J.P. Morgan AI Research)
Sumitra Ganesh (JPMorgan - AI Research)
Prashant Reddy (JP Morgan)
Manuela Veloso (J.P. Morgan AI Research )

More from the Same Authors

  • 2021 : Mixture of Basis for Interpretable Continual Learning with Distribution Shifts »
    Mengda Xu · Sumitra Ganesh · Pranay Pasula
  • 2022 : Conditional Demographic Parity Through Optimal Transport »
    Luhao Zhang · Mohsen Ghassemi · Ivan Brugere · Niccolo Dalmasso · Alan Mishler · Vamsi Potluru · Tucker Balch · Manuela Veloso
  • 2021 Poster: Factored Policy Gradients: Leveraging Structure for Efficient Learning in MOMDPs »
    Thomas Spooner · Nelson Vadori · Sumitra Ganesh
  • 2019 : Poster Session »
    Nathalie Baracaldo · Seth Neel · Tuyen Le · Dan Philps · Suheng Tao · Sotirios Chatzis · Toyo Suzumura · Wei Wang · WENHANG BAO · Solon Barocas · Manish Raghavan · Samuel Maina · Reginald Bryant · Kush Varshney · Skyler D. Speakman · Navdeep Gill · Nicholas Schmidt · Kevin Compher · Naveen Sundar Govindarajulu · Vivek Sharma · Praneeth Vepakomma · Tristan Swedish · Jayashree Kalpathy-Cramer · Ramesh Raskar · Shihao Zheng · Mykola Pechenizkiy · Marco Schreyer · Li Ling · Chirag Nagpal · Robert Tillman · Manuela Veloso · Hanjie Chen · Xintong Wang · Michael Wellman · Matthew van Adelsberg · Ben Wood · Hans Buehler · Mahmoud Mahfouz · Antonios Alexos · Megan Shearer · Antigoni Polychroniadou · Lucia Larise Stavarache · Dmitry Efimov · Johnston P Hall · Yukun Zhang · Emily Diana · Sumitra Ganesh · Vineeth Ravi · · Swetasudha Panda · Xavier Renard · Matthew Jagielski · Yonadav Shavit · Joshua Williams · Haoran Wei · Shuang (Sophie) Zhai · Xinyi Li · Hongda Shen · Daiki Matsunaga · Jaesik Choi · Alexis Laignelet · Batuhan Guler · Jacobo Roa Vicens · Ajit Desai · Jonathan Aigrain · Robert Samoilescu