Timezone: »
Each year, expert-level performance is attained in increasingly-complex multiagent domains, where notable examples include Go, Poker, and StarCraft II. This rapid progression is accompanied by a commensurate need to better understand how such agents attain this performance, to enable their safe deployment, identify limitations, and reveal potential means of improving them. In this paper we take a step back from performance-focused multiagent learning, and instead turn our attention towards agent behavior analysis. We introduce a model-agnostic method for discovery of behavior clusters in multiagent domains, using variational inference to learn a hierarchy of behaviors at the joint and local agent levels. Our framework makes no assumption about agents' underlying learning algorithms, does not require access to their latent states or policies, and is trained using only offline observational data. We illustrate the effectiveness of our method for enabling the coupled understanding of behaviors at the joint and local agent level, detection of behavior changepoints throughout training, discovery of core behavioral concepts, demonstrate the approach's scalability to a high-dimensional multiagent MuJoCo control domain, and also illustrate that the approach can disentangle previously-trained policies in OpenAI's hide-and-seek domain.
Author Information
Shayegan Omidshafiei (Google)
Andrei Kapishnikov (Google)
Yannick Assogba (Google Research)
Lucas Dixon (Research, Google)

I'm interested in visualization, understanding machine learning, and language as a boundary object for AI systems - for example natural language preferences for recommenders. My PhD and postdocs were at the University of Edinburgh, and then at Google I worked in Google and then founded the engineering and research efforts at Jigsaw. I've worked on a diverse range of topics including network security, foundations of mathematics and logic, graphical languages for quantum information, ML for NLP, and data visualization. I helped make uProxy/Outline, Project Shield, DigitalAttackMap, Syria Defection Tracker, and unfiltered.news. I led Jigsaw's early research into disinformation, and was the technical lead of the Conversation AI/Perspective efforts, and then cofounded Keen, an Area 120 project.
Been Kim (Google Brain)
More from the Same Authors
-
2021 : Advanced Methods for Connectome-Based Predictive Modeling of Human Intelligence: A Novel Approach Based on Individual Differences in Cortical Topography »
Evan Anderson · Anuj Nayak · Pablo Robles-Granda · Lav Varshney · Been Kim · Aron K Barbey -
2022 : Multi-Agent Reinforcement Learning for Microprocessor Design Space Exploration »
Srivatsan Krishnan · Natasha Jaques · Shayegan Omidshafiei · Dan Zhang · Izzeddin Gur · Vijay Janapa Reddi · Aleksandra Faust -
2022 : Concept-based Understanding of Emergent Multi-Agent Behavior »
Niko Grupen · Shayegan Omidshafiei · Natasha Jaques · Been Kim -
2023 Poster: State2Explanation: Concept-Based Explanations to Benefit Agent Learning and User Understanding »
Devleena Das · Sonia Chernova · Been Kim -
2023 Poster: Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models »
Peter Hase · Mohit Bansal · Been Kim · Asma Ghandeharioun -
2023 Poster: Gaussian Process Probes (GPP) for Uncertainty-Aware Probing »
Alexander Ku · Zi Wang · Jason Baldridge · Tom Griffiths · Been Kim -
2022 : Panel: Explainability/Predictability Robotics (Q&A 4) »
Katherine Driggs-Campbell · Been Kim · Leila Takayama -
2022 : Panel Discussion »
Kamalika Chaudhuri · Been Kim · Dorsa Sadigh · Huan Zhang · Linyi Li -
2022 : Invited Talk: Been Kim »
Been Kim -
2022 : Multi-Agent Reinforcement Learning for Microprocessor Design Space Exploration »
Srivatsan Krishnan · Natasha Jaques · Shayegan Omidshafiei · Dan Zhang · Izzeddin Gur · Vijay Janapa Reddi · Aleksandra Faust -
2020 Poster: Real World Games Look Like Spinning Tops »
Wojciech Czarnecki · Gauthier Gidel · Brendan Tracey · Karl Tuyls · Shayegan Omidshafiei · David Balduzzi · Max Jaderberg -
2019 : Invited talk #5 »
Been Kim -
2019 : Responsibilities »
Been Kim · Liz O'Sullivan · Friederike Schuur · Andrew Smart · Jacob Metcalf -
2019 Poster: Multiagent Evaluation under Incomplete Information »
Mark Rowland · Shayegan Omidshafiei · Karl Tuyls · Julien Perolat · Michal Valko · Georgios Piliouras · Remi Munos -
2019 Spotlight: Multiagent Evaluation under Incomplete Information »
Mark Rowland · Shayegan Omidshafiei · Karl Tuyls · Julien Perolat · Michal Valko · Georgios Piliouras · Remi Munos -
2017 : Invited Talk 1 »
Been Kim -
2016 Workshop: Interpretable Machine Learning for Complex Systems »
Andrew Wilson · Been Kim · William Herlands -
2016 Oral: Examples are not enough, learn to criticize! Criticism for Interpretability »
Been Kim · Sanmi Koyejo · Rajiv Khanna -
2016 Poster: Examples are not enough, learn to criticize! Criticism for Interpretability »
Been Kim · Sanmi Koyejo · Rajiv Khanna -
2015 Poster: Mind the Gap: A Generative Approach to Interpretable Feature Selection and Extraction »
Been Kim · Julie A Shah · Finale Doshi-Velez -
2014 Poster: The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification »
Been Kim · Cynthia Rudin · Julie A Shah