Timezone: »
Understanding the emerging behaviors of deep reinforcement learning agents may be difficult because such agents are often trained using highly complex and expressive models. In recent years, most approaches developed for explaining agent behaviors rely on domain knowledge or on an analysis of the agent’s learned policy. For some domains, relevant knowledge may not be available or may be insufficient for producing meaningful explanations. We suggest using formal model abstractions and transforms, previously used mainly for expediting the search for optimal policies, to automatically explain discrepancies that may arise between the behavior of an agent and the behavior that is anticipated by an observer. We formally define this problem of Reinforcement Learning Policy Explanation (RLPE), suggest a class of transforms which can be used for explaining emergent behaviors, and suggest methods for searching efficiently for an explanation. We demonstrate the approach on standard benchmarks.
Author Information
Sarah Keren (Technion, Technion)
Yoav Kolumbus (Hebrew University of Jerusalem)
Jeffrey S Rosenschein (The Hebrew University of Jerusalem)
David Parkes (Harvard University)
David C. Parkes is Gordon McKay Professor of Computer Science in the School of Engineering and Applied Sciences at Harvard University. He was the recipient of the NSF Career Award, the Alfred P. Sloan Fellowship, the Thouron Scholarship and the Harvard University Roslyn Abramson Award for Teaching. Parkes received his Ph.D. degree in Computer and Information Science from the University of Pennsylvania in 2001, and an M.Eng. (First class) in Engineering and Computing Science from Oxford University in 1995. At Harvard, Parkes leads the EconCS group and teaches classes in artificial intelligence, optimization, and topics at the intersection between computer science and economics. Parkes has served as Program Chair of ACM EC’07 and AAMAS’08 and General Chair of ACM EC’10, served on the editorial board of Journal of Artificial Intelligence Research, and currently serves as Editor of Games and Economic Behavior and on the boards of Journal of Autonomous Agents and Multi-agent Systems and INFORMS Journal of Computing. His research interests include computational mechanism design, electronic commerce, stochastic optimization, preference elicitation, market design, bounded rationality, computational social choice, networks and incentives, multi-agent systems, crowd-sourcing and social computing.
Mira Finkelstein (The Hebrew University)
More from the Same Authors
-
2021 : Promoting Resilience of Multi-Agent Reinforcement Learning via Confusion-Based Communication »
Ofir Abu · Sarah Keren · Matthias Gerstgrasser · Jeffrey S Rosenschein -
2021 : Promoting Resilience in Multi-Agent Reinforcement Learning via Confusion-Based Communication »
Ofir Abu · Matthias Gerstgrasser · Jeffrey S Rosenschein · Sarah Keren -
2021 : Promoting Resilience of Multi-Agent Reinforcement Learning via Confusion-Based Communication »
Ofir Abu · Sarah Keren · Matthias Gerstgrasser · Jeffrey S Rosenschein -
2021 Workshop: Learning in Presence of Strategic Behavior »
Omer Ben-Porat · Nika Haghtalab · Annie Liang · Yishay Mansour · David Parkes -
2020 : Panel: Kate Larson (DeepMind) [moderator], Natasha Jaques (Google), Jeffrey Rosenschein (The Hebrew University of Jerusalem), Michael Wooldridge (University of Oxford) »
Kate Larson · Natasha Jaques · Jeffrey S Rosenschein · Michael Wooldridge -
2020 Workshop: Machine Learning for Economic Policy »
Stephan Zheng · Alexander Trott · Annie Liang · Jamie Morgenstern · David Parkes · Nika Haghtalab -
2020 Poster: From Predictions to Decisions: Using Lookahead Regularization »
Nir Rosenfeld · Anna Hilgard · Sai Srivatsa Ravindranath · David Parkes -
2019 Poster: Finding Friend and Foe in Multi-Agent Games »
Jack Serrino · Max Kleiman-Weiner · David Parkes · Josh Tenenbaum -
2019 Spotlight: Finding Friend and Foe in Multi-Agent Games »
Jack Serrino · Max Kleiman-Weiner · David Parkes · Josh Tenenbaum -
2017 : Optimal Economic Design through Deep Learning »
David Parkes -
2017 Poster: Multi-View Decision Processes: The Helper-AI Problem »
Christos Dimitrakakis · David Parkes · Goran Radanovic · Paul Tylkin -
2016 Poster: Long-term Causal Effects via Behavioral Game Theory »
Panagiotis Toulis · David Parkes -
2015 Poster: Learnability of Influence in Networks »
Harikrishna Narasimhan · David Parkes · Yaron Singer -
2014 Workshop: NIPS’14 Workshop on Crowdsourcing and Machine Learning »
David Parkes · Denny Zhou · Chien-Ju Ho · Nihar Bhadresh Shah · Adish Singla · Jared Heyman · Edwin Simpson · Andreas Krause · Rafael Frongillo · Jennifer Wortman Vaughan · Panagiotis Papadimitriou · Damien Peters -
2014 Workshop: Analysis of Rank Data: Confluence of Social Choice, Operations Research, and Machine Learning »
Shivani Agarwal · Hossein Azari Soufiani · Guy Bresler · Sewoong Oh · David Parkes · Arun Rajkumar · Devavrat Shah -
2014 Workshop: NIPS Workshop on Transactional Machine Learning and E-Commerce »
David Parkes · David H Wolpert · Jennifer Wortman Vaughan · Jacob D Abernethy · Amos Storkey · Mark Reid · Ping Jin · Nihar Bhadresh Shah · Mehryar Mohri · Luis E Ortiz · Robin Hanson · Aaron Roth · Satyen Kale · Sebastien Lahaie -
2014 Poster: A Statistical Decision-Theoretic Framework for Social Choice »
Hossein Azari Soufiani · David Parkes · Lirong Xia -
2014 Oral: A Statistical Decision-Theoretic Framework for Social Choice »
Hossein Azari Soufiani · David Parkes · Lirong Xia -
2013 Poster: Generalized Random Utility Models with Multiple Types »
Hossein Azari Soufiani · Hansheng Diao · Zhenyu Lai · David Parkes -
2013 Poster: Contrastive Learning Using Spectral Methods »
James Y Zou · Daniel Hsu · David Parkes · Ryan Adams -
2013 Poster: Generalized Method-of-Moments for Rank Aggregation »
Hossein Azari Soufiani · William Z Chen · David Parkes · Lirong Xia -
2010 Invited Talk: The Interplay of Machine Learning and Mechanism Design »
David Parkes