Timezone: »
Recent progress in reinforcement learning (RL) has started producing generally capable agents that can solve a distribution of complex environments. These agents are typically tested on fixed, human-authored environments. On the other hand, quality diversity (QD) optimization has been proven to be an effective component of environment generation algorithms, which can generate collections of high-quality environments that are diverse in the resulting agent behaviors. However, these algorithms require potentially expensive simulations of agents on newly generated environments. We propose Deep Surrogate Assisted Generation of Environments (DSAGE), a sample-efficient QD environment generation algorithm that maintains a deep surrogate model for predicting agent behaviors in new environments. Results in two benchmark domains show that DSAGE significantly outperforms existing QD environment generation algorithms in discovering collections of environments that elicit diverse behaviors of a state-of-the-art RL agent and a planning agent. Our source code and videos are available at https://dsagepaper.github.io/.
Author Information
Varun Bhatt (University of Southern California)
Bryon Tjanaka (University of Southern California)
Matthew Fontaine (University of Southern California)
Stefanos Nikolaidis (University of Southern California)
More from the Same Authors
-
2021 : On the Importance of Environments in Human-Robot Coordination »
Matthew Fontaine · Ya-Chuan Hsu · Yulun Zhang · Bryon Tjanaka · Stefanos Nikolaidis -
2022 : Scaling Covariance Matrix Adaptation MAP-Annealing to High-Dimensional Controllers »
Bryon Tjanaka · Matthew Fontaine · Aniruddha Kalkar · Stefanos Nikolaidis -
2022 : Panel Discussion »
Chuchu Fan · Stefanos Nikolaidis · Katherine Driggs-Campbell · Matthew Gombolay · Daniel Omeiza -
2022 : Scenario Generation via Quality Diversity for Trustworthy AI »
Stefanos Nikolaidis -
2021 Poster: Differentiable Quality Diversity »
Matthew Fontaine · Stefanos Nikolaidis -
2021 Oral: Differentiable Quality Diversity »
Matthew Fontaine · Stefanos Nikolaidis -
2019 Demonstration: Robot-Assisted Hair-Brushing »
Eura Shin · Hejia Zhang · Rey J Pocius · Nathaniel Dennler · Heather Culbertson · Naghmeh Zamani · Stefanos Nikolaidis