Timezone: »

 
Scaling Covariance Matrix Adaptation MAP-Annealing to High-Dimensional Controllers
Bryon Tjanaka · Matthew Fontaine · Aniruddha Kalkar · Stefanos Nikolaidis

@
in Deep Reinforcement Learning Workshop »
Event URL: https://openreview.net/forum?id=hHxTxdSClOz »

Pre-training a diverse set of robot controllers in simulation has enabled robots to adapt online to damage in robot locomotion tasks. However, finding diverse, high-performing controllers requires specialized hardware and extensive tuning of a large number of hyperparameters. On the other hand, the Covariance Matrix Adaptation MAP-Annealing algorithm, an evolution strategies (ES)-based quality diversity algorithm, does not have these limitations and has been shown to achieve state-of-the-art performance in standard benchmark domains. However, CMA-MAE cannot scale to modern neural network controllers due to its quadratic complexity. We leverage efficient approximation methods in ES to propose three new CMA-MAE variants that scale to very high dimensions. Our experiments show that the variants outperform ES-based baselines in benchmark robotic locomotion tasks, while being comparable with state-of-the-art deep reinforcement learning-based quality diversity algorithms. Source code and videos are available in the supplementary material.

[ Poster]  [ OpenReview]  [ Topia] 

Author Information

Bryon Tjanaka (University of Southern California)
Matthew Fontaine (University of Southern California)
Aniruddha Kalkar (University of Southern California)
Stefanos Nikolaidis (University of Southern California)

More from the Same Authors