Timezone: »

 
Poster
Multi-agent Dynamic Algorithm Configuration
Ke Xue · Jiacheng Xu · Lei Yuan · Miqing Li · Chao Qian · Zongzhang Zhang · Yang Yu

Tue Nov 29 09:00 AM -- 11:00 AM (PST) @ Hall J #734
in Poster Session 1 »

Automated algorithm configuration relieves users from tedious, trial-and-error tuning tasks. A popular algorithm configuration tuning paradigm is dynamic algorithm configuration (DAC), in which an agent learns dynamic configuration policies across instances by reinforcement learning (RL). However, in many complex algorithms, there may exist different types of configuration hyperparameters, and such heterogeneity may bring difficulties for classic DAC which uses a single-agent RL policy. In this paper, we aim to address this issue and propose multi-agent DAC (MA-DAC), with one agent working for one type of configuration hyperparameter. MA-DAC formulates the dynamic configuration of a complex algorithm with multiple types of hyperparameters as a contextual multi-agent Markov decision process and solves it by a cooperative multi-agent RL (MARL) algorithm. To instantiate, we apply MA-DAC to a well-known optimization algorithm for multi-objective optimization problems. Experimental results show the effectiveness of MA-DAC in not only achieving superior performance compared with other configuration tuning approaches based on heuristic rules, multi-armed bandits, and single-agent RL, but also being capable of generalizing to different problem classes. Furthermore, we release the environments in this paper as a benchmark for testing MARL algorithms, with the hope of facilitating the application of MARL.

Keywords: multi-agent reinforcement learning · Multi-Objective Optimization · Automated Machine Learning · Dynamic algorithm configuration. · Evolutionary algorithm
[ Paper]  [ Slides]  [ Poster]  [ OpenReview] 

Author Information

Ke Xue (Nanjing University)
Jiacheng Xu (Nanjing University)
Lei Yuan (None)
Miqing Li (University of Birmingham)

Dr Miqing Li is an Assistant Professor at the University of Birmingham and a Turing Fellow of the Alan Turing Institute, UK. His research is principally on multi-objective optimisation, where he focuses on developing evolutionary algorithms for both general challenging problems (e.g. many-objective optimisation, constrained optimisation, robust optimisation, expensive optimisation) and specific application problems (e.g. those in software engineering, high-performance computing, neural architecture search, product disassembly and supply chain).

Chao Qian (Nanjing University)
Zongzhang Zhang (Nanjing University)
Zongzhang Zhang

I am now an associate professor at the School of Artificial Intelligence, Nanjing University.

Yang Yu (Nanjing University)

More from the Same Authors