Timezone: »

 
Poster
Snowflake: Scaling GNNs to high-dimensional continuous control via parameter freezing
Charles Blake · Vitaly Kurin · Maximilian Igl · Shimon Whiteson

Tue Dec 07 08:30 AM -- 10:00 AM (PST) @
in Poster Session 1 »

Recent research has shown that graph neural networks (GNNs) can learn policies for locomotion control that are as effective as a typical multi-layer perceptron (MLP), with superior transfer and multi-task performance. However, results have so far been limited to training on small agents, with the performance of GNNs deteriorating rapidly as the number of sensors and actuators grows. A key motivation for the use of GNNs in the supervised learning setting is their applicability to large graphs, but this benefit has not yet been realised for locomotion control. We show that poor scaling in GNNs is a result of increasingly unstable policy updates, caused by overfitting in parts of the network during training. To combat this, we introduce Snowflake, a GNN training method for high-dimensional continuous control that freezes parameters in selected parts of the network. Snowflake significantly boosts the performance of GNNs for locomotion control on large agents, now matching the performance of MLPs while offering superior transfer properties.

Keywords: Deep Learning · Reinforcement Learning and Planning · Graph Learning

Author Information

Charles Blake (Graphcore)

AI Researcher at Graphcore. Interests in RL, Graph Networks, Sparsity & all things Solitaire. MSc Computer Science, Oxford; BSc Computer Science, St Andrews.

Vitaly Kurin (University of Oxford)
Maximilian Igl (University of Oxford)
Shimon Whiteson (University of Oxford)

More from the Same Authors