Timezone: »

Community-based Layerwise Distributed Training of Graph Convolutional Networks
Hongyi Li · Junxiang Wang · Yongchao Wang · Yue Cheng · Liang Zhao

Graph convolutional network (GCN) has been successfully applied to many graph-based applications. Training a large-scale GCN model, however, is still challenging: Due to the node dependency and layer dependency of the GCN architecture, a huge amount of computational time and memory is required in the training process. In this paper, we propose a parallel and distributed GCN training algorithm based on the Alternating Direction Method of Multipliers (ADMM) to tackle the two challenges simultaneously. We first split the GCN layers into independent blocks to achieve layer parallelism. Furthermore, we reduce node dependency by dividing the graph into several dense communities such that each of them can be trained with an agent in parallel. Finally, we provide solutions for all subproblems in the community-based ADMM algorithm. Preliminary results demonstrate that our proposed community-based ADMM training algorithm can lead to more than triple speedup while achieving the best performance compared to state-of-the-art methods.

Author Information

Hongyi Li (Xidian University)
Junxiang Wang (Emory University)
Yongchao Wang (Xidian University)
Yue Cheng (George Mason University)
Liang Zhao (Emory University)

More from the Same Authors

  • 2022 Poster: Deep Generative Model for Periodic Graphs »
    Shiyu Wang · Xiaojie Guo · Liang Zhao
  • 2022 Poster: Multi-objective Deep Data Generation with Correlated Property Control »
    Shiyu Wang · Xiaojie Guo · Xuanyang Lin · Bo Pan · Yuanqi Du · Yinkai Wang · Yanfang Ye · Ashley Petersen · Austin Leitgeb · Saleh Alkhalifa · Kevin Minbiole · William M. Wuest · Amarda Shehu · Liang Zhao
  • 2021 : Poster Session 1 (gather.town) »
    Hamed Jalali · Robert Hönig · Maximus Mutschler · Manuel Madeira · Abdurakhmon Sadiev · Egor Shulgin · Alasdair Paren · Pascal Esser · Simon Roburin · Julius Kunze · Agnieszka Słowik · Frederik Benzing · Futong Liu · Hongyi Li · Ryotaro Mitsuboshi · Grigory Malinovsky · Jayadev Naram · Zhize Li · Igor Sokolov · Sharan Vaswani