Timezone: »
Graph Neural Networks (GNNs) are widely used deep learning models that learn meaningful representations from graph-structured data. Due to the finite nature of the underlying recurrent structure, current GNN methods may struggle to capture long-range dependencies in underlying graphs. To overcome this difficulty, we propose a graph learning framework, called Implicit Graph Neural Networks (IGNN), where predictions are based on the solution of a fixed-point equilibrium equation involving implicitly defined "state" vectors. We use the Perron-Frobenius theory to derive sufficient conditions that ensure well-posedness of the framework. Leveraging implicit differentiation, we derive a tractable projected gradient descent method to train the framework. Experiments on a comprehensive range of tasks show that IGNNs consistently capture long-range dependencies and outperform state-of-the-art GNN models.
Author Information
Fangda Gu (UC Berkeley)
Fangda Gu is a second-year PhD student at the University of California at Berkeley majoring in Electrical Engineering and Computer Sciences. Fangda is advised by Professor Laurent El Ghaoui. Fangda's research interests include machine learning, optimization, control, and power systems. Before coming to Berkeley, Fangda completed an undergraduate degree in Physics at Tsinghua University.
Heng Chang (Tsinghua University)
Wenwu Zhu (Tsinghua University)
Somayeh Sojoudi (University of California, Berkeley)
Laurent El Ghaoui (UC Berkeley)
More from the Same Authors
-
2022 Poster: Module-Aware Optimization for Auxiliary Learning »
Hong Chen · Xin Wang · Yue Liu · Yuwei Zhou · Chaoyu Guan · Wenwu Zhu -
2022 Poster: Learning Invariant Graph Representations for Out-of-Distribution Generalization »
Haoyang Li · Ziwei Zhang · Xin Wang · Wenwu Zhu -
2022 Poster: Dynamic Graph Neural Networks Under Spatio-Temporal Distribution Shift »
Zeyang Zhang · Xin Wang · Ziwei Zhang · Haoyang Li · Zhou Qin · Wenwu Zhu -
2022 Poster: NAS-Bench-Graph: Benchmarking Graph Neural Architecture Search »
Yijian Qin · Ziwei Zhang · Xin Wang · Zeyang Zhang · Wenwu Zhu -
2022 : On the Abilities of Mathematical Extrapolation with Implicit Models »
Alicia Tsai · Juliette Decugis · Ashwin Ganesh · Max Emerling · Laurent El Ghaoui -
2022 : On the Abilities of Mathematical Extrapolation with Implicit Models »
Juliette Decugis · Alicia Tsai · Ashwin Ganesh · Max Emerling · Laurent El Ghaoui -
2022 Spotlight: NAS-Bench-Graph: Benchmarking Graph Neural Architecture Search »
Yijian Qin · Ziwei Zhang · Xin Wang · Zeyang Zhang · Wenwu Zhu -
2022 Poster: On the Convergence of Stochastic Multi-Objective Gradient Manipulation and Beyond »
Shiji Zhou · Wenpeng Zhang · Jiyan Jiang · Wenliang Zhong · Jinjie GU · Wenwu Zhu -
2021 Poster: Asynchronous Decentralized Online Learning »
Jiyan Jiang · Wenpeng Zhang · Jinjie GU · Wenwu Zhu -
2021 Poster: Curriculum Disentangled Recommendation with Noisy Multi-feedback »
Hong Chen · Yudong Chen · Xin Wang · Ruobing Xie · Rui Wang · Feng Xia · Wenwu Zhu -
2021 Poster: Disentangled Contrastive Learning on Graphs »
Haoyang Li · Xin Wang · Ziwei Zhang · Zehuan Yuan · Hang Li · Wenwu Zhu -
2021 Poster: Graph Differentiable Architecture Search with Structure Learning »
Yijian Qin · Xin Wang · Zeyang Zhang · Wenwu Zhu -
2021 Poster: Not All Low-Pass Filters are Robust in Graph Convolutional Networks »
Heng Chang · Yu Rong · Tingyang Xu · Yatao Bian · Shiji Zhou · Xin Wang · Junzhou Huang · Wenwu Zhu -
2019 Poster: Semantic Conditioned Dynamic Modulation for Temporal Sentence Grounding in Videos »
Yitian Yuan · Lin Ma · Jingwen Wang · Wei Liu · Wenwu Zhu -
2019 Poster: Learning Disentangled Representations for Recommendation »
Jianxin Ma · Chang Zhou · Peng Cui · Hongxia Yang · Wenwu Zhu -
2018 Poster: How Much Restricted Isometry is Needed In Nonconvex Matrix Recovery? »
Richard Zhang · Cedric Josz · Somayeh Sojoudi · Javad Lavaei -
2018 Spotlight: How Much Restricted Isometry is Needed In Nonconvex Matrix Recovery? »
Richard Zhang · Cedric Josz · Somayeh Sojoudi · Javad Lavaei -
2018 Poster: A theory on the absence of spurious solutions for nonconvex and nonsmooth optimization »
Cedric Josz · Yi Ouyang · Richard Zhang · Javad Lavaei · Somayeh Sojoudi -
2018 Poster: Weakly Supervised Dense Event Captioning in Videos »
Xin Wang · Wenbing Huang · Chuang Gan · Jingdong Wang · Wenwu Zhu · Junzhou Huang -
2011 Poster: Large-Scale Sparse Principal Component Analysis with Application to Text Data »
Youwei Zhang · Laurent El Ghaoui