A Continuous Formulation of Anisotropy in Graph Neural Networks

Anisotropy in the literature of graph neural networks is typically treated as a binary property, determined by the dependence of the model's message function on both the source and target nodes. We argue that anisotropy should instead be viewed as a continuous property and is dependent on the context in which a model is used. To facilitate this, we introduce a method to measure model anisotropy, framed as the deviation of a node's message weight distribution from a uniform distribution, and a synthetic task in which the need for anisotropy is fixed a priori. We further demonstrate the binding of anisotropy and synthetic task performance through a regularisation of our proposed measure and observe a degradation in performance as the model approaches isotropy.