Computer Science > Machine Learning
[Submitted on 24 May 2023 (v1), last revised 7 Feb 2024 (this version, v4)]
Title:Size Generalization of Graph Neural Networks on Biological Data: Insights and Practices from the Spectral Perspective
Download PDF HTML (experimental)Abstract:We investigate size-induced distribution shifts in graphs and assess their impact on the ability of graph neural networks (GNNs) to generalize to larger graphs relative to the training data. Existing literature presents conflicting conclusions on GNNs' size generalizability, primarily due to disparities in application domains and underlying assumptions concerning size-induced distribution shifts. Motivated by this, we take a data-driven approach: we focus on real biological datasets and seek to characterize the types of size-induced distribution shifts. Diverging from prior approaches, we adopt a spectral perspective and identify that spectrum differences induced by size are related to differences in subgraph patterns (e.g., average cycle lengths). While previous studies have identified that the inability of GNNs in capturing subgraph information negatively impacts their in-distribution generalization, our findings further show that this decline is more pronounced when evaluating on larger test graphs not encountered during training. Based on these spectral insights, we introduce a simple yet effective model-agnostic strategy, which makes GNNs aware of these important subgraph patterns to enhance their size generalizability. Our empirical results reveal that our proposed size-insensitive attention strategy substantially enhances graph classification performance on large test graphs, which are 2-10 times larger than the training graphs, resulting in an improvement in F1 scores by up to 8%.
Submission history
From: Gaotang Li [view email][v1] Wed, 24 May 2023 23:01:14 UTC (5,283 KB)
[v2] Fri, 29 Sep 2023 21:51:27 UTC (22,881 KB)
[v3] Tue, 6 Feb 2024 04:15:14 UTC (23,031 KB)
[v4] Wed, 7 Feb 2024 03:27:12 UTC (23,031 KB)
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)