A hybrid similarity-aware graph neural network with transformer for node classification

Abstract

Node classification has emerged as a critical task in graph deep learning, with diverse real-world applications such as recommendation systems, drug discovery, and citation networks. Although Graph Convolutional Networks and Graph Transformers have shown strong performance in this domain, they face inherent limitations. GCNs suffer from over-squashing, which restricts their ability to model long-range dependencies, while Graph Transformers encounter scalability issues when applied to large graphs. To address these challenges, we propose SIGNNet, A Hybrid SImilarity-aware Graph Neural Network with Transformer for Node Classification. SIGNNet effectively captures both local and global structural information, enhancing the model’s ability to learn fine-grained relationships and broader contextual patterns in graph data. The framework combines GCNs with a score-based similarity mechanism to improve local and global node interaction modeling while mitigating the effects of oversquashing. To tackle scalability, we introduce a Personalized PageRank-based node sampling strategy that enables efficient subgraph generation. Additionally, SIGNNet incorporates a novel Structure-Aware Multi-Head Attention (SA-MHA) mechanism that integrates structural features into the attention process, allowing the model to prioritize nodes based on their topological importance. We have conducted extensive experiments on both homophilic and heterophilic benchmark datasets to evaluate the effectiveness of our proposed method, SIGNNet. Our method consistently outperforms existing state-of-the-art approaches, demonstrating significant improvements across all datasets. Specifically, SIGNNet achieves average accuracy gains of 6.03%, 5.47%, and 4.78% on the homophilic datasets Cora, Citeseer, and CS, respectively. Even more substantial improvements are observed on the heterophilic datasets, with gains of 19.10% on Wisconsin, 19.61% on Texas, 19.54% on Cornell, 7.22% on Actor, and 14.94% on Chameleon. These results highlight the robustness and generalizability of SIGNNet in handling diverse graph structures.