IGT: Modeling local and global patterns of graph data by interleaving GNNs and transformers

The Graph Transformer (GT) combines graph neural networks (GNNs) with Transformers and achieves remarkable performance across a wide range of graph learning tasks. However, existing GTs remain structurally constrained, which hinders their ability to jointly exploit global and local information. To address this issue, we establish a unified GT framework, termed IGT, by generalizing the interleaving of GNNs and Transformers across different graph representations and tasks. IGT allows the local operator to be flexibly instantiated according to the characteristics of the graph data and its representation. In this study, EELA and TextGINConv are used as representative sparse and dense local operators, respectively. By interleaving local message passing and Transformer modules, IGT iteratively integrates information from neighboring and distant nodes. We instantiate the framework for graph representation learning as IGT-GRL, where a configurable sparse local message passing operator is interleaved with the S2D-Transformer, and for text classification as IGT-TC, where TextGINConv is interleaved with Transformer layers for aspect-based sentiment analysis. Extensive experiments on GRL and TC tasks show that the IGT structure achieves competitive performance relative to existing GNN and GT models.