HyPE-GT: where Graph Transformers meet Hyperbolic Positional Encodings
Kushal Bose Swagatam Das
Graph Transformers (GTs) facilitate the comprehension of graph-structured
data by calculating the self-attention of node pairs without considering node
position information. To address this limitation, we introduce an innovative
and efficient framework that introduces Positional Encodings (PEs) into the
Transformer, generating a set of learnable positional encodings in the
hyperbolic space, a non-Euclidean domain. This approach empowers us to explore
diverse options for optimal selection of PEs for specific downstream tasks,
leveraging hyperbolic neural networks or hyperbolic graph convolutional
networks. Additionally, we repurpose these positional encodings to mitigate the
impact of over-smoothing in deep Graph Neural Networks (GNNs). Comprehensive
experiments on molecular benchmark datasets, co-author, and co-purchase
networks substantiate the effectiveness of hyperbolic positional encodings in
enhancing the performance of deep GNNs.