Low-bit Quantization for Deep Graph Neural Networks with
  Smoothness-aware Message Propagation
Shuang Wang Bahaeddin Eravci Rustam Guliyev Hakan Ferhatosmanoglu
Graph Neural Network (GNN) training and inference involve significant
challenges of scalability with respect to both model sizes and number of
layers, resulting in degradation of efficiency and accuracy for large and deep
GNNs. We present an end-to-end solution that aims to address these challenges
for efficient GNNs in resource constrained environments while avoiding the
oversmoothing problem in deep GNNs. We introduce a quantization based approach
for all stages of GNNs, from message passing in training to node
classification, compressing the model and enabling efficient processing. The
proposed GNN quantizer learns quantization ranges and reduces the model size
with comparable accuracy even under low-bit quantization. To scale with the
number of layers, we devise a message propagation mechanism in training that
controls layer-wise changes of similarities between neighboring nodes. This
objective is incorporated into a Lagrangian function with constraints and a
differential multiplier method is utilized to iteratively find optimal
embeddings. This mitigates oversmoothing and suppresses the quantization error
to a bound. Significant improvements are demonstrated over state-of-the-art
quantization methods and deep GNN approaches in both full-precision and
quantized models. The proposed quantizer demonstrates superior performance in
INT2 configurations across all stages of GNN, achieving a notable level of
accuracy. In contrast, existing quantization approaches fail to generate
satisfactory accuracy levels. Finally, the inference with INT2 and INT4
representations exhibits a speedup of 5.11 $\times$ and 4.70 $\times$ compared
to full precision counterparts, respectively.