Cell Graph Transformer for Nuclei Classification
Wei Lou Guanbin Li Xiang Wan Haofeng Li
Nuclei classification is a critical step in computer-aided diagnosis with
histopathology images. In the past, various methods have employed graph neural
networks (GNN) to analyze cell graphs that model inter-cell relationships by
considering nuclei as vertices. However, they are limited by the GNN mechanism
that only passes messages among local nodes via fixed edges. To address the
issue, we develop a cell graph transformer (CGT) that treats nodes and edges as
input tokens to enable learnable adjacency and information exchange among all
nodes. Nevertheless, training the transformer with a cell graph presents
another challenge. Poorly initialized features can lead to noisy self-attention
scores and inferior convergence, particularly when processing the cell graphs
with numerous connections. Thus, we further propose a novel topology-aware
pretraining method that leverages a graph convolutional network (GCN) to learn
a feature extractor. The pre-trained features may suppress unreasonable
correlations and hence ease the finetuning of CGT. Experimental results suggest
that the proposed cell graph transformer with topology-aware pretraining
significantly improves the nuclei classification results, and achieves the
state-of-the-art performance. Code and models are available at
https://github.com/lhaof/CGT