Multi-Scale Prototypical Transformer for Whole Slide Image
  Classification
Saisai Ding Jun Wang Juncheng Li Jun Shi
Whole slide image (WSI) classification is an essential task in computational
pathology. Despite the recent advances in multiple instance learning (MIL) for
WSI classification, accurate classification of WSIs remains challenging due to
the extreme imbalance between the positive and negative instances in bags, and
the complicated pre-processing to fuse multi-scale information of WSI. To this
end, we propose a novel multi-scale prototypical Transformer (MSPT) for WSI
classification, which includes a prototypical Transformer (PT) module and a
multi-scale feature fusion module (MFFM). The PT is developed to reduce
redundant instances in bags by integrating prototypical learning into the
Transformer architecture. It substitutes all instances with cluster prototypes,
which are then re-calibrated through the self-attention mechanism of the
Trans-former. Thereafter, an MFFM is proposed to fuse the clustered prototypes
of different scales, which employs MLP-Mixer to enhance the information
communication between prototypes. The experimental results on two public WSI
datasets demonstrate that the proposed MSPT outperforms all the compared
algorithms, suggesting its potential applications.