MAST: Video Polyp Segmentation with a Mixture-Attention Siamese
  Transformer
Geng Chen Junqing Yang Xiaozhou Pu Ge-Peng Ji Huan Xiong Yongsheng Pan Hengfei Cui Yong Xia
Accurate segmentation of polyps from colonoscopy videos is of great
significance to polyp treatment and early prevention of colorectal cancer.
However, it is challenging due to the difficulties associated with modelling
long-range spatio-temporal relationships within a colonoscopy video. In this
paper, we address this challenging task with a novel Mixture-Attention Siamese
Transformer (MAST), which explicitly models the long-range spatio-temporal
relationships with a mixture-attention mechanism for accurate polyp
segmentation. Specifically, we first construct a Siamese transformer
architecture to jointly encode paired video frames for their feature
representations. We then design a mixture-attention module to exploit the
intra-frame and inter-frame correlations, enhancing the features with rich
spatio-temporal relationships. Finally, the enhanced features are fed to two
parallel decoders for predicting the segmentation maps. To the best of our
knowledge, our MAST is the first transformer model dedicated to video polyp
segmentation. Extensive experiments on the large-scale SUN-SEG benchmark
demonstrate the superior performance of MAST in comparison with the
cutting-edge competitors. Our code is publicly available at
https://github.com/Junqing-Yang/MAST.