A Spatial-Temporal Deformable Attention based Framework for Breast
  Lesion Detection in Videos
Chao Qin Jiale Cao Huazhu Fu Rao Muhammad Anwer Fahad Shahbaz Khan
Detecting breast lesion in videos is crucial for computer-aided diagnosis.
Existing video-based breast lesion detection approaches typically perform
temporal feature aggregation of deep backbone features based on the
self-attention operation. We argue that such a strategy struggles to
effectively perform deep feature aggregation and ignores the useful local
information. To tackle these issues, we propose a spatial-temporal deformable
attention based framework, named STNet. Our STNet introduces a spatial-temporal
deformable attention module to perform local spatial-temporal feature fusion.
The spatial-temporal deformable attention module enables deep feature
aggregation in each stage of both encoder and decoder. To further accelerate
the detection speed, we introduce an encoder feature shuffle strategy for
multi-frame prediction during inference. In our encoder feature shuffle
strategy, we share the backbone and encoder features, and shuffle encoder
features for decoder to generate the predictions of multiple frames. The
experiments on the public breast lesion ultrasound video dataset show that our
STNet obtains a state-of-the-art detection performance, while operating twice
as fast inference speed. The code and model are available at
https://github.com/AlfredQin/STNet.