Breast Ultrasound Tumor Classification Using a Hybrid Multitask
  CNN-Transformer Network
Bryar Shareef Min Xian Aleksandar Vakanski Haotian Wang
Capturing global contextual information plays a critical role in breast
ultrasound (BUS) image classification. Although convolutional neural networks
(CNNs) have demonstrated reliable performance in tumor classification, they
have inherent limitations for modeling global and long-range dependencies due
to the localized nature of convolution operations. Vision Transformers have an
improved capability of capturing global contextual information but may distort
the local image patterns due to the tokenization operations. In this study, we
proposed a hybrid multitask deep neural network called Hybrid-MT-ESTAN,
designed to perform BUS tumor classification and segmentation using a hybrid
architecture composed of CNNs and Swin Transformer components. The proposed
approach was compared to nine BUS classification methods and evaluated using
seven quantitative metrics on a dataset of 3,320 BUS images. The results
indicate that Hybrid-MT-ESTAN achieved the highest accuracy, sensitivity, and
F1 score of 82.7%, 86.4%, and 86.0%, respectively.