Benchmarking Image Transformers for Prostate Cancer Detection from
  Ultrasound Data
Mohamed Harmanani Paul F. R. Wilson Fahimeh Fooladgar Amoon Jamzad Mahdi Gilany Minh Nguyen Nhat To Brian Wodlinger Purang Abolmaesumi Parvin Mousavi
PURPOSE: Deep learning methods for classifying prostate cancer (PCa) in
ultrasound images typically employ convolutional networks (CNNs) to detect
cancer in small regions of interest (ROI) along a needle trace region. However,
this approach suffers from weak labelling, since the ground-truth
histopathology labels do not describe the properties of individual ROIs.
Recently, multi-scale approaches have sought to mitigate this issue by
combining the context awareness of transformers with a CNN feature extractor to
detect cancer from multiple ROIs using multiple-instance learning (MIL). In
this work, we present a detailed study of several image transformer
architectures for both ROI-scale and multi-scale classification, and a
comparison of the performance of CNNs and transformers for ultrasound-based
prostate cancer classification. We also design a novel multi-objective learning
strategy that combines both ROI and core predictions to further mitigate label
noise. METHODS: We evaluate 3 image transformers on ROI-scale cancer
classification, then use the strongest model to tune a multi-scale classifier
with MIL. We train our MIL models using our novel multi-objective learning
strategy and compare our results to existing baselines. RESULTS: We find that
for both ROI-scale and multi-scale PCa detection, image transformer backbones
lag behind their CNN counterparts. This deficit in performance is even more
noticeable for larger models. When using multi-objective learning, we can
improve performance of MIL, with a 77.9% AUROC, a sensitivity of 75.9%, and a
specificity of 66.3%. CONCLUSION: Convolutional networks are better suited for
modelling sparse datasets of prostate ultrasounds, producing more robust
features than transformers in PCa detection. Multi-scale methods remain the
best architecture for this task, with multi-objective learning presenting an
effective way to improve performance.