Seeing Beyond Cancer: Multi-Institutional Validation of Object
  Localization and 3D Semantic Segmentation using Deep Learning for Breast MRI
Arda Pekis Vignesh Kannan Evandros Kaklamanos Anu Antony Snehal Patel Tyler Earnest
The clinical management of breast cancer depends on an accurate understanding
of the tumor and its anatomical context to adjacent tissues and landmark
structures. This context may be provided by semantic segmentation methods;
however, previous works have been largely limited to a singular focus on the
tumor alone and rarely other tissue types. In contrast, we present a method
that exploits tissue-tissue interactions to accurately segment every major
tissue type in the breast including: chest wall, skin, adipose tissue,
fibroglandular tissue, vasculature and tumor via standard-of-care Dynamic
Contrast Enhanced MRI. Comparing our method to prior state-of-the-art, we
achieved a superior Dice score on tumor segmentation while maintaining
competitive performance on other studied tissues across multiple institutions.
Briefly, our method proceeds by localizing the tumor using 2D object detectors,
then segmenting the tumor and surrounding tissues independently using two 3D
U-nets, and finally integrating these results while mitigating false positives
by checking for anatomically plausible tissue-tissue contacts. The object
detection models were pre-trained on ImageNet and COCO, and operated on MIP
(maximum intensity projection) images in the axial and sagittal planes,
establishing a 3D tumor bounding box. By integrating multiple relevant
peri-tumoral tissues, our work enables clinical applications in breast cancer
staging, prognosis and surgical planning.