LOTUS: Learning to Optimize Task-based US representations
Yordanka Velikova Mohammad Farid Azampour Walter Simson Vanessa Gonzalez Duque Nassir Navab
Anatomical segmentation of organs in ultrasound images is essential to many
clinical applications, particularly for diagnosis and monitoring. Existing deep
neural networks require a large amount of labeled data for training in order to
achieve clinically acceptable performance. Yet, in ultrasound, due to
characteristic properties such as speckle and clutter, it is challenging to
obtain accurate segmentation boundaries, and precise pixel-wise labeling of
images is highly dependent on the expertise of physicians. In contrast, CT
scans have higher resolution and improved contrast, easing organ
identification. In this paper, we propose a novel approach for learning to
optimize task-based ultra-sound image representations. Given annotated CT
segmentation maps as a simulation medium, we model acoustic propagation through
tissue via ray-casting to generate ultrasound training data. Our ultrasound
simulator is fully differentiable and learns to optimize the parameters for
generating physics-based ultrasound images guided by the downstream
segmentation task. In addition, we train an image adaptation network between
real and simulated images to achieve simultaneous image synthesis and automatic
segmentation on US images in an end-to-end training setting. The proposed
method is evaluated on aorta and vessel segmentation tasks and shows promising
quantitative results. Furthermore, we also conduct qualitative results of
optimized image representations on other organs.