A Multi-Scale Spatial Transformer U-Net for Simultaneously Automatic
  Reorientation and Segmentation of 3D Nuclear Cardiac Images
Yangfan Ni Duo Zhang Gege Ma Lijun Lu Zhongke Huang Wentao Zhu
Accurate reorientation and segmentation of the left ventricular (LV) is
essential for the quantitative analysis of myocardial perfusion imaging (MPI),
in which one critical step is to reorient the reconstructed transaxial nuclear
cardiac images into standard short-axis slices for subsequent image processing.
Small-scale LV myocardium (LV-MY) region detection and the diverse cardiac
structures of individual patients pose challenges to LV segmentation operation.
To mitigate these issues, we propose an end-to-end model, named as multi-scale
spatial transformer UNet (MS-ST-UNet), that involves the multi-scale spatial
transformer network (MSSTN) and multi-scale UNet (MSUNet) modules to perform
simultaneous reorientation and segmentation of LV region from nuclear cardiac
images. The proposed method is trained and tested using two different nuclear
cardiac image modalities: 13N-ammonia PET and 99mTc-sestamibi SPECT. We use a
multi-scale strategy to generate and extract image features with different
scales. Our experimental results demonstrate that the proposed method
significantly improves the reorientation and segmentation performance. This
joint learning framework promotes mutual enhancement between reorientation and
segmentation tasks, leading to cutting edge performance and an efficient image
processing workflow. The proposed end-to-end deep network has the potential to
reduce the burden of manual delineation for cardiac images, thereby providing
multimodal quantitative analysis assistance for physicists.