JOINEDTrans: Prior Guided Multi-task Transformer for Joint Optic
  Disc/Cup Segmentation and Fovea Detection
Huaqing He Li Lin Zhiyuan Cai Pujin Cheng Xiaoying Tang
Deep learning-based image segmentation and detection models have largely
improved the efficiency of analyzing retinal landmarks such as optic disc (OD),
optic cup (OC), and fovea. However, factors including ophthalmic
disease-related lesions and low image quality issues may severely complicate
automatic OD/OC segmentation and fovea detection. Most existing works treat the
identification of each landmark as a single task, and take into account no
prior information. To address these issues, we propose a prior guided
multi-task transformer framework for joint OD/OC segmentation and fovea
detection, named JOINEDTrans. JOINEDTrans effectively combines various spatial
features of the fundus images, relieving the structural distortions induced by
lesions and other imaging issues. It contains a segmentation branch and a
detection branch. To be noted, we employ an encoder pretrained in a vessel
segmentation task to effectively exploit the positional relationship among
vessel, OD/OC, and fovea, successfully incorporating spatial prior into the
proposed JOINEDTrans framework. There are a coarse stage and a fine stage in
JOINEDTrans. In the coarse stage, OD/OC coarse segmentation and fovea heatmap
localization are obtained through a joint segmentation and detection module. In
the fine stage, we crop regions of interest for subsequent refinement and use
predictions obtained in the coarse stage to provide additional information for
better performance and faster convergence. Experimental results demonstrate
that JOINEDTrans outperforms existing state-of-the-art methods on the publicly
available GAMMA, REFUGE, and PALM fundus image datasets. We make our code
available at https://github.com/HuaqingHe/JOINEDTrans