GlanceSeg: Real-time microaneurysm lesion segmentation with
  gaze-map-guided foundation model for early detection of diabetic retinopathy
Hongyang Jiang Mengdi Gao Zirong Liu Chen Tang Xiaoqing Zhang Shuai Jiang Wu Yuan Jiang Liu
Early-stage diabetic retinopathy (DR) presents challenges in clinical
diagnosis due to inconspicuous and minute microangioma lesions, resulting in
limited research in this area. Additionally, the potential of emerging
foundation models, such as the segment anything model (SAM), in medical
scenarios remains rarely explored. In this work, we propose a
human-in-the-loop, label-free early DR diagnosis framework called GlanceSeg,
based on SAM. GlanceSeg enables real-time segmentation of microangioma lesions
as ophthalmologists review fundus images. Our human-in-the-loop framework
integrates the ophthalmologist's gaze map, allowing for rough localization of
minute lesions in fundus images. Subsequently, a saliency map is generated
based on the located region of interest, which provides prompt points to assist
the foundation model in efficiently segmenting microangioma lesions. Finally, a
domain knowledge filter refines the segmentation of minute lesions. We
conducted experiments on two newly-built public datasets, i.e., IDRiD and
Retinal-Lesions, and validated the feasibility and superiority of GlanceSeg
through visualized illustrations and quantitative measures. Additionally, we
demonstrated that GlanceSeg improves annotation efficiency for clinicians and
enhances segmentation performance through fine-tuning using annotations. This
study highlights the potential of GlanceSeg-based annotations for self-model
optimization, leading to enduring performance advancements through continual
learning.