Comprehensive Multimodal Segmentation in Medical Imaging: Combining
  YOLOv8 with SAM and HQ-SAM Models
Sumit Pandey Kuan-Fu Chen Erik B. Dam
This paper introduces a comprehensive approach for segmenting regions of
interest (ROI) in diverse medical imaging datasets, encompassing ultrasound, CT
scans, and X-ray images. The proposed method harnesses the capabilities of the
YOLOv8 model for approximate boundary box detection across modalities,
alongside the Segment Anything Model (SAM) and High Quality (HQ) SAM for fully
automatic and precise segmentation. To generate boundary boxes, the YOLOv8
model was trained using a limited set of 100 images and masks from each
modality. The results obtained from our approach are extensively computed and
analyzed, demonstrating its effectiveness and potential in medical image
analysis. Various evaluation metrics, including precision, recall, F1 score,
and Dice Score, were employed to quantify the accuracy of the segmentation
results. A comparative analysis was conducted to assess the individual and
combined performance of the YOLOv8, YOLOv8+SAM, and YOLOv8+HQ-SAM models. The
results indicate that the SAM model performs better than the other two models,
exhibiting higher segmentation accuracy and overall performance. While HQ-SAM
offers potential advantages, its incremental gains over the standard SAM model
may not justify the additional computational cost. The YOLOv8+SAM model shows
promise for enhancing medical image segmentation and its clinical implications.