Vision Transformer-based Multimodal Feature Fusion Network for Lymphoma
  Segmentation on PET/CT Images
Huan Huang Liheng Qiu Shenmiao Yang Longxi Li Jiaofen Nan Yanting Li Chuang Han Fubao Zhu Chen Zhao Weihua Zhou
Background: Diffuse large B-cell lymphoma (DLBCL) segmentation is a challenge
in medical image analysis. Traditional segmentation methods for lymphoma
struggle with the complex patterns and the presence of DLBCL lesions.
Objective: We aim to develop an accurate method for lymphoma segmentation with
18F-Fluorodeoxyglucose positron emission tomography (PET) and computed
tomography (CT) images. Methods: Our lymphoma segmentation approach combines a
vision transformer with dual encoders, adeptly fusing PET and CT data via
multimodal cross-attention fusion (MMCAF) module. In this study, PET and CT
data from 165 DLBCL patients were analyzed. A 5-fold cross-validation was
employed to evaluate the performance and generalization ability of our method.
Ground truths were annotated by experienced nuclear medicine experts. We
calculated the total metabolic tumor volume (TMTV) and performed a statistical
analysis on our results. Results: The proposed method exhibited accurate
performance in DLBCL lesion segmentation, achieving a Dice similarity
coefficient of 0.9173$\pm$0.0071, a Hausdorff distance of 2.71$\pm$0.25mm, a
sensitivity of 0.9462$\pm$0.0223, and a specificity of 0.9986$\pm$0.0008.
Additionally, a Pearson correlation coefficient of 0.9030$\pm$0.0179 and an
R-square of 0.8586$\pm$0.0173 were observed in TMTV when measured on manual
annotation compared to our segmentation results. Conclusion: This study
highlights the advantages of MMCAF and vision transformer for lymphoma
segmentation using PET and CT, offering great promise for computer-aided
lymphoma diagnosis and treatment.