Towards multi-modal anatomical landmark detection for ultrasound-guided
  brain tumor resection with contrastive learning
Soorena Salari Amirhossein Rasoulian Hassan Rivaz Yiming Xiao
Homologous anatomical landmarks between medical scans are instrumental in
quantitative assessment of image registration quality in various clinical
applications, such as MRI-ultrasound registration for tissue shift correction
in ultrasound-guided brain tumor resection. While manually identified landmark
pairs between MRI and ultrasound (US) have greatly facilitated the validation
of different registration algorithms for the task, the procedure requires
significant expertise, labor, and time, and can be prone to inter- and
intra-rater inconsistency. So far, many traditional and machine learning
approaches have been presented for anatomical landmark detection, but they
primarily focus on mono-modal applications. Unfortunately, despite the clinical
needs, inter-modal/contrast landmark detection has very rarely been attempted.
Therefore, we propose a novel contrastive learning framework to detect
corresponding landmarks between MRI and intra-operative US scans in
neurosurgery. Specifically, two convolutional neural networks were trained
jointly to encode image features in MRI and US scans to help match the US image
patch that contain the corresponding landmarks in the MRI. We developed and
validated the technique using the public RESECT database. With a mean landmark
detection accuracy of 5.88+-4.79 mm against 18.78+-4.77 mm with SIFT features,
the proposed method offers promising results for MRI-US landmark detection in
neurosurgical applications for the first time.