Robust Tumor Segmentation with Hyperspectral Imaging and Graph Neural
  Networks
Mayar Lotfy Anna Alperovich Tommaso Giannantonio Bjorn Barz Xiaohan Zhang Felix Holm Nassir Navab Felix Boehm Carolin Schwamborn Thomas K. Hoffmann Patrick J. Schuler
Segmenting the boundary between tumor and healthy tissue during surgical
cancer resection poses a significant challenge. In recent years, Hyperspectral
Imaging (HSI) combined with Machine Learning (ML) has emerged as a promising
solution. However, due to the extensive information contained within the
spectral domain, most ML approaches primarily classify individual HSI
(super-)pixels, or tiles, without taking into account their spatial context. In
this paper, we propose an improved methodology that leverages the spatial
context of tiles for more robust and smoother segmentation. To address the
irregular shapes of tiles, we utilize Graph Neural Networks (GNNs) to propagate
context information across neighboring regions. The features for each tile
within the graph are extracted using a Convolutional Neural Network (CNN),
which is trained simultaneously with the subsequent GNN. Moreover, we
incorporate local image quality metrics into the loss function to enhance the
training procedure's robustness against low-quality regions in the training
images. We demonstrate the superiority of our proposed method using a clinical
ex vivo dataset consisting of 51 HSI images from 30 patients. Despite the
limited dataset, the GNN-based model significantly outperforms context-agnostic
approaches, accurately distinguishing between healthy and tumor tissues, even
in images from previously unseen patients. Furthermore, we show that our
carefully designed loss function, accounting for local image quality, results
in additional improvements. Our findings demonstrate that context-aware GNN
algorithms can robustly find tumor demarcations on HSI images, ultimately
contributing to better surgery success and patient outcome.