Histopathology Whole Slide Image Analysis with Heterogeneous Graph
  Representation Learning
Tsai Hor Chan Fernando Julio Cendra Lan Ma Guosheng Yin Lequan Yu
Graph-based methods have been extensively applied to whole-slide
histopathology image (WSI) analysis due to the advantage of modeling the
spatial relationships among different entities. However, most of the existing
methods focus on modeling WSIs with homogeneous graphs (e.g., with homogeneous
node type). Despite their successes, these works are incapable of mining the
complex structural relations between biological entities (e.g., the diverse
interaction among different cell types) in the WSI. We propose a novel
heterogeneous graph-based framework to leverage the inter-relationships among
different types of nuclei for WSI analysis. Specifically, we formulate the WSI
as a heterogeneous graph with "nucleus-type" attribute to each node and a
semantic similarity attribute to each edge. We then present a new
heterogeneous-graph edge attribute transformer (HEAT) to take advantage of the
edge and node heterogeneity during massage aggregating. Further, we design a
new pseudo-label-based semantic-consistent pooling mechanism to obtain
graph-level features, which can mitigate the over-parameterization issue of
conventional cluster-based pooling. Additionally, observing the limitations of
existing association-based localization methods, we propose a causal-driven
approach attributing the contribution of each node to improve the
interpretability of our framework. Extensive experiments on three public TCGA
benchmark datasets demonstrate that our framework outperforms the
state-of-the-art methods with considerable margins on various tasks. Our codes
are available at https://github.com/HKU-MedAI/WSI-HGNN.