Graph Relation Distillation for Efficient Biomedical Instance
  Segmentation
Xiaoyu Liu Yueyi Zhang Zhiwei Xiong Wei Huang Bo Hu Xiaoyan Sun Feng Wu
Instance-aware embeddings predicted by deep neural networks have
revolutionized biomedical instance segmentation, but its resource requirements
are substantial. Knowledge distillation offers a solution by transferring
distilled knowledge from heavy teacher networks to lightweight yet
high-performance student networks. However, existing knowledge distillation
methods struggle to extract knowledge for distinguishing instances and overlook
global relation information. To address these challenges, we propose a graph
relation distillation approach for efficient biomedical instance segmentation,
which considers three essential types of knowledge: instance-level features,
instance relations, and pixel-level boundaries. We introduce two graph
distillation schemes deployed at both the intra-image level and the inter-image
level: instance graph distillation (IGD) and affinity graph distillation (AGD).
IGD constructs a graph representing instance features and relations,
transferring these two types of knowledge by enforcing instance graph
consistency. AGD constructs an affinity graph representing pixel relations to
capture structured knowledge of instance boundaries, transferring
boundary-related knowledge by ensuring pixel affinity consistency. Experimental
results on a number of biomedical datasets validate the effectiveness of our
approach, enabling student models with less than $ 1\%$ parameters and less
than $10\%$ inference time while achieving promising performance compared to
teacher models.