Incremental Learning for Heterogeneous Structure Segmentation in Brain
  Tumor MRI
Xiaofeng Liu Helen A. Shih Fangxu Xing Emiliano Santarnecchi Georges El Fakhri Jonghye Woo
Deep learning (DL) models for segmenting various anatomical structures have
achieved great success via a static DL model that is trained in a single source
domain. Yet, the static DL model is likely to perform poorly in a continually
evolving environment, requiring appropriate model updates. In an incremental
learning setting, we would expect that well-trained static models are updated,
following continually evolving target domain data -- e.g., additional lesions
or structures of interest -- collected from different sites, without
catastrophic forgetting. This, however, poses challenges, due to distribution
shifts, additional structures not seen during the initial model training, and
the absence of training data in a source domain. To address these challenges,
in this work, we seek to progressively evolve an ``off-the-shelf" trained
segmentation model to diverse datasets with additional anatomical categories in
a unified manner. Specifically, we first propose a divergence-aware dual-flow
module with balanced rigidity and plasticity branches to decouple old and new
tasks, which is guided by continuous batch renormalization. Then, a
complementary pseudo-label training scheme with self-entropy regularized
momentum MixUp decay is developed for adaptive network optimization. We
evaluated our framework on a brain tumor segmentation task with continually
changing target domains -- i.e., new MRI scanners/modalities with incremental
structures. Our framework was able to well retain the discriminability of
previously learned structures, hence enabling the realistic life-long
segmentation model extension along with the widespread accumulation of big
medical data.