CoinSeg: Contrast Inter- and Intra- Class Representations for
  Incremental Segmentation
Zekang Zhang Guangyu Gao Jianbo Jiao Chi Harold Liu Yunchao Wei
Class incremental semantic segmentation aims to strike a balance between the
model's stability and plasticity by maintaining old knowledge while adapting to
new concepts. However, most state-of-the-art methods use the freeze strategy
for stability, which compromises the model's plasticity.In contrast, releasing
parameter training for plasticity could lead to the best performance for all
categories, but this requires discriminative feature representation.Therefore,
we prioritize the model's plasticity and propose the Contrast inter- and
intra-class representations for Incremental Segmentation (CoinSeg), which
pursues discriminative representations for flexible parameter tuning. Inspired
by the Gaussian mixture model that samples from a mixture of Gaussian
distributions, CoinSeg emphasizes intra-class diversity with multiple
contrastive representation centroids. Specifically, we use mask proposals to
identify regions with strong objectness that are likely to be diverse
instances/centroids of a category. These mask proposals are then used for
contrastive representations to reinforce intra-class diversity. Meanwhile, to
avoid bias from intra-class diversity, we also apply category-level
pseudo-labels to enhance category-level consistency and inter-category
diversity. Additionally, CoinSeg ensures the model's stability and alleviates
forgetting through a specific flexible tuning strategy. We validate CoinSeg on
Pascal VOC 2012 and ADE20K datasets with multiple incremental scenarios and
achieve superior results compared to previous state-of-the-art methods,
especially in more challenging and realistic long-term scenarios. Code is
available at https://github.com/zkzhang98/CoinSeg.