Advancing Incremental Few-shot Semantic Segmentation via Semantic-guided
  Relation Alignment and Adaptation
Yuan Zhou Xin Chen Yanrong Guo Shijie Hao Richang Hong Qi Tian
Incremental few-shot semantic segmentation (IFSS) aims to incrementally
extend a semantic segmentation model to novel classes according to only a few
pixel-level annotated data, while preserving its segmentation capability on
previously learned base categories. This task faces a severe semantic-aliasing
issue between base and novel classes due to data imbalance, which makes
segmentation results unsatisfactory. To alleviate this issue, we propose the
Semantic-guided Relation Alignment and Adaptation (SRAA) method that fully
considers the guidance of prior semantic information. Specifically, we first
conduct Semantic Relation Alignment (SRA) in the base step, so as to
semantically align base class representations to their semantics. As a result,
the embeddings of base classes are constrained to have relatively low semantic
correlations to categories that are different from them. Afterwards, based on
the semantically aligned base categories, Semantic-Guided Adaptation (SGA) is
employed during the incremental learning stage. It aims to ensure affinities
between visual and semantic embeddings of encountered novel categories, thereby
making the feature representations be consistent with their semantic
information. In this way, the semantic-aliasing issue can be suppressed. We
evaluate our model on the PASCAL VOC 2012 and the COCO dataset. The
experimental results on both these two datasets exhibit its competitive
performance, which demonstrates the superiority of our method.