Domain-Rectifying Adapter for Cross-Domain Few-Shot Segmentation
Jiapeng Su Qi Fan Guangming Lu Fanglin Chen Wenjie Pei
Few-shot semantic segmentation (FSS) has achieved great success on segmenting
objects of novel classes, supported by only a few annotated samples. However,
existing FSS methods often underperform in the presence of domain shifts,
especially when encountering new domain styles that are unseen during training.
It is suboptimal to directly adapt or generalize the entire model to new
domains in the few-shot scenario. Instead, our key idea is to adapt a small
adapter for rectifying diverse target domain styles to the source domain.
Consequently, the rectified target domain features can fittingly benefit from
the well-optimized source domain segmentation model, which is intently trained
on sufficient source domain data. Training domain-rectifying adapter requires
sufficiently diverse target domains. We thus propose a novel local-global style
perturbation method to simulate diverse potential target domains by
perturbating the feature channel statistics of the individual images and
collective statistics of the entire source domain, respectively. Additionally,
we propose a cyclic domain alignment module to facilitate the adapter
effectively rectifying domains using a reverse domain rectification
supervision. The adapter is trained to rectify the image features from diverse
synthesized target domains to align with the source domain. During testing on
target domains, we start by rectifying the image features and then conduct
few-shot segmentation on the domain-rectified features. Extensive experiments
demonstrate the effectiveness of our method, achieving promising results on
cross-domain few-shot semantic segmentation tasks. Our code is available at
https://github.com/Matt-Su/DR-Adapter.