DGMamba: Domain Generalization via Generalized State Space Model
Shaocong Long Qianyu Zhou Xiangtai Li Xuequan Lu Chenhao Ying Yuan Luo Lizhuang Ma Shuicheng Yan
Domain generalization~(DG) aims at solving distribution shift problems in
various scenes. Existing approaches are based on Convolution Neural Networks
(CNNs) or Vision Transformers (ViTs), which suffer from limited receptive
fields or quadratic complexities issues. Mamba, as an emerging state space
model (SSM), possesses superior linear complexity and global receptive fields.
Despite this, it can hardly be applied to DG to address distribution shifts,
due to the hidden state issues and inappropriate scan mechanisms. In this
paper, we propose a novel framework for DG, named DGMamba, that excels in
strong generalizability toward unseen domains and meanwhile has the advantages
of global receptive fields, and efficient linear complexity. Our DGMamba
compromises two core components: Hidden State Suppressing~(HSS) and
Semantic-aware Patch refining~(SPR). In particular, HSS is introduced to
mitigate the influence of hidden states associated with domain-specific
features during output prediction. SPR strives to encourage the model to
concentrate more on objects rather than context, consisting of two designs:
Prior-Free Scanning~(PFS), and Domain Context Interchange~(DCI). Concretely,
PFS aims to shuffle the non-semantic patches within images, creating more
flexible and effective sequences from images, and DCI is designed to regularize
Mamba with the combination of mismatched non-semantic and semantic information
by fusing patches among domains. Extensive experiments on four commonly used DG
benchmarks demonstrate that the proposed DGMamba achieves remarkably superior
results to state-of-the-art models. The code will be made publicly available.