DPStyler: Dynamic PromptStyler for Source-Free Domain Generalization
Yunlong Tang Yuxuan Wan Lei Qi Xin Geng
Source-Free Domain Generalization (SFDG) aims to develop a model that works
for unseen target domains without relying on any source domain. Recent work,
PromptStyler, employs text prompts to simulate different distribution shifts in
the joint vision-language space, allowing the model to generalize effectively
to unseen domains without using any images. However, 1) PromptStyler's style
generation strategy has limitations, as all style patterns are fixed after the
first training phase. This leads to the training set in the second training
phase being restricted to a limited set of styles. Additionally, 2) the frozen
text encoder in PromptStyler result in the encoder's output varying with the
style of the input text prompts, making it difficult for the model to learn
domain-invariant features. In this paper, we introduce Dynamic PromptStyler
(DPStyler), comprising Style Generation and Style Removal modules to address
these issues. The Style Generation module refreshes all styles at every
training epoch, while the Style Removal module eliminates variations in the
encoder's output features caused by input styles. Moreover, since the Style
Generation module, responsible for generating style word vectors using random
sampling or style mixing, makes the model sensitive to input text prompts, we
introduce a model ensemble method to mitigate this sensitivity. Extensive
experiments demonstrate that our framework outperforms state-of-the-art methods
on benchmark datasets.