MoP-CLIP: A Mixture of Prompt-Tuned CLIP Models for Domain Incremental
  Learning
Julien Nicolas Florent Chiaroni Imtiaz Ziko Ola Ahmad Christian Desrosiers Jose Dolz
Despite the recent progress in incremental learning, addressing catastrophic
forgetting under distributional drift is still an open and important problem.
Indeed, while state-of-the-art domain incremental learning (DIL) methods
perform satisfactorily within known domains, their performance largely degrades
in the presence of novel domains. This limitation hampers their
generalizability, and restricts their scalability to more realistic settings
where train and test data are drawn from different distributions. To address
these limitations, we present a novel DIL approach based on a mixture of
prompt-tuned CLIP models (MoP-CLIP), which generalizes the paradigm of
S-Prompting to handle both in-distribution and out-of-distribution data at
inference. In particular, at the training stage we model the features
distribution of every class in each domain, learning individual text and visual
prompts to adapt to a given domain. At inference, the learned distributions
allow us to identify whether a given test sample belongs to a known domain,
selecting the correct prompt for the classification task, or from an unseen
domain, leveraging a mixture of the prompt-tuned CLIP models. Our empirical
evaluation reveals the poor performance of existing DIL methods under domain
shift, and suggests that the proposed MoP-CLIP performs competitively in the
standard DIL settings while outperforming state-of-the-art methods in OOD
scenarios. These results demonstrate the superiority of MoP-CLIP, offering a
robust and general solution to the problem of domain incremental learning.