A streamlined Approach to Multimodal Few-Shot Class Incremental Learning
  for Fine-Grained Datasets
Thang Doan Sima Behpour Xin Li Wenbin He Liang Gou Liu Ren
Few-shot Class-Incremental Learning (FSCIL) poses the challenge of retaining
prior knowledge while learning from limited new data streams, all without
overfitting. The rise of Vision-Language models (VLMs) has unlocked numerous
applications, leveraging their existing knowledge to fine-tune on custom data.
However, training the whole model is computationally prohibitive, and VLMs
while being versatile in general domains still struggle with fine-grained
datasets crucial for many applications. We tackle these challenges with two
proposed simple modules. The first, Session-Specific Prompts (SSP), enhances
the separability of image-text embeddings across sessions. The second,
Hyperbolic distance, compresses representations of image-text pairs within the
same class while expanding those from different classes, leading to better
representations. Experimental results demonstrate an average 10-point increase
compared to baselines while requiring at least 8 times fewer trainable
parameters. This improvement is further underscored on our three newly
introduced fine-grained datasets.