Transfer and Alignment Network for Generalized Category Discovery
Wenbin An Feng Tian Wenkai Shi Yan Chen Yaqiang Wu Qianying Wang Ping Chen
Generalized Category Discovery is a crucial real-world task. Despite the
improved performance on known categories, current methods perform poorly on
novel categories. We attribute the poor performance to two reasons: biased
knowledge transfer between labeled and unlabeled data and noisy representation
learning on the unlabeled data. To mitigate these two issues, we propose a
Transfer and Alignment Network (TAN), which incorporates two knowledge transfer
mechanisms to calibrate the biased knowledge and two feature alignment
mechanisms to learn discriminative features. Specifically, we model different
categories with prototypes and transfer the prototypes in labeled data to
correct model bias towards known categories. On the one hand, we pull instances
with known categories in unlabeled data closer to these prototypes to form more
compact clusters and avoid boundary overlap between known and novel categories.
On the other hand, we use these prototypes to calibrate noisy prototypes
estimated from unlabeled data based on category similarities, which allows for
more accurate estimation of prototypes for novel categories that can be used as
reliable learning targets later. After knowledge transfer, we further propose
two feature alignment mechanisms to acquire both instance- and category-level
knowledge from unlabeled data by aligning instance features with both augmented
features and the calibrated prototypes, which can boost model performance on
both known and novel categories with less noise. Experiments on three benchmark
datasets show that our model outperforms SOTA methods, especially on novel
categories. Theoretical analysis is provided for an in-depth understanding of
our model in general. Our code and data are available at
https://github.com/Lackel/TAN.