Universal Metric Learning with Parameter-Efficient Transfer Learning
Sungyeon Kim Donghyun Kim Suha Kwak
A common practice in metric learning is to train and test an embedding model
for each dataset. This dataset-specific approach fails to simulate real-world
scenarios that involve multiple heterogeneous distributions of data. In this
regard, we introduce a novel metric learning paradigm, called Universal Metric
Learning (UML), which learns a unified distance metric capable of capturing
relations across multiple data distributions. UML presents new challenges, such
as imbalanced data distribution and bias towards dominant distributions. To
address these challenges, we propose Parameter-efficient Universal Metric
leArning (PUMA), which consists of a pre-trained frozen model and two
additional modules, stochastic adapter and prompt pool. These modules enable to
capture dataset-specific knowledge while avoiding bias towards dominant
distributions. Additionally, we compile a new universal metric learning
benchmark with a total of 8 different datasets. PUMA outperformed the
state-of-the-art dataset-specific models while using about 69 times fewer
trainable parameters.