To transfer or not transfer: Unified transferability metric and analysis
Qianshan Zhan Xiao-Jun Zeng
In transfer learning, transferability is one of the most fundamental
problems, which aims to evaluate the effectiveness of arbitrary transfer tasks.
Existing research focuses on classification tasks and neglects domain or task
differences. More importantly, there is a lack of research to determine whether
to transfer or not. To address these, we propose a new analytical approach and
metric, Wasserstein Distance based Joint Estimation (WDJE), for transferability
estimation and determination in a unified setting: classification and
regression problems with domain and task differences. The WDJE facilitates
decision-making on whether to transfer or not by comparing the target risk with
and without transfer. To enable the comparison, we approximate the target
transfer risk by proposing a non-symmetric, easy-to-understand and
easy-to-calculate target risk bound that is workable even with limited target
labels. The proposed bound relates the target risk to source model performance,
domain and task differences based on Wasserstein distance. We also extend our
bound into unsupervised settings and establish the generalization bound from
finite empirical samples. Our experiments in image classification and remaining
useful life regression prediction illustrate the effectiveness of the WDJE in
determining whether to transfer or not, and the proposed bound in approximating
the target transfer risk.