Distributionally Robust Transfer Learning
Xin Xiong Zijian Guo Tianxi Cai
Many existing transfer learning methods rely on leveraging information from
source data that closely resembles the target data. However, this approach
often overlooks valuable knowledge that may be present in different yet
potentially related auxiliary samples. When dealing with a limited amount of
target data and a diverse range of source models, our paper introduces a novel
approach, Distributionally Robust Optimization for Transfer Learning
(TransDRO), that breaks free from strict similarity constraints. TransDRO is
designed to optimize the most adversarial loss within an uncertainty set,
defined as a collection of target populations generated as a convex combination
of source distributions that guarantee excellent prediction performances for
the target data. TransDRO effectively bridges the realms of transfer learning
and distributional robustness prediction models. We establish the
identifiability of TransDRO and its interpretation as a weighted average of
source models closest to the baseline model. We also show that TransDRO
achieves a faster convergence rate than the model fitted with the target data.
Our comprehensive numerical studies and analysis of multi-institutional
electronic health records data using TransDRO further substantiate the
robustness and accuracy of TransDRO, highlighting its potential as a powerful
tool in transfer learning applications.