Improving Heterogeneous Model Reuse by Density Estimation
Anke Tang Yong Luo Han Hu Fengxiang He Kehua Su Bo Du Yixin Chen Dacheng Tao
This paper studies multiparty learning, aiming to learn a model using the
private data of different participants. Model reuse is a promising solution for
multiparty learning, assuming that a local model has been trained for each
party. Considering the potential sample selection bias among different parties,
some heterogeneous model reuse approaches have been developed. However,
although pre-trained local classifiers are utilized in these approaches, the
characteristics of the local data are not well exploited. This motivates us to
estimate the density of local data and design an auxiliary model together with
the local classifiers for reuse. To address the scenarios where some local
models are not well pre-trained, we further design a multiparty cross-entropy
loss for calibration. Upon existing works, we address a challenging problem of
heterogeneous model reuse from a decision theory perspective and take advantage
of recent advances in density estimation. Experimental results on both
synthetic and benchmark data demonstrate the superiority of the proposed
method.