SepRep-Net: Multi-source Free Domain Adaptation via Model Separation And
  Reparameterization
Ying Jin Jiaqi Wang Dahua Lin
We consider multi-source free domain adaptation, the problem of adapting
multiple existing models to a new domain without accessing the source data.
Among existing approaches, methods based on model ensemble are effective in
both the source and target domains, but incur significantly increased
computational costs. Towards this dilemma, in this work, we propose a novel
framework called SepRep-Net, which tackles multi-source free domain adaptation
via model Separation and Reparameterization.Concretely, SepRep-Net reassembled
multiple existing models to a unified network, while maintaining separate
pathways (Separation). During training, separate pathways are optimized in
parallel with the information exchange regularly performed via an additional
feature merging unit. With our specific design, these pathways can be further
reparameterized into a single one to facilitate inference (Reparameterization).
SepRep-Net is characterized by 1) effectiveness: competitive performance on the
target domain, 2) efficiency: low computational costs, and 3) generalizability:
maintaining more source knowledge than existing solutions. As a general
approach, SepRep-Net can be seamlessly plugged into various methods. Extensive
experiments validate the performance of SepRep-Net on mainstream benchmarks.