Only Send What You Need: Learning to Communicate Efficiently in
  Federated Multilingual Machine Translation
Yun-Wei Chu Dong-Jun Han Christopher G. Brinton
Federated learning (FL) is a promising approach for solving multilingual
tasks, potentially enabling clients with their own language-specific data to
collaboratively construct a high-quality neural machine translation (NMT)
model. However, communication constraints in practical network systems present
challenges for exchanging large-scale NMT engines between FL parties. In this
paper, we propose a meta-learning-based adaptive parameter selection
methodology, MetaSend, that improves the communication efficiency of model
transmissions from clients during FL-based multilingual NMT training. Our
approach learns a dynamic threshold for filtering parameters prior to
transmission without compromising the NMT model quality, based on the tensor
deviations of clients between different FL rounds. Through experiments on two
NMT datasets with different language distributions, we demonstrate that
MetaSend obtains substantial improvements over baselines in translation quality
in the presence of a limited communication budget.