FedFusion: Manifold Driven Federated Learning for Multi-satellite and
  Multi-modality Fusion
DaiXun Li Weiying Xie Yunsong Li Leyuan Fang
Multi-satellite, multi-modality in-orbit fusion is a challenging task as it
explores the fusion representation of complex high-dimensional data under
limited computational resources. Deep neural networks can reveal the underlying
distribution of multi-modal remote sensing data, but the in-orbit fusion of
multimodal data is more difficult because of the limitations of different
sensor imaging characteristics, especially when the multimodal data follows
non-independent identically distribution (Non-IID) distributions. To address
this problem while maintaining classification performance, this paper proposes
a manifold-driven multi-modality fusion framework, FedFusion, which randomly
samples local data on each client to jointly estimate the prominent manifold
structure of shallow features of each client and explicitly compresses the
feature matrices into a low-rank subspace through cascading and additive
approaches, which is used as the feature input of the subsequent classifier.
Considering the physical space limitations of the satellite constellation, we
developed a multimodal federated learning module designed specifically for
manifold data in a deep latent space. This module achieves iterative updating
of the sub-network parameters of each client through global weighted averaging,
constructing a framework that can represent compact representations of each
client. The proposed framework surpasses existing methods in terms of
performance on three multimodal datasets, achieving a classification average
accuracy of 94.35$\%$ while compressing communication costs by a factor of 4.
Furthermore, extensive numerical evaluations of real-world satellite images
were conducted on the orbiting edge computing architecture based on Jetson TX2
industrial modules, which demonstrated that FedFusion significantly reduced
training time by 48.4 minutes (15.18%) while optimizing accuracy.}