FedLPPA: Learning Personalized Prompt and Aggregation for Federated
  Weakly-supervised Medical Image Segmentation
Li Lin Yixiang Liu Jiewei Wu Pujin Cheng Zhiyuan Cai Kenneth K. Y. Wong Xiaoying Tang
Federated learning (FL) effectively mitigates the data silo challenge brought
about by policies and privacy concerns, implicitly harnessing more data for
deep model training. However, traditional centralized FL models grapple with
diverse multi-center data, especially in the face of significant data
heterogeneity, notably in medical contexts. In the realm of medical image
segmentation, the growing imperative to curtail annotation costs has amplified
the importance of weakly-supervised techniques which utilize sparse annotations
such as points, scribbles, etc. A pragmatic FL paradigm shall accommodate
diverse annotation formats across different sites, which research topic remains
under-investigated. In such context, we propose a novel personalized FL
framework with learnable prompt and aggregation (FedLPPA) to uniformly leverage
heterogeneous weak supervision for medical image segmentation. In FedLPPA, a
learnable universal knowledge prompt is maintained, complemented by multiple
learnable personalized data distribution prompts and prompts representing the
supervision sparsity. Integrated with sample features through a dual-attention
mechanism, those prompts empower each local task decoder to adeptly adjust to
both the local distribution and the supervision form. Concurrently, a
dual-decoder strategy, predicated on prompt similarity, is introduced for
enhancing the generation of pseudo-labels in weakly-supervised learning,
alleviating overfitting and noise accumulation inherent to local data, while an
adaptable aggregation method is employed to customize the task decoder on a
parameter-wise basis. Extensive experiments on three distinct medical image
segmentation tasks involving different modalities underscore the superiority of
FedLPPA, with its efficacy closely parallels that of fully supervised
centralized training. Our code and data will be available.