Knowledge Distillation Based Semantic Communications For Multiple Users
Chenguang Liu Yuxin Zhou Yunfei Chen Shuang-Hua Yang
Deep learning (DL) has shown great potential in revolutionizing the
traditional communications system. Many applications in communications have
adopted DL techniques due to their powerful representation ability. However,
the learning-based methods can be dependent on the training dataset and perform
worse on unseen interference due to limited model generalizability and
complexity. In this paper, we consider the semantic communication (SemCom)
system with multiple users, where there is a limited number of training samples
and unexpected interference. To improve the model generalization ability and
reduce the model size, we propose a knowledge distillation (KD) based system
where Transformer based encoder-decoder is implemented as the semantic
encoder-decoder and fully connected neural networks are implemented as the
channel encoder-decoder. Specifically, four types of knowledge transfer and
model compression are analyzed. Important system and model parameters are
considered, including the level of noise and interference, the number of
interfering users and the size of the encoder and decoder. Numerical results
demonstrate that KD significantly improves the robustness and the
generalization ability when applied to unexpected interference, and it reduces
the performance loss when compressing the model size.