Distilling Knowledge from Resource Management Algorithms to Neural
  Networks: A Unified Training Assistance Approach
Longfei Ma Nan Cheng Xiucheng Wang Zhisheng Yin Haibo Zhou Wei Quan
As a fundamental problem, numerous methods are dedicated to the optimization
of signal-to-interference-plus-noise ratio (SINR), in a multi-user setting.
Although traditional model-based optimization methods achieve strong
performance, the high complexity raises the research of neural network (NN)
based approaches to trade-off the performance and complexity. To fully leverage
the high performance of traditional model-based methods and the low complexity
of the NN-based method, a knowledge distillation (KD) based algorithm
distillation (AD) method is proposed in this paper to improve the performance
and convergence speed of the NN-based method, where traditional SINR
optimization methods are employed as ``teachers" to assist the training of NNs,
which are ``students", thus enhancing the performance of unsupervised and
reinforcement learning techniques. This approach aims to alleviate common
issues encountered in each of these training paradigms, including the
infeasibility of obtaining optimal solutions as labels and overfitting in
supervised learning, ensuring higher convergence performance in unsupervised
learning, and improving training efficiency in reinforcement learning.
Simulation results demonstrate the enhanced performance of the proposed
AD-based methods compared to traditional learning methods. Remarkably, this
research paves the way for the integration of traditional optimization insights
and emerging NN techniques in wireless communication system optimization.