Dynamic Encoding and Decoding of Information for Split Learning in
  Mobile-Edge Computing: Leveraging Information Bottleneck Theory
Omar Alhussein Moshi Wei Arashmid Akhavain
Split learning is a privacy-preserving distributed learning paradigm in which
an ML model (e.g., a neural network) is split into two parts (i.e., an encoder
and a decoder). The encoder shares so-called latent representation, rather than
raw data, for model training. In mobile-edge computing, network functions (such
as traffic forecasting) can be trained via split learning where an encoder
resides in a user equipment (UE) and a decoder resides in the edge network.
Based on the data processing inequality and the information bottleneck (IB)
theory, we present a new framework and training mechanism to enable a dynamic
balancing of the transmission resource consumption with the informativeness of
the shared latent representations, which directly impacts the predictive
performance. The proposed training mechanism offers an encoder-decoder neural
network architecture featuring multiple modes of complexity-relevance
tradeoffs, enabling tunable performance. The adaptability can accommodate
varying real-time network conditions and application requirements, potentially
reducing operational expenditure and enhancing network agility. As a proof of
concept, we apply the training mechanism to a millimeter-wave (mmWave)-enabled
throughput prediction problem. We also offer new insights and highlight some
challenges related to recurrent neural networks from the perspective of the IB
theory. Interestingly, we find a compression phenomenon across the temporal
domain of the sequential model, in addition to the compression phase that
occurs with the number of training epochs.