Multi-objective Deep Reinforcement Learning for Mobile Edge Computing
Ning Yang Junrui Wen Meng Zhang Ming Tang
Mobile edge computing (MEC) is essential for next-generation mobile network
applications that prioritize various performance metrics, including delays and
energy consumption. However, conventional single-objective scheduling solutions
cannot be directly applied to practical systems in which the preferences of
these applications (i.e., the weights of different objectives) are often
unknown or challenging to specify in advance. In this study, we address this
issue by formulating a multi-objective offloading problem for MEC with multiple
edges to minimize expected long-term energy consumption and transmission delay
while considering unknown preferences as parameters. To address the challenge
of unknown preferences, we design a multi-objective (deep) reinforcement
learning (MORL)-based resource scheduling scheme with proximal policy
optimization (PPO). In addition, we introduce a well-designed state encoding
method for constructing features for multiple edges in MEC systems, a
sophisticated reward function for accurately computing the utilities of delay
and energy consumption. Simulation results demonstrate that our proposed MORL
scheme enhances the hypervolume of the Pareto front by up to 233.1% compared to
benchmarks. Our full framework is available at
https://github.com/gracefulning/mec_morl_multipolicy.