Multi-Objective Optimization Using Adaptive Distributed Reinforcement
  Learning
Jing Tan Ramin Khalili Holger Karl
The Intelligent Transportation System (ITS) environment is known to be
dynamic and distributed, where participants (vehicle users, operators, etc.)
have multiple, changing and possibly conflicting objectives. Although
Reinforcement Learning (RL) algorithms are commonly applied to optimize ITS
applications such as resource management and offloading, most RL algorithms
focus on single objectives. In many situations, converting a multi-objective
problem into a single-objective one is impossible, intractable or insufficient,
making such RL algorithms inapplicable. We propose a multi-objective,
multi-agent reinforcement learning (MARL) algorithm with high learning
efficiency and low computational requirements, which automatically triggers
adaptive few-shot learning in a dynamic, distributed and noisy environment with
sparse and delayed reward. We test our algorithm in an ITS environment with
edge cloud computing. Empirical results show that the algorithm is quick to
adapt to new environments and performs better in all individual and system
metrics compared to the state-of-the-art benchmark. Our algorithm also
addresses various practical concerns with its modularized and asynchronous
online training method. In addition to the cloud simulation, we test our
algorithm on a single-board computer and show that it can make inference in 6
milliseconds.