Realtime Spectrum Monitoring via Reinforcement Learning -- A Comparison
  Between Q-Learning and Heuristic Methods
Tobias Braun Tobias Korzyzkowske Larissa Putzar Jan Mietzner Peter A. Hoeher
Due to technological advances in the field of radio technology and its
availability, the number of interference signals in the radio spectrum is
continuously increasing. Interference signals must be detected in a timely
fashion, in order to maintain standards and keep emergency frequencies open. To
this end, specialized (multi-channel) receivers are used for spectrum
monitoring. In this paper, the performances of two different approaches for
controlling the available receiver resources are compared. The methods used for
resource management (ReMa) are linear frequency tuning as a heuristic approach
and a Q-learning algorithm from the field of reinforcement learning. To test
the methods to be investigated, a simplified scenario was designed with two
receiver channels monitoring ten non-overlapping frequency bands with
non-uniform signal activity. For this setting, it is shown that the Q-learning
algorithm used has a significantly higher detection rate than the heuristic
approach at the expense of a smaller exploration rate. In particular, the
Q-learning approach can be parameterized to allow for a suitable trade-off
between detection and exploration rate.