Semi-supervised learning via DQN for log anomaly detection
Yingying He Xiaobing Pei Lihong Shen
Log anomaly detection plays a critical role in ensuring the security and
maintenance of modern software systems. At present, the primary approach for
detecting anomalies in log data is through supervised anomaly detection.
Nonetheless, existing supervised methods heavily rely on labeled data, which
can be frequently limited in real-world scenarios. In this paper, we propose a
semi-supervised log anomaly detection method that combines the DQN algorithm
from deep reinforcement learning, which is called DQNLog. DQNLog leverages a
small amount of labeled data and a large-scale unlabeled dataset, effectively
addressing the challenges of imbalanced data and limited labeling. This
approach not only learns known anomalies by interacting with an environment
biased towards anomalies but also discovers unknown anomalies by actively
exploring the unlabeled dataset. Additionally, DQNLog incorporates a
cross-entropy loss term to prevent model overestimation during Deep
Reinforcement Learning (DRL). Our evaluation on three widely-used datasets
demonstrates that DQNLog significantly improves recall rate and F1-score while
maintaining precision, validating its practicality.