Adaptive Thresholding Heuristic for KPI Anomaly Detection
Ebenezer R. H. P. Isaac Akshat Sharma
A plethora of outlier detectors have been explored in the time series domain,
however, in a business sense, not all outliers are anomalies of interest.
Existing anomaly detection solutions are confined to certain outlier detectors
limiting their applicability to broader anomaly detection use cases. Network
KPIs (Key Performance Indicators) tend to exhibit stochastic behaviour
producing statistical outliers, most of which do not adversely affect business
operations. Thus, a heuristic is required to capture the business definition of
an anomaly for time series KPI. This article proposes an Adaptive Thresholding
Heuristic (ATH) to dynamically adjust the detection threshold based on the
local properties of the data distribution and adapt to changes in time series
patterns. The heuristic derives the threshold based on the expected periodicity
and the observed proportion of anomalies minimizing false positives and
addressing concept drift. ATH can be used in conjunction with any underlying
seasonality decomposition method and an outlier detector that yields an outlier
score. This method has been tested on EON1-Cell-U, a labeled KPI anomaly
dataset produced by Ericsson, to validate our hypothesis. Experimental results
show that ATH is computationally efficient making it scalable for near real
time anomaly detection and flexible with multiple forecasters and outlier
detectors.