Towards efficient deep autoencoders for multivariate time series anomaly
  detection
Marcin Pietroń Dominik Żurek Kamil Faber Roberto Corizzo
Multivariate time series anomaly detection is a crucial problem in many
industrial and research applications. Timely detection of anomalies allows, for
instance, to prevent defects in manufacturing processes and failures in
cyberphysical systems. Deep learning methods are preferred among others for
their accuracy and robustness for the analysis of complex multivariate data.
However, a key aspect is being able to extract predictions in a timely manner,
to accommodate real-time requirements in different applications. In the case of
deep learning models, model reduction is extremely important to achieve optimal
results in real-time systems with limited time and memory constraints. In this
paper, we address this issue by proposing a novel compression method for deep
autoencoders that involves three key factors. First, pruning reduces the number
of weights, while preventing catastrophic drops in accuracy by means of a fast
search process that identifies high sparsity levels. Second, linear and
non-linear quantization reduces model complexity by reducing the number of bits
for every single weight. The combined contribution of these three aspects allow
the model size to be reduced, by removing a subset of the weights (pruning),
and decreasing their bit-width (quantization). As a result, the compressed
model is faster and easier to adopt in highly constrained hardware
environments. Experiments performed on popular multivariate anomaly detection
benchmarks, show that our method is capable of achieving significant model
compression ratio (between 80% and 95%) without a significant reduction in the
anomaly detection performance.