Knowledge Distillation for Anomaly Detection
Adrian Alan Pol Ekaterina Govorkova Sonja Gronroos Nadezda Chernyavskaya Philip Harris Maurizio Pierini Isobel Ojalvo Peter Elmer
Unsupervised deep learning techniques are widely used to identify anomalous
behaviour. The performance of such methods is a product of the amount of
training data and the model size. However, the size is often a limiting factor
for the deployment on resource-constrained devices. We present a novel
procedure based on knowledge distillation for compressing an unsupervised
anomaly detection model into a supervised deployable one and we suggest a set
of techniques to improve the detection sensitivity. Compressed models perform
comparably to their larger counterparts while significantly reducing the size
and memory footprint.