MixedTeacher : Knowledge Distillation for fast inference textural
  anomaly detection
Simon Thomine Hichem Snoussi Mahmoud Soua
For a very long time, unsupervised learning for anomaly detection has been at
the heart of image processing research and a stepping stone for high
performance industrial automation process. With the emergence of CNN, several
methods have been proposed such as Autoencoders, GAN, deep feature extraction,
etc. In this paper, we propose a new method based on the promising concept of
knowledge distillation which consists of training a network (the student) on
normal samples while considering the output of a larger pretrained network (the
teacher). The main contributions of this paper are twofold: First, a reduced
student architecture with optimal layer selection is proposed, then a new
Student-Teacher architecture with network bias reduction combining two teachers
is proposed in order to jointly enhance the performance of anomaly detection
and its localization accuracy. The proposed texture anomaly detector has an
outstanding capability to detect defects in any texture and a fast inference
time compared to the SOTA methods.