An Explainable Model-Agnostic Algorithm for CNN-based Biometrics
  Verification
Fernando Alonso-Fernandez Kevin Hernandez-Diaz Jose M. Buades Prayag Tiwari Josef Bigun
This paper describes an adaptation of the Local Interpretable Model-Agnostic
Explanations (LIME) AI method to operate under a biometric verification
setting. LIME was initially proposed for networks with the same output classes
used for training, and it employs the softmax probability to determine which
regions of the image contribute the most to classification. However, in a
verification setting, the classes to be recognized have not been seen during
training. In addition, instead of using the softmax output, face descriptors
are usually obtained from a layer before the classification layer. The model is
adapted to achieve explainability via cosine similarity between feature vectors
of perturbated versions of the input image. The method is showcased for face
biometrics with two CNN models based on MobileNetv2 and ResNet50.