Towards Zero-Shot Interpretable Human Recognition: A 2D-3D Registration
  Framework
Henrique Jesus Hugo Proença
Large vision models based in deep learning architectures have been
consistently advancing the state-of-the-art in biometric recognition. However,
three weaknesses are commonly reported for such kind of approaches: 1) their
extreme demands in terms of learning data; 2) the difficulties in generalising
between different domains; and 3) the lack of interpretability/explainability,
with biometrics being of particular interest, as it is important to provide
evidence able to be used for forensics/legal purposes (e.g., in courts). To the
best of our knowledge, this paper describes the first recognition
framework/strategy that aims at addressing the three weaknesses simultaneously.
At first, it relies exclusively in synthetic samples for learning purposes.
Instead of requiring a large amount and variety of samples for each subject,
the idea is to exclusively enroll a 3D point cloud per identity. Then, using
generative strategies, we synthesize a very large (potentially infinite) number
of samples, containing all the desired covariates (poses, clothing, distances,
perspectives, lighting, occlusions,...). Upon the synthesizing method used, it
is possible to adapt precisely to different kind of domains, which accounts for
generalization purposes. Such data are then used to learn a model that performs
local registration between image pairs, establishing positive correspondences
between body parts that are the key, not only to recognition (according to
cardinality and distribution), but also to provide an interpretable description
of the response (e.g.: "both samples are from the same person, as they have
similar facial shape, hair color and legs thickness").