Improving Hand Recognition in Uncontrolled and Uncooperative
  Environments using Multiple Spatial Transformers and Loss Functions
Wojciech Michal Matkowski Xiaojie Li Adams Wai Kin Kong
The prevalence of smartphone and consumer camera has led to more evidence in
the form of digital images, which are mostly taken in uncontrolled and
uncooperative environments. In these images, criminals likely hide or cover
their faces while their hands are observable in some cases, creating a
challenging use case for forensic investigation. Many existing hand-based
recognition methods perform well for hand images collected in controlled
environments with user cooperation. However, their performance deteriorates
significantly in uncontrolled and uncooperative environments. A recent work has
exposed the potential of hand recognition in these environments. However, only
the palmar regions were considered, and the recognition performance is still
far from satisfactory. To improve the recognition accuracy, an algorithm
integrating a multi-spatial transformer network (MSTN) and multiple loss
functions is proposed to fully utilize information in full hand images. MSTN is
firstly employed to localize the palms and fingers and estimate the alignment
parameters. Then, the aligned images are further fed into pretrained
convolutional neural networks, where features are extracted. Finally, a
training scheme with multiple loss functions is used to train the network
end-to-end. To demonstrate the effectiveness of the proposed algorithm, the
trained model is evaluated on NTU-PI-v1 database and six benchmark databases
from different domains. Experimental results show that the proposed algorithm
performs significantly better than the existing methods in these uncontrolled
and uncooperative environments and has good generalization capabilities to
samples from different domains.