Free-text Keystroke Authentication using Transformers: A Comparative
  Study of Architectures and Loss Functions
Saleh Momeni Bagher BabaAli
Keystroke biometrics is a promising approach for user identification and
verification, leveraging the unique patterns in individuals' typing behavior.
In this paper, we propose a Transformer-based network that employs
self-attention to extract informative features from keystroke sequences,
surpassing the performance of traditional Recurrent Neural Networks. We explore
two distinct architectures, namely bi-encoder and cross-encoder, and compare
their effectiveness in keystroke authentication. Furthermore, we investigate
different loss functions, including triplet, batch-all triplet, and WDCL loss,
along with various distance metrics such as Euclidean, Manhattan, and cosine
distances. These experiments allow us to optimize the training process and
enhance the performance of our model. To evaluate our proposed model, we employ
the Aalto desktop keystroke dataset. The results demonstrate that the
bi-encoder architecture with batch-all triplet loss and cosine distance
achieves the best performance, yielding an exceptional Equal Error Rate of
0.0186%. Furthermore, alternative algorithms for calculating similarity scores
are explored to enhance accuracy. Notably, the utilization of a one-class
Support Vector Machine reduces the Equal Error Rate to an impressive 0.0163%.
The outcomes of this study indicate that our model surpasses the previous
state-of-the-art in free-text keystroke authentication. These findings
contribute to advancing the field of keystroke authentication and offer
practical implications for secure user verification systems.