Language Models for Novelty Detection in System Call Traces
Quentin Fournier Daniel Aloise Leandro R. Costa
Due to the complexity of modern computer systems, novel and unexpected
behaviors frequently occur. Such deviations are either normal occurrences, such
as software updates and new user activities, or abnormalities, such as
misconfigurations, latency issues, intrusions, and software bugs. Regardless,
novel behaviors are of great interest to developers, and there is a genuine
need for efficient and effective methods to detect them. Nowadays, researchers
consider system calls to be the most fine-grained and accurate source of
information to investigate the behavior of computer systems. Accordingly, this
paper introduces a novelty detection methodology that relies on a probability
distribution over sequences of system calls, which can be seen as a language
model. Language models estimate the likelihood of sequences, and since
novelties deviate from previously observed behaviors by definition, they would
be unlikely under the model. Following the success of neural networks for
language models, three architectures are evaluated in this work: the widespread
LSTM, the state-of-the-art Transformer, and the lower-complexity Longformer.
However, large neural networks typically require an enormous amount of data to
be trained effectively, and to the best of our knowledge, no massive modern
datasets of kernel traces are publicly available. This paper addresses this
limitation by introducing a new open-source dataset of kernel traces comprising
over 2 million web requests with seven distinct behaviors. The proposed
methodology requires minimal expert hand-crafting and achieves an F-score and
AuROC greater than 95% on most novelties while being data- and task-agnostic.
The source code and trained models are publicly available on GitHub while the
datasets are available on Zenodo.