Limits of Machine Learning for Automatic Vulnerability Detection
Niklas Risse Marcel Böhme
Recent results of machine learning for automatic vulnerability detection have
been very promising indeed: Given only the source code of a function $f$,
models trained by machine learning techniques can decide if $f$ contains a
security flaw with up to 70% accuracy.
  But how do we know that these results are general and not specific to the
datasets? To study this question, researchers proposed to amplify the testing
set by injecting semantic preserving changes and found that the model's
accuracy significantly drops. In other words, the model uses some unrelated
features during classification. In order to increase the robustness of the
model, researchers proposed to train on amplified training data, and indeed
model accuracy increased to previous levels.
  In this paper, we replicate and continue this investigation, and provide an
actionable model benchmarking methodology to help researchers better evaluate
advances in machine learning for vulnerability detection. Specifically, we
propose (i) a cross validation algorithm, where a semantic preserving
transformation is applied during the amplification of either the training set
or the testing set, and (ii) the amplification of the testing set with code
snippets where the vulnerabilities are fixed. Using 11 transformations, 3 ML
techniques, and 2 datasets, we find that the improved robustness only applies
to the specific transformations used during training data amplification. In
other words, the robustified models still rely on unrelated features for
predicting the vulnerabilities in the testing data. Additionally, we find that
the trained models are unable to generalize to the modified setting which
requires to distinguish vulnerable functions from their patches.