The Fourth International Verification of Neural Networks Competition
  (VNN-COMP 2023): Summary and Results
Christopher Brix Stanley Bak Changliu Liu Taylor T. Johnson
This report summarizes the 4th International Verification of Neural Networks
Competition (VNN-COMP 2023), held as a part of the 6th Workshop on Formal
Methods for ML-Enabled Autonomous Systems (FoMLAS), that was collocated with
the 35th International Conference on Computer-Aided Verification (CAV).
VNN-COMP is held annually to facilitate the fair and objective comparison of
state-of-the-art neural network verification tools, encourage the
standardization of tool interfaces, and bring together the neural network
verification community. To this end, standardized formats for networks (ONNX)
and specification (VNN-LIB) were defined, tools were evaluated on equal-cost
hardware (using an automatic evaluation pipeline based on AWS instances), and
tool parameters were chosen by the participants before the final test sets were
made public. In the 2023 iteration, 7 teams participated on a diverse set of 10
scored and 4 unscored benchmarks. This report summarizes the rules, benchmarks,
participating tools, results, and lessons learned from this iteration of this
competition.