Functional trustworthiness of AI systems by statistically valid testing
Bernhard Nessler Thomas Doms Sepp Hochreiter
The authors are concerned about the safety, health, and rights of the
European citizens due to inadequate measures and procedures required by the
current draft of the EU Artificial Intelligence (AI) Act for the conformity
assessment of AI systems. We observe that not only the current draft of the EU
AI Act, but also the accompanying standardization efforts in CEN/CENELEC, have
resorted to the position that real functional guarantees of AI systems
supposedly would be unrealistic and too complex anyways. Yet enacting a
conformity assessment procedure that creates the false illusion of trust in
insufficiently assessed AI systems is at best naive and at worst grossly
negligent. The EU AI Act thus misses the point of ensuring quality by
functional trustworthiness and correctly attributing responsibilities.
  The trustworthiness of an AI decision system lies first and foremost in the
correct statistical testing on randomly selected samples and in the precision
of the definition of the application domain, which enables drawing samples in
the first place. We will subsequently call this testable quality functional
trustworthiness. It includes a design, development, and deployment that enables
correct statistical testing of all relevant functions.
  We are firmly convinced and advocate that a reliable assessment of the
statistical functional properties of an AI system has to be the indispensable,
mandatory nucleus of the conformity assessment. In this paper, we describe the
three necessary elements to establish a reliable functional trustworthiness,
i.e., (1) the definition of the technical distribution of the application, (2)
the risk-based minimum performance requirements, and (3) the statistically
valid testing based on independent random samples.