Towards a Framework for Deep Learning Certification in Safety-Critical
  Applications Using Inherently Safe Design and Run-Time Error Detection
Romeo Valentin
Although an ever-growing number of applications employ deep learning based
systems for prediction, decision-making, or state estimation, almost no
certification processes have been established that would allow such systems to
be deployed in safety-critical applications. In this work we consider
real-world problems arising in aviation and other safety-critical areas, and
investigate their requirements for a certified model. To this end, we
investigate methodologies from the machine learning research community aimed
towards verifying robustness and reliability of deep learning systems, and
evaluate these methodologies with regard to their applicability to real-world
problems. Then, we establish a new framework towards deep learning
certification based on (i) inherently safe design, and (ii) run-time error
detection. Using a concrete use case from aviation, we show how deep learning
models can recover disentangled variables through the use of weakly-supervised
representation learning. We argue that such a system design is inherently less
prone to common model failures, and can be verified to encode underlying
mechanisms governing the data. Then, we investigate four techniques related to
the run-time safety of a model, namely (i) uncertainty quantification, (ii)
out-of-distribution detection, (iii) feature collapse, and (iv) adversarial
attacks. We evaluate each for their applicability and formulate a set of
desiderata that a certified model should fulfill. Finally, we propose a novel
model structure that exhibits all desired properties discussed in this work,
and is able to make regression and uncertainty predictions, as well as detect
out-of-distribution inputs, while requiring no regression labels to train. We
conclude with a discussion of the current state and expected future progress of
deep learning certification, and its industrial and social implications.