Run-time Introspection of 2D Object Detection in Automated Driving
  Systems Using Learning Representations
Hakan Yekta Yatbaz Mehrdad Dianati Konstantinos Koufos Roger Woodman
Reliable detection of various objects and road users in the surrounding
environment is crucial for the safe operation of automated driving systems
(ADS). Despite recent progresses in developing highly accurate object detectors
based on Deep Neural Networks (DNNs), they still remain prone to detection
errors, which can lead to fatal consequences in safety-critical applications
such as ADS. An effective remedy to this problem is to equip the system with
run-time monitoring, named as introspection in the context of autonomous
systems. Motivated by this, we introduce a novel introspection solution, which
operates at the frame level for DNN-based 2D object detection and leverages
neural network activation patterns. The proposed approach pre-processes the
neural activation patterns of the object detector's backbone using several
different modes. To provide extensive comparative analysis and fair comparison,
we also adapt and implement several state-of-the-art (SOTA) introspection
mechanisms for error detection in 2D object detection, using one-stage and
two-stage object detectors evaluated on KITTI and BDD datasets. We compare the
performance of the proposed solution in terms of error detection, adaptability
to dataset shift, and, computational and memory resource requirements. Our
performance evaluation shows that the proposed introspection solution
outperforms SOTA methods, achieving an absolute reduction in the missed error
ratio of 9% to 17% in the BDD dataset.