Measuring the Impact of Scene Level Objects on Object Detection: Towards
  Quantitative Explanations of Detection Decisions
Lynn Vonder Haar Timothy Elvira Luke Newcomb Omar Ochoa
Although accuracy and other common metrics can provide a useful window into
the performance of an object detection model, they lack a deeper view of the
model's decision process. Regardless of the quality of the training data and
process, the features that an object detection model learns cannot be
guaranteed. A model may learn a relationship between certain background
context, i.e., scene level objects, and the presence of the labeled classes.
Furthermore, standard performance verification and metrics would not identify
this phenomenon. This paper presents a new black box explainability method for
additional verification of object detection models by finding the impact of
scene level objects on the identification of the objects within the image. By
comparing the accuracies of a model on test data with and without certain scene
level objects, the contributions of these objects to the model's performance
becomes clearer. The experiment presented here will assess the impact of
buildings and people in image context on the detection of emergency road
vehicles by a fine-tuned YOLOv8 model. A large increase in accuracy in the
presence of a scene level object will indicate the model's reliance on that
object to make its detections. The results of this research lead to providing a
quantitative explanation of the object detection model's decision process,
enabling a deeper understanding of the model's performance.