Investigating the Robustness and Properties of Detection Transformers
  (DETR) Toward Difficult Images
Zhao Ning Zou Yuhang Zhang Robert Wijaya
Transformer-based object detectors (DETR) have shown significant performance
across machine vision tasks, ultimately in object detection. This detector is
based on a self-attention mechanism along with the transformer encoder-decoder
architecture to capture the global context in the image. The critical issue to
be addressed is how this model architecture can handle different image
nuisances, such as occlusion and adversarial perturbations. We studied this
issue by measuring the performance of DETR with different experiments and
benchmarking the network with convolutional neural network (CNN) based
detectors like YOLO and Faster-RCNN. We found that DETR performs well when it
comes to resistance to interference from information loss in occlusion images.
Despite that, we found that the adversarial stickers put on the image require
the network to produce a new unnecessary set of keys, queries, and values,
which in most cases, results in a misdirection of the network. DETR also
performed poorer than YOLOv5 in the image corruption benchmark. Furthermore, we
found that DETR depends heavily on the main query when making a prediction,
which leads to imbalanced contributions between queries since the main query
receives most of the gradient flow.