Drawing the Same Bounding Box Twice? Coping Noisy Annotations in Object
  Detection with Repeated Labels
David Tschirschwitz Christian Benz Morris Florek Henrik Norderhus Benno Stein Volker Rodehorst
The reliability of supervised machine learning systems depends on the
accuracy and availability of ground truth labels. However, the process of human
annotation, being prone to error, introduces the potential for noisy labels,
which can impede the practicality of these systems. While training with noisy
labels is a significant consideration, the reliability of test data is also
crucial to ascertain the dependability of the results. A common approach to
addressing this issue is repeated labeling, where multiple annotators label the
same example, and their labels are combined to provide a better estimate of the
true label. In this paper, we propose a novel localization algorithm that
adapts well-established ground truth estimation methods for object detection
and instance segmentation tasks. The key innovation of our method lies in its
ability to transform combined localization and classification tasks into
classification-only problems, thus enabling the application of techniques such
as Expectation-Maximization (EM) or Majority Voting (MJV). Although our main
focus is the aggregation of unique ground truth for test data, our algorithm
also shows superior performance during training on the TexBiG dataset,
surpassing both noisy label training and label aggregation using Weighted Boxes
Fusion (WBF). Our experiments indicate that the benefits of repeated labels
emerge under specific dataset and annotation configurations. The key factors
appear to be (1) dataset complexity, the (2) annotator consistency, and (3) the
given annotation budget constraints.