Zone Evaluation: Revealing Spatial Bias in Object Detection
Zhaohui Zheng Yuming Chen Qibin Hou Xiang Li Ping Wang Ming-Ming Cheng
A fundamental limitation of object detectors is that they suffer from
"spatial bias", and in particular perform less satisfactorily when detecting
objects near image borders. For a long time, there has been a lack of effective
ways to measure and identify spatial bias, and little is known about where it
comes from and what degree it is. To this end, we present a new zone evaluation
protocol, extending from the traditional evaluation to a more generalized one,
which measures the detection performance over zones, yielding a series of Zone
Precisions (ZPs). For the first time, we provide numerical results, showing
that the object detectors perform quite unevenly across the zones.
Surprisingly, the detector's performance in the 96\% border zone of the image
does not reach the AP value (Average Precision, commonly regarded as the
average detection performance in the entire image zone). To better understand
spatial bias, a series of heuristic experiments are conducted. Our
investigation excludes two intuitive conjectures about spatial bias that the
object scale and the absolute positions of objects barely influence the spatial
bias. We find that the key lies in the human-imperceptible divergence in data
patterns between objects in different zones, thus eventually forming a visible
performance gap between the zones. With these findings, we finally discuss a
future direction for object detection, namely, spatial disequilibrium problem,
aiming at pursuing a balanced detection ability over the entire image zone. By
broadly evaluating 10 popular object detectors and 5 detection datasets, we
shed light on the spatial bias of object detectors. We hope this work could
raise a focus on detection robustness. The source codes, evaluation protocols,
and tutorials are publicly available at
\url{https://github.com/Zzh-tju/ZoneEval}.