Checklist to Transparently Define Test Oracles for TP, FP, and FN
  Objects in Automated Driving
Michael Hoss
Popular test oracles for the perception subsystem of driving automation
systems identify true-positive (TP), false-positive (FP), and false-negative
(FN) objects. Oracle transparency is needed for comparing test results and for
safety cases. To date, there exists a common notion of TPs, FPs, and FNs in the
field, but apparently no published way to comprehensively define their oracles.
Therefore, this paper provides a checklist of functional aspects and
implementation details that affect the oracle behavior. Besides labeling
policies of the test set, we cover fields of view, occlusion handling,
safety-relevant areas, matching criteria, temporal and probabilistic issues,
and further aspects. Even though our checklist can hardly be formalized, it can
help practitioners maximize the transparency of their oracles, which, in turn,
makes statements on object perception more reliable and comparable.