Few-Shot Scenario Testing for Autonomous Vehicles Based on Neighborhood
  Coverage and Similarity
Shu Li Jingxuan Yang Honglin He Yi Zhang Jianming Hu Shuo Feng
Testing and evaluating the safety performance of autonomous vehicles (AVs) is
essential before the large-scale deployment. Practically, the acceptable cost
of testing specific AV model can be restricted within an extremely small limit
because of testing cost or time. With existing testing methods, the limitations
imposed by strictly restricted testing numbers often result in significant
uncertainties or challenges in quantifying testing results. In this paper, we
formulate this problem for the first time the "few-shot testing" (FST) problem
and propose a systematic FST framework to address this challenge. To alleviate
the considerable uncertainty inherent in a small testing scenario set and
optimize scenario utilization, we frame the FST problem as an optimization
problem and search for a small scenario set based on neighborhood coverage and
similarity. By leveraging the prior information on surrogate models (SMs), we
dynamically adjust the testing scenario set and the contribution of each
scenario to the testing result under the guidance of better generalization
ability on AVs. With certain hypotheses on SMs, a theoretical upper bound of
testing error is established to verify the sufficiency of testing accuracy
within given limited number of tests. The experiments of the cut-in scenario
using FST method demonstrate a notable reduction in testing error and variance
compared to conventional testing methods, especially for situations with a
strict limitation on the number of scenarios.