Composing Efficient, Robust Tests for Policy Selection
Dustin Morrill Thomas J. Walsh Daniel Hernandez Peter R. Wurman Peter Stone
Modern reinforcement learning systems produce many high-quality policies
throughout the learning process. However, to choose which policy to actually
deploy in the real world, they must be tested under an intractable number of
environmental conditions. We introduce RPOSST, an algorithm to select a small
set of test cases from a larger pool based on a relatively small number of
sample evaluations. RPOSST treats the test case selection problem as a
two-player game and optimizes a solution with provable $k$-of-$N$ robustness,
bounding the error relative to a test that used all the test cases in the pool.
Empirical results demonstrate that RPOSST finds a small set of test cases that
identify high quality policies in a toy one-shot game, poker datasets, and a
high-fidelity racing simulator.