When is Off-Policy Evaluation Useful? A Data-Centric Perspective
Hao Sun Alex J. Chan Nabeel Seedat Alihan Hüyük Mihaela van der Schaar
Evaluating the value of a hypothetical target policy with only a logged
dataset is important but challenging. On the one hand, it brings opportunities
for safe policy improvement under high-stakes scenarios like clinical
guidelines. On the other hand, such opportunities raise a need for precise
off-policy evaluation (OPE). While previous work on OPE focused on improving
the algorithm in value estimation, in this work, we emphasize the importance of
the offline dataset, hence putting forward a data-centric framework for
evaluating OPE problems. We propose DataCOPE, a data-centric framework for
evaluating OPE, that answers the questions of whether and to what extent we can
evaluate a target policy given a dataset. DataCOPE (1) forecasts the overall
performance of OPE algorithms without access to the environment, which is
especially useful before real-world deployment where evaluating OPE is
impossible; (2) identifies the sub-group in the dataset where OPE can be
inaccurate; (3) permits evaluations of datasets or data-collection strategies
for OPE problems. Our empirical analysis of DataCOPE in the logged contextual
bandit settings using healthcare datasets confirms its ability to evaluate both
machine-learning and human expert policies like clinical guidelines.