Towards Practicable Sequential Shift Detectors
Oliver Cobb Arnaud Van Looveren
There is a growing awareness of the harmful effects of distribution shift on
the performance of deployed machine learning models. Consequently, there is a
growing interest in detecting these shifts before associated costs have time to
accumulate. However, desiderata of crucial importance to the practicable
deployment of sequential shift detectors are typically overlooked by existing
works, precluding their widespread adoption. We identify three such desiderata,
highlight existing works relevant to their satisfaction, and recommend
impactful directions for future research.