SUDO: a framework for evaluating clinical artificial intelligence
  systems without ground-truth annotations
Dani Kiyasseh Aaron Cohen Chengsheng Jiang Nicholas Altieri
A clinical artificial intelligence (AI) system is often validated on a
held-out set of data which it has not been exposed to before (e.g., data from a
different hospital with a distinct electronic health record system). This
evaluation process is meant to mimic the deployment of an AI system on data in
the wild; those which are currently unseen by the system yet are expected to be
encountered in a clinical setting. However, when data in the wild differ from
the held-out set of data, a phenomenon referred to as distribution shift, and
lack ground-truth annotations, it becomes unclear the extent to which AI-based
findings can be trusted on data in the wild. Here, we introduce SUDO, a
framework for evaluating AI systems without ground-truth annotations. SUDO
assigns temporary labels to data points in the wild and directly uses them to
train distinct models, with the highest performing model indicative of the most
likely label. Through experiments with AI systems developed for dermatology
images, histopathology patches, and clinical reports, we show that SUDO can be
a reliable proxy for model performance and thus identify unreliable
predictions. We also demonstrate that SUDO informs the selection of models and
allows for the previously out-of-reach assessment of algorithmic bias for data
in the wild without ground-truth annotations. The ability to triage unreliable
predictions for further inspection and assess the algorithmic bias of AI
systems can improve the integrity of research findings and contribute to the
deployment of ethical AI systems in medicine.