Monitoring Fidelity of Online Reinforcement Learning Algorithms in
  Clinical Trials
Anna L. Trella Kelly W. Zhang Inbal Nahum-Shani Vivek Shetty Iris Yan Finale Doshi-Velez Susan A. Murphy
Online reinforcement learning (RL) algorithms offer great potential for
personalizing treatment for participants in clinical trials. However, deploying
an online, autonomous algorithm in the high-stakes healthcare setting makes
quality control and data quality especially difficult to achieve. This paper
proposes algorithm fidelity as a critical requirement for deploying online RL
algorithms in clinical trials. It emphasizes the responsibility of the
algorithm to (1) safeguard participants and (2) preserve the scientific utility
of the data for post-trial analyses. We also present a framework for
pre-deployment planning and real-time monitoring to help algorithm developers
and clinical researchers ensure algorithm fidelity. To illustrate our
framework's practical application, we present real-world examples from the
Oralytics clinical trial. Since Spring 2023, this trial successfully deployed
an autonomous, online RL algorithm to personalize behavioral interventions for
participants at risk for dental disease.