Deep Reinforcement Learning for Personalized Diagnostic Decision
  Pathways Using Electronic Health Records: A Comparative Study on Anemia and
  Systemic Lupus Erythematosus
Lillian Muyama Antoine Neuraz Adrien Coulet
Background: Clinical diagnosis is typically reached by following a series of
steps recommended by guidelines authored by colleges of experts. Accordingly,
guidelines play a crucial role in rationalizing clinical decisions but suffer
from limitations as they are built to cover the majority of the population and
fail at covering patients with uncommon conditions. Moreover, their updates are
long and expensive, making them unsuitable for emerging diseases and practices.
  Methods: Inspired by guidelines, we formulate the task of diagnosis as a
sequential decision-making problem and study the use of Deep Reinforcement
Learning (DRL) algorithms to learn the optimal sequence of actions to perform
in order to obtain a correct diagnosis from Electronic Health Records (EHRs).
We apply DRL on synthetic, but realistic EHRs and develop two clinical use
cases: Anemia diagnosis, where the decision pathways follow the schema of a
decision tree; and Systemic Lupus Erythematosus (SLE) diagnosis, which follows
a weighted criteria score. We particularly evaluate the robustness of our
approaches to noisy and missing data since these frequently occur in EHRs.
  Results: In both use cases, and in the presence of imperfect data, our best
DRL algorithms exhibit competitive performance when compared to the traditional
classifiers, with the added advantage that they enable the progressive
generation of a pathway to the suggested diagnosis which can both guide and
explain the decision-making process.
  Conclusion: DRL offers the opportunity to learn personalized decision
pathways to diagnosis. We illustrate with our two use cases their advantages:
they generate step-by-step pathways that are self-explanatory; and their
correctness is competitive when compared to state-of-the-art approaches.