Reinforcement learning in large, structured action spaces: A simulation
  study of decision support for spinal cord injury rehabilitation
Nathan Phelps Stephanie Marrocco Stephanie Cornell Dalton L. Wolfe Daniel J. Lizotte
Reinforcement learning (RL) has helped improve decision-making in several
applications. However, applying traditional RL is challenging in some
applications, such as rehabilitation of people with a spinal cord injury (SCI).
Among other factors, using RL in this domain is difficult because there are
many possible treatments (i.e., large action space) and few patients (i.e.,
limited training data). Treatments for SCIs have natural groupings, so we
propose two approaches to grouping treatments so that an RL agent can learn
effectively from limited data. One relies on domain knowledge of SCI
rehabilitation and the other learns similarities among treatments using an
embedding technique. We then use Fitted Q Iteration to train an agent that
learns optimal treatments. Through a simulation study designed to reflect the
properties of SCI rehabilitation, we find that both methods can help improve
the treatment decisions of physiotherapists, but the approach based on domain
knowledge offers better performance. Our findings provide a "proof of concept"
that RL can be used to help improve the treatment of those with an SCI and
indicates that continued efforts to gather data and apply RL to this domain are
worthwhile.