Assessing the Impact of Context Inference Error and Partial
  Observability on RL Methods for Just-In-Time Adaptive Interventions
Karine Karine Predrag Klasnja Susan A. Murphy Benjamin M. Marlin
Just-in-Time Adaptive Interventions (JITAIs) are a class of personalized
health interventions developed within the behavioral science community. JITAIs
aim to provide the right type and amount of support by iteratively selecting a
sequence of intervention options from a pre-defined set of components in
response to each individual's time varying state. In this work, we explore the
application of reinforcement learning methods to the problem of learning
intervention option selection policies. We study the effect of context
inference error and partial observability on the ability to learn effective
policies. Our results show that the propagation of uncertainty from context
inferences is critical to improving intervention efficacy as context
uncertainty increases, while policy gradient algorithms can provide remarkable
robustness to partially observed behavioral state information.