Discovering User Types: Mapping User Traits by Task-Specific Behaviors
  in Reinforcement Learning
L. L. Ankile B. S. Ham K. Mao E. Shin S. Swaroop F. Doshi-Velez W. Pan
When assisting human users in reinforcement learning (RL), we can represent
users as RL agents and study key parameters, called \emph{user traits}, to
inform intervention design. We study the relationship between user behaviors
(policy classes) and user traits. Given an environment, we introduce an
intuitive tool for studying the breakdown of "user types": broad sets of traits
that result in the same behavior. We show that seemingly different real-world
environments admit the same set of user types and formalize this observation as
an equivalence relation defined on environments. By transferring intervention
design between environments within the same equivalence class, we can help
rapidly personalize interventions.