Automated Feature Selection for Inverse Reinforcement Learning
Daulet Baimukashev Gokhan Alcan Ville Kyrki
Inverse reinforcement learning (IRL) is an imitation learning approach to
learning reward functions from expert demonstrations. Its use avoids the
difficult and tedious procedure of manual reward specification while retaining
the generalization power of reinforcement learning. In IRL, the reward is
usually represented as a linear combination of features. In continuous state
spaces, the state variables alone are not sufficiently rich to be used as
features, but which features are good is not known in general. To address this
issue, we propose a method that employs polynomial basis functions to form a
candidate set of features, which are shown to allow the matching of statistical
moments of state distributions. Feature selection is then performed for the
candidates by leveraging the correlation between trajectory probabilities and
feature expectations. We demonstrate the approach's effectiveness by recovering
reward functions that capture expert policies across non-linear control tasks
of increasing complexity. Code, data, and videos are available at
https://sites.google.com/view/feature4irl.