Inverse Reinforcement Learning With Constraint Recovery
Nirjhar Das Arpan Chattopadhyay
In this work, we propose a novel inverse reinforcement learning (IRL)
algorithm for constrained Markov decision process (CMDP) problems. In standard
IRL problems, the inverse learner or agent seeks to recover the reward function
of the MDP, given a set of trajectory demonstrations for the optimal policy. In
this work, we seek to infer not only the reward functions of the CMDP, but also
the constraints. Using the principle of maximum entropy, we show that the IRL
with constraint recovery (IRL-CR) problem can be cast as a constrained
non-convex optimization problem. We reduce it to an alternating constrained
optimization problem whose sub-problems are convex. We use exponentiated
gradient descent algorithm to solve it. Finally, we demonstrate the efficacy of
our algorithm for the grid world environment.