Convergence of a model-free entropy-regularized inverse reinforcement
  learning algorithm
Titouan Renard Andreas Schlaginhaufen Tingting Ni Maryam Kamgarpour
Given a dataset of expert demonstrations, inverse reinforcement learning
(IRL) aims to recover a reward for which the expert is optimal. This work
proposes a model-free algorithm to solve entropy-regularized IRL problem. In
particular, we employ a stochastic gradient descent update for the reward and a
stochastic soft policy iteration update for the policy. Assuming access to a
generative model, we prove that our algorithm is guaranteed to recover a reward
for which the expert is $\varepsilon$-optimal using
$\mathcal{O}(1/\varepsilon^{2})$ samples of the Markov decision process (MDP).
Furthermore, with $\mathcal{O}(1/\varepsilon^{4})$ samples we prove that the
optimal policy corresponding to the recovered reward is $\varepsilon$-close to
the expert policy in total variation distance.