Signal Temporal Logic-Guided Apprenticeship Learning
Aniruddh G. Puranic Jyotirmoy V. Deshmukh Stefanos Nikolaidis
Apprenticeship learning crucially depends on effectively learning rewards,
and hence control policies from user demonstrations. Of particular difficulty
is the setting where the desired task consists of a number of sub-goals with
temporal dependencies. The quality of inferred rewards and hence policies are
typically limited by the quality of demonstrations, and poor inference of these
can lead to undesirable outcomes. In this letter, we show how temporal logic
specifications that describe high level task objectives, are encoded in a graph
to define a temporal-based metric that reasons about behaviors of demonstrators
and the learner agent to improve the quality of inferred rewards and policies.
Through experiments on a diverse set of robot manipulator simulations, we show
how our framework overcomes the drawbacks of prior literature by drastically
improving the number of demonstrations required to learn a control policy.