TrackAgent: 6D Object Tracking via Reinforcement Learning
Konstantin Röhrl Dominik Bauer Timothy Patten Markus Vincze
Tracking an object's 6D pose, while either the object itself or the observing
camera is moving, is important for many robotics and augmented reality
applications. While exploiting temporal priors eases this problem,
object-specific knowledge is required to recover when tracking is lost. Under
the tight time constraints of the tracking task, RGB(D)-based methods are often
conceptionally complex or rely on heuristic motion models. In comparison, we
propose to simplify object tracking to a reinforced point cloud (depth only)
alignment task. This allows us to train a streamlined approach from scratch
with limited amounts of sparse 3D point clouds, compared to the large datasets
of diverse RGBD sequences required in previous works. We incorporate temporal
frame-to-frame registration with object-based recovery by frame-to-model
refinement using a reinforcement learning (RL) agent that jointly solves for
both objectives. We also show that the RL agent's uncertainty and a
rendering-based mask propagation are effective reinitialization triggers.