DARTH: Holistic Test-time Adaptation for Multiple Object Tracking
Mattia Segu Bernt Schiele Fisher Yu
Multiple object tracking (MOT) is a fundamental component of perception
systems for autonomous driving, and its robustness to unseen conditions is a
requirement to avoid life-critical failures. Despite the urge of safety in
driving systems, no solution to the MOT adaptation problem to domain shift in
test-time conditions has ever been proposed. However, the nature of a MOT
system is manifold - requiring object detection and instance association - and
adapting all its components is non-trivial. In this paper, we analyze the
effect of domain shift on appearance-based trackers, and introduce DARTH, a
holistic test-time adaptation framework for MOT. We propose a detection
consistency formulation to adapt object detection in a self-supervised fashion,
while adapting the instance appearance representations via our novel patch
contrastive loss. We evaluate our method on a variety of domain shifts -
including sim-to-real, outdoor-to-indoor, indoor-to-outdoor - and substantially
improve the source model performance on all metrics. Code:
https://github.com/mattiasegu/darth.