NetTrack: Tracking Highly Dynamic Objects with a Net
Guangze Zheng Shijie Lin Haobo Zuo Changhong Fu Jia Pan
The complex dynamicity of open-world objects presents non-negligible
challenges for multi-object tracking (MOT), often manifested as severe
deformations, fast motion, and occlusions. Most methods that solely depend on
coarse-grained object cues, such as boxes and the overall appearance of the
object, are susceptible to degradation due to distorted internal relationships
of dynamic objects. To address this problem, this work proposes NetTrack, an
efficient, generic, and affordable tracking framework to introduce fine-grained
learning that is robust to dynamicity. Specifically, NetTrack constructs a
dynamicity-aware association with a fine-grained Net, leveraging point-level
visual cues. Correspondingly, a fine-grained sampler and matching method have
been incorporated. Furthermore, NetTrack learns object-text correspondence for
fine-grained localization. To evaluate MOT in extremely dynamic open-world
scenarios, a bird flock tracking (BFT) dataset is constructed, which exhibits
high dynamicity with diverse species and open-world scenarios. Comprehensive
evaluation on BFT validates the effectiveness of fine-grained learning on
object dynamicity, and thorough transfer experiments on challenging open-world
benchmarks, i.e., TAO, TAO-OW, AnimalTrack, and GMOT-40, validate the strong
generalization ability of NetTrack even without finetuning. Project page:
https://george-zhuang.github.io/nettrack/.