CML-MOTS: Collaborative Multi-task Learning for Multi-Object Tracking
  and Segmentation
Yiming Cui Cheng Han Dongfang Liu
The advancement of computer vision has pushed visual analysis tasks from
still images to the video domain. In recent years, video instance segmentation,
which aims to track and segment multiple objects in video frames, has drawn
much attention for its potential applications in various emerging areas such as
autonomous driving, intelligent transportation, and smart retail. In this
paper, we propose an effective framework for instance-level visual analysis on
video frames, which can simultaneously conduct object detection, instance
segmentation, and multi-object tracking. The core idea of our method is
collaborative multi-task learning which is achieved by a novel structure, named
associative connections among detection, segmentation, and tracking task heads
in an end-to-end learnable CNN. These additional connections allow information
propagation across multiple related tasks, so as to benefit these tasks
simultaneously. We evaluate the proposed method extensively on KITTI MOTS and
MOTS Challenge datasets and obtain quite encouraging results.