1st Place Solution for CVPR2023 BURST Long Tail and Open World
  Challenges
Kaer Huang
Currently, Video Instance Segmentation (VIS) aims at segmenting and
categorizing objects in videos from a closed set of training categories that
contain only a few dozen of categories, lacking the ability to handle diverse
objects in real-world videos. As TAO and BURST datasets release, we have the
opportunity to research VIS in long-tailed and open-world scenarios.
Traditional VIS methods are evaluated on benchmarks limited to a small number
of common classes, But practical applications require trackers that go beyond
these common classes, detecting and tracking rare and even never-before-seen
objects. Inspired by the latest MOT paper for the long tail task (Tracking
Every Thing in the Wild, Siyuan Li et), for the BURST long tail challenge, we
train our model on a combination of LVISv0.5 and the COCO dataset using repeat
factor sampling. First, train the detector with segmentation and CEM on
LVISv0.5 + COCO dataset. And then, train the instance appearance similarity
head on the TAO dataset. at last, our method (LeTracker) gets 14.9 HOTAall in
the BURST test set, ranking 1st in the benchmark. for the open-world
challenges, we only use 64 classes (Intersection classes of BURST Train subset
and COCO dataset, without LVIS dataset) annotations data training, and testing
on BURST test set data and get 61.4 OWTAall, ranking 1st in the benchmark. Our
code will be released to facilitate future research.