Multi Self-supervised Pre-fine-tuned Transformer Fusion for Better
  Intelligent Transportation Detection
Juwu Zheng Jiangtao Ren
Intelligent transportation system combines advanced information technology to
provide intelligent services such as monitoring, detection, and early warning
for modern transportation. Intelligent transportation detection is the
cornerstone of many intelligent traffic services by identifying task targets
through object detection methods. However existing detection methods in
intelligent transportation are limited by two aspects. First, there is a
difference between the model knowledge pre-trained on large-scale datasets and
the knowledge required for target task. Second, most detection models follow
the pattern of single-source learning, which limits the learning ability. To
address these problems, we propose a Multi Self-supervised Pre-fine-tuned
Transformer Fusion (MSPTF) network, consisting of two steps: unsupervised
pre-fine-tune domain knowledge learning and multi-model fusion target task
learning. In the first step, we introduced self-supervised learning methods
into transformer model pre-fine-tune which could reduce data costs and
alleviate the knowledge gap between pre-trained model and target task. In the
second step, we take feature information differences between different model
architectures and different pre-fine-tune tasks into account and propose
Multi-model Semantic Consistency Cross-attention Fusion (MSCCF) network to
combine different transformer model features by considering channel semantic
consistency and feature vector semantic consistency, which obtain more complete
and proper fusion features for detection task. We experimented the proposed
method on vehicle recognition dataset and road disease detection dataset and
achieved 1.1%, 5.5%, 4.2% improvement compared with baseline and 0.7%, 1.8%,
1.7% compared with sota, which proved the effectiveness of our method.