Multi-Task Consistency for Active Learning
Aral Hekimoglu Philipp Friedrich Walter Zimmer Michael Schmidt Alvaro Marcos-Ramiro Alois C. Knoll
Learning-based solutions for vision tasks require a large amount of labeled
training data to ensure their performance and reliability. In single-task
vision-based settings, inconsistency-based active learning has proven to be
effective in selecting informative samples for annotation. However, there is a
lack of research exploiting the inconsistency between multiple tasks in
multi-task networks. To address this gap, we propose a novel multi-task active
learning strategy for two coupled vision tasks: object detection and semantic
segmentation. Our approach leverages the inconsistency between them to identify
informative samples across both tasks. We propose three constraints that
specify how the tasks are coupled and introduce a method for determining the
pixels belonging to the object detected by a bounding box, to later quantify
the constraints as inconsistency scores. To evaluate the effectiveness of our
approach, we establish multiple baselines for multi-task active learning and
introduce a new metric, mean Detection Segmentation Quality (mDSQ), tailored
for the multi-task active learning comparison that addresses the performance of
both tasks. We conduct extensive experiments on the nuImages and A9 datasets,
demonstrating that our approach outperforms existing state-of-the-art methods
by up to 3.4% mDSQ on nuImages. Our approach achieves 95% of the fully-trained
performance using only 67% of the available data, corresponding to 20% fewer
labels compared to random selection and 5% fewer labels compared to
state-of-the-art selection strategy. Our code will be made publicly available
after the review process.