Transparent Object Tracking with Enhanced Fusion Module
Kalyan Garigapati Erik Blasch Jie Wei Haibin Ling
Accurate tracking of transparent objects, such as glasses, plays a critical
role in many robotic tasks such as robot-assisted living. Due to the adaptive
and often reflective texture of such objects, traditional tracking algorithms
that rely on general-purpose learned features suffer from reduced performance.
Recent research has proposed to instill transparency awareness into existing
general object trackers by fusing purpose-built features. However, with the
existing fusion techniques, the addition of new features causes a change in the
latent space making it impossible to incorporate transparency awareness on
trackers with fixed latent spaces. For example, many of the current days
transformer-based trackers are fully pre-trained and are sensitive to any
latent space perturbations. In this paper, we present a new feature fusion
technique that integrates transparency information into a fixed feature space,
enabling its use in a broader range of trackers. Our proposed fusion module,
composed of a transformer encoder and an MLP module, leverages key query-based
transformations to embed the transparency information into the tracking
pipeline. We also present a new two-step training strategy for our fusion
module to effectively merge transparency features. We propose a new tracker
architecture that uses our fusion techniques to achieve superior results for
transparent object tracking. Our proposed method achieves competitive results
with state-of-the-art trackers on TOTB, which is the largest transparent object
tracking benchmark recently released. Our results and the implementation of
code will be made publicly available at https://github.com/kalyan0510/TOTEM.