UniCat: Crafting a Stronger Fusion Baseline for Multimodal
  Re-Identification
Jennifer Crawford Haoli Yin Luke McDermott Daniel Cummings
Multimodal Re-Identification (ReID) is a popular retrieval task that aims to
re-identify objects across diverse data streams, prompting many researchers to
integrate multiple modalities into a unified representation. While such fusion
promises a holistic view, our investigations shed light on potential pitfalls.
We uncover that prevailing late-fusion techniques often produce suboptimal
latent representations when compared to methods that train modalities in
isolation. We argue that this effect is largely due to the inadvertent
relaxation of the training objectives on individual modalities when using
fusion, what others have termed modality laziness. We present a nuanced
point-of-view that this relaxation can lead to certain modalities failing to
fully harness available task-relevant information, and yet, offers a protective
veil to noisy modalities, preventing them from overfitting to task-irrelevant
data. Our findings also show that unimodal concatenation (UniCat) and other
late-fusion ensembling of unimodal backbones, when paired with best-known
training techniques, exceed the current state-of-the-art performance across
several multimodal ReID benchmarks. By unveiling the double-edged sword of
"modality laziness", we motivate future research in balancing local modality
strengths with global representations.