Sinkhorn Transformations for Single-Query Postprocessing in Text-Video
  Retrieval
Konstantin Yakovlev Gregory Polyakov Ilseyar Alimova Alexander Podolskiy Andrey Bout Sergey Nikolenko Irina Piontkovskaya
A recent trend in multimodal retrieval is related to postprocessing test set
results via the dual-softmax loss (DSL). While this approach can bring
significant improvements, it usually presumes that an entire matrix of test
samples is available as DSL input. This work introduces a new postprocessing
approach based on Sinkhorn transformations that outperforms DSL. Further, we
propose a new postprocessing setting that does not require access to multiple
test queries. We show that our approach can significantly improve the results
of state of the art models such as CLIP4Clip, BLIP, X-CLIP, and DRL, thus
achieving a new state-of-the-art on several standard text-video retrieval
datasets both with access to the entire test set and in the single-query
setting.