Some Like It Small: Czech Semantic Embedding Models for Industry
  Applications
Jiří Bednář Jakub Náplava Petra Barančíková Ondřej Lisický
This article focuses on the development and evaluation of Small-sized Czech
sentence embedding models. Small models are important components for real-time
industry applications in resource-constrained environments. Given the limited
availability of labeled Czech data, alternative approaches, including
pre-training, knowledge distillation, and unsupervised contrastive fine-tuning,
are investigated. Comprehensive intrinsic and extrinsic analyses are conducted,
showcasing the competitive performance of our models compared to significantly
larger counterparts, with approximately 8 times smaller size and 5 times faster
speed than conventional Base-sized models. To promote cooperation and
reproducibility, both the models and the evaluation pipeline are made publicly
accessible. Ultimately, this article presents practical applications of the
developed sentence embedding models in Seznam.cz, the Czech search engine.
These models have effectively replaced previous counterparts, enhancing the
overall search experience for instance, in organic search, featured snippets,
and image search. This transition has yielded improved performance.