VNLP: Turkish NLP Package
Meliksah Turker Mehmet Erdi Ari Aydin Han
In this work, we present VNLP: the first dedicated, complete, open-source,
well-documented, lightweight, production-ready, state-of-the-art Natural
Language Processing (NLP) package for the Turkish language. It contains a wide
variety of tools, ranging from the simplest tasks, such as sentence splitting
and text normalization, to the more advanced ones, such as text and token
classification models. Its token classification models are based on "Context
Model", a novel architecture that is both an encoder and an auto-regressive
model. NLP tasks solved by VNLP models include but are not limited to Sentiment
Analysis, Named Entity Recognition, Morphological Analysis \& Disambiguation
and Part-of-Speech Tagging. Moreover, it comes with pre-trained word embeddings
and corresponding SentencePiece Unigram tokenizers. VNLP has an open-source
GitHub repository, ReadtheDocs documentation, PyPi package for convenient
installation, Python and command-line API and a demo page to test all the
functionality. Consequently, our main contribution is a complete, compact,
easy-to-install and easy-to-use NLP package for Turkish.