TurkishBERTweet: Fast and Reliable Large Language Model for Social Media
  Analysis
Ali Najafi Onur Varol
Turkish is one of the most popular languages in the world. Wide us of this
language on social media platforms such as Twitter, Instagram, or Tiktok and
strategic position of the country in the world politics makes it appealing for
the social network researchers and industry. To address this need, we introduce
TurkishBERTweet, the first large scale pre-trained language model for Turkish
social media built using almost 900 million tweets. The model shares the same
architecture as base BERT model with smaller input length, making
TurkishBERTweet lighter than BERTurk and can have significantly lower inference
time. We trained our model using the same approach for RoBERTa model and
evaluated on two text classification tasks: Sentiment Classification and Hate
Speech Detection. We demonstrate that TurkishBERTweet outperforms the other
available alternatives on generalizability and its lower inference time gives
significant advantage to process large-scale datasets. We also compared our
models with the commercial OpenAI solutions in terms of cost and performance to
demonstrate TurkishBERTweet is scalable and cost-effective solution. As part of
our research, we released TurkishBERTweet and fine-tuned LoRA adapters for the
mentioned tasks under the MIT License to facilitate future research and
applications on Turkish social media. Our TurkishBERTweet model is available
at: https://github.com/ViralLab/TurkishBERTweet