Detection of depression on social networks using transformers and
  ensembles
Ilija Tavchioski Marko Robnik-Šikonja Senja Pollak
As the impact of technology on our lives is increasing, we witness increased
use of social media that became an essential tool not only for communication
but also for sharing information with community about our thoughts and
feelings. This can be observed also for people with mental health disorders
such as depression where they use social media for expressing their thoughts
and asking for help. This opens a possibility to automatically process social
media posts and detect signs of depression. We build several large pre-trained
language model based classifiers for depression detection from social media
posts. Besides fine-tuning BERT, RoBERTA, BERTweet, and mentalBERT were also
construct two types of ensembles. We analyze the performance of our models on
two data sets of posts from social platforms Reddit and Twitter, and
investigate also the performance of transfer learning across the two data sets.
The results show that transformer ensembles improve over the single
transformer-based classifiers.