Reliability Analysis of Psychological Concept Extraction and
  Classification in User-penned Text
Muskan Garg MSVPJ Sathvik Amrit Chadha Shaina Raza Sunghwan Sohn
The social NLP research community witness a recent surge in the computational
advancements of mental health analysis to build responsible AI models for a
complex interplay between language use and self-perception. Such responsible AI
models aid in quantifying the psychological concepts from user-penned texts on
social media. On thinking beyond the low-level (classification) task, we
advance the existing binary classification dataset, towards a higher-level task
of reliability analysis through the lens of explanations, posing it as one of
the safety measures. We annotate the LoST dataset to capture nuanced textual
cues that suggest the presence of low self-esteem in the posts of Reddit users.
We further state that the NLP models developed for determining the presence of
low self-esteem, focus more on three types of textual cues: (i) Trigger: words
that triggers mental disturbance, (ii) LoST indicators: text indicators
emphasizing low self-esteem, and (iii) Consequences: words describing the
consequences of mental disturbance. We implement existing classifiers to
examine the attention mechanism in pre-trained language models (PLMs) for a
domain-specific psychology-grounded task. Our findings suggest the need of
shifting the focus of PLMs from Trigger and Consequences to a more
comprehensive explanation, emphasizing LoST indicators while determining low
self-esteem in Reddit posts.