Socially Aware Synthetic Data Generation for Suicidal Ideation Detection
  Using Large Language Models
Hamideh Ghanadian Isar Nejadgholi Hussein Al Osman
Suicidal ideation detection is a vital research area that holds great
potential for improving mental health support systems. However, the sensitivity
surrounding suicide-related data poses challenges in accessing large-scale,
annotated datasets necessary for training effective machine learning models. To
address this limitation, we introduce an innovative strategy that leverages the
capabilities of generative AI models, such as ChatGPT, Flan-T5, and Llama, to
create synthetic data for suicidal ideation detection. Our data generation
approach is grounded in social factors extracted from psychology literature and
aims to ensure coverage of essential information related to suicidal ideation.
In our study, we benchmarked against state-of-the-art NLP classification
models, specifically, those centered around the BERT family structures. When
trained on the real-world dataset, UMD, these conventional models tend to yield
F1-scores ranging from 0.75 to 0.87. Our synthetic data-driven method, informed
by social factors, offers consistent F1-scores of 0.82 for both models,
suggesting that the richness of topics in synthetic data can bridge the
performance gap across different model complexities. Most impressively, when we
combined a mere 30% of the UMD dataset with our synthetic data, we witnessed a
substantial increase in performance, achieving an F1-score of 0.88 on the UMD
test set. Such results underscore the cost-effectiveness and potential of our
approach in confronting major challenges in the field, such as data scarcity
and the quest for diversity in data representation.