LLM Agents for Psychology: A Study on Gamified Assessments
Qisen Yang Zekun Wang Honghui Chen Shenzhi Wang Yifan Pu Xin Gao Wenhao Huang Shiji Song Gao Huang
Psychological measurement is essential for mental health, self-understanding,
and personal development. Traditional methods, such as self-report scales and
psychologist interviews, often face challenges with engagement and
accessibility. While game-based and LLM-based tools have been explored to
improve user interest and automate assessment, they struggle to balance
engagement with generalizability. In this work, we propose PsychoGAT
(Psychological Game AgenTs) to achieve a generic gamification of psychological
assessment. The main insight is that powerful LLMs can function both as adept
psychologists and innovative game designers. By incorporating LLM agents into
designated roles and carefully managing their interactions, PsychoGAT can
transform any standardized scales into personalized and engaging interactive
fiction games. To validate the proposed method, we conduct psychometric
evaluations to assess its effectiveness and employ human evaluators to examine
the generated content across various psychological constructs, including
depression, cognitive distortions, and personality traits. Results demonstrate
that PsychoGAT serves as an effective assessment tool, achieving statistically
significant excellence in psychometric metrics such as reliability, convergent
validity, and discriminant validity. Moreover, human evaluations confirm
PsychoGAT's enhancements in content coherence, interactivity, interest,
immersion, and satisfaction.