LLM-empowered Chatbots for Psychiatrist and Patient Simulation:
  Application and Evaluation
Siyuan Chen Mengyue Wu Kenny Q. Zhu Kunyao Lan Zhiling Zhang Lyuchun Cui
Empowering chatbots in the field of mental health is receiving increasing
amount of attention, while there still lacks exploration in developing and
evaluating chatbots in psychiatric outpatient scenarios. In this work, we focus
on exploring the potential of ChatGPT in powering chatbots for psychiatrist and
patient simulation. We collaborate with psychiatrists to identify objectives
and iteratively develop the dialogue system to closely align with real-world
scenarios. In the evaluation experiments, we recruit real psychiatrists and
patients to engage in diagnostic conversations with the chatbots, collecting
their ratings for assessment. Our findings demonstrate the feasibility of using
ChatGPT-powered chatbots in psychiatric scenarios and explore the impact of
prompt designs on chatbot behavior and user experience.