How Far Are We from Believable AI Agents? A Framework for Evaluating the
  Believability of Human Behavior Simulation
Yang Xiao Yi Cheng Jinlan Fu Jiashuo Wang Wenjie Li Pengfei Liu
Human behavior simulation of AI agents necessitates the agents to possess a
quality of believability, which is crucial as it facilitates users in
establishing trust toward the agents and streamlines the fulfillment of the
agents' goal. While recent advancements in Large Language Model (LLM) based
agents have improved human behavior simulation, challenges inherent to LLMs
(e.g., long context modeling) can undermine their believability. Consequently,
evaluating AI agent believability becomes imperative. Unfortunately, prior
research often neglects the negative impacts of LLM deficiencies. To address
these gaps, we introduce two metrics for assessing LLM-based agent
believability: consistency, and robustness, together with a benchmark,
SimulateBench, with which, we evaluate the consistency and robustness of agents
implemented with popular LLMs. We find that agents (i) struggle to accurately
depict character information when presented with lengthy profile inputs; (ii)
exhibit vulnerability to profile perturbations; and (iii) are significantly
affected by certain key factors that impact their overall believability. Code
and SimulateBench are public at https://github.com/GAIR-NLP/GPTMan.