Loquacity and Visible Emotion: ChatGPT as a Policy Advisor
Claudia Biancotti Carolina Camassa
ChatGPT, a software seeking to simulate human conversational abilities, is
attracting increasing attention. It is sometimes portrayed as a groundbreaking
productivity aid, including for creative work. In this paper, we run an
experiment to assess its potential in complex writing tasks. We ask the
software to compose a policy brief for the Board of the Bank of Italy. We find
that ChatGPT can accelerate workflows by providing well-structured content
suggestions, and by producing extensive, linguistically correct text in a
matter of seconds. It does, however, require a significant amount of expert
supervision, which partially offsets productivity gains. If the app is used
naively, output can be incorrect, superficial, or irrelevant. Superficiality is
an especially problematic limitation in the context of policy advice intended
for high-level audiences.