How Well Can LLMs Negotiate? NegotiationArena Platform and Analysis
Federico Bianchi Patrick John Chia Mert Yuksekgonul Jacopo Tagliabue Dan Jurafsky James Zou
Negotiation is the basis of social interactions; humans negotiate everything
from the price of cars to how to share common resources. With rapidly growing
interest in using large language models (LLMs) to act as agents on behalf of
human users, such LLM agents would also need to be able to negotiate. In this
paper, we study how well LLMs can negotiate with each other. We develop
NegotiationArena: a flexible framework for evaluating and probing the
negotiation abilities of LLM agents. We implemented three types of scenarios in
NegotiationArena to assess LLM's behaviors in allocating shared resources
(ultimatum games), aggregate resources (trading games) and buy/sell goods
(price negotiations). Each scenario allows for multiple turns of flexible
dialogues between LLM agents to allow for more complex negotiations.
Interestingly, LLM agents can significantly boost their negotiation outcomes by
employing certain behavioral tactics. For example, by pretending to be desolate
and desperate, LLMs can improve their payoffs by 20\% when negotiating against
the standard GPT-4. We also quantify irrational negotiation behaviors exhibited
by the LLM agents, many of which also appear in humans. Together,
\NegotiationArena offers a new environment to investigate LLM interactions,
enabling new insights into LLM's theory of mind, irrationality, and reasoning
abilities.