Innate-Values-driven Reinforcement Learning for Cooperative Multi-Agent
  Systems
Qin Yang
Innate values describe agents' intrinsic motivations, which reflect their
inherent interests and preferences to pursue goals and drive them to develop
diverse skills satisfying their various needs. The essence of reinforcement
learning (RL) is learning from interaction based on reward-driven (such as
utilities) behaviors, much like natural agents. It is an excellent model to
describe the innate-values-driven (IV) behaviors of AI agents. Especially in
multi-agent systems (MAS), building the awareness of AI agents to balance the
group utilities and system costs and satisfy group members' needs in their
cooperation is a crucial problem for individuals learning to support their
community and integrate human society in the long term. This paper proposes a
hierarchical compound intrinsic value reinforcement learning model --
innate-values-driven reinforcement learning termed IVRL to describe the complex
behaviors of multi-agent interaction in their cooperation. We implement the
IVRL architecture in the StarCraft Multi-Agent Challenge (SMAC) environment and
compare the cooperative performance within three characteristics of innate
value agents (Coward, Neutral, and Reckless) through three benchmark
multi-agent RL algorithms: QMIX, IQL, and QTRAN. The results demonstrate that
by organizing individual various needs rationally, the group can achieve better
performance with lower costs effectively.