Co-Optimization of Environment and Policies for Decentralized
  Multi-Agent Navigation
Zhan Gao Guang Yang Amanda Prorok
This work views the multi-agent system and its surrounding environment as a
co-evolving system, where the behavior of one affects the other. The goal is to
take both agent actions and environment configurations as decision variables,
and optimize these two components in a coordinated manner to improve some
measure of interest. Towards this end, we consider the problem of decentralized
multi-agent navigation in cluttered environments. By introducing two
sub-objectives of multi-agent navigation and environment optimization, we
propose an $\textit{agent-environment co-optimization}$ problem and develop a
$\textit{coordinated algorithm}$ that alternates between these sub-objectives
to search for an optimal synthesis of agent actions and obstacle configurations
in the environment; ultimately, improving the navigation performance. Due to
the challenge of explicitly modeling the relation between agents, environment
and performance, we leverage policy gradient to formulate a model-free learning
mechanism within the coordinated framework. A formal convergence analysis shows
that our coordinated algorithm tracks the local minimum trajectory of an
associated time-varying non-convex optimization problem. Extensive numerical
results corroborate theoretical findings and show the benefits of
co-optimization over baselines. Interestingly, the results also indicate that
optimized environment configurations are able to offer structural guidance that
is key to de-conflicting agents in motion.