A Competition Winning Deep Reinforcement Learning Agent in microRTS
Scott Goodfriend
Scripted agents have predominantly won the five previous iterations of the
IEEE microRTS ($\mu$RTS) competitions hosted at CIG and CoG. Despite Deep
Reinforcement Learning (DRL) algorithms making significant strides in real-time
strategy (RTS) games, their adoption in this primarily academic competition has
been limited due to the considerable training resources required and the
complexity inherent in creating and debugging such agents. RAISocketAI is the
first DRL agent to win the IEEE microRTS competition. In a benchmark without
performance constraints, RAISocketAI regularly defeated the two prior
competition winners. This first competition-winning DRL submission can be a
benchmark for future microRTS competitions and a starting point for future DRL
research. Iteratively fine-tuning the base policy and transfer learning to
specific maps were critical to RAISocketAI's winning performance. These
strategies can be used to economically train future DRL agents. Further work in
Imitation Learning using Behavior Cloning and fine-tuning these models with DRL
has proven promising as an efficient way to bootstrap models with demonstrated,
competitive behaviors.