A Review of Real-Time Strategy Game AI
---
paper_title: Call for AI Research in RTS Games

paper_content:

This position paper discusses AI challenges in the area of real‐time strategy games and presents a research agenda aimed at improving AI performance in these popular multi‐ player computer games.
---
paper_title: A Gamut of Games

paper_content:

In 1950, Claude Shannon published his seminal work on how to program a computer to play chess. Since then, developing game-playing programs that can compete with (and even exceed) the abilities of the human world champions has been a long-sought-after goal of the AI research community. In Shannon's time, it would have seemed unlikely that only a scant 50 years would be needed to develop programs that play world-class backgammon, checkers, chess, Othello, and Scrabble. These remarkable achievements are the result of a better understanding of the problems being solved, major algorithmic insights, and tremendous advances in hardware technology. Computer games research is one of the important success stories of AI. This article reviews the past successes, current projects, and future research directions for AI using computer games as a research test bed.
---
paper_title: Human-level AI's Killer Application: Interactive Computer Games

paper_content:

Although one of the fundamental goals of AI is to understand and develop intelligent systems that have all the capabilities of humans, there is little active research directly pursuing this goal. We propose that AI for interactive computer games is an emerging application area in which this goal of human-level AI can successfully be pursued. Interactive computer games have increasingly complex and realistic worlds and increasingly complex and intelligent computer-controlled characters. In this article, we further motivate our proposal of using interactive computer games for AI research, review previous research on AI and games, and present the different game genres and the roles that human-level AI could play within these genres. We then describe the research issues and AI techniques that are relevant to each of these roles. Our conclusion is that interactive computer games provide a rich environment for incremental research on human-level AI.
---
paper_title: Call for AI Research in RTS Games

paper_content:

This position paper discusses AI challenges in the area of real‐time strategy games and presents a research agenda aimed at improving AI performance in these popular multi‐ player computer games.
---
paper_title: A Gamut of Games

paper_content:

In 1950, Claude Shannon published his seminal work on how to program a computer to play chess. Since then, developing game-playing programs that can compete with (and even exceed) the abilities of the human world champions has been a long-sought-after goal of the AI research community. In Shannon's time, it would have seemed unlikely that only a scant 50 years would be needed to develop programs that play world-class backgammon, checkers, chess, Othello, and Scrabble. These remarkable achievements are the result of a better understanding of the problems being solved, major algorithmic insights, and tremendous advances in hardware technology. Computer games research is one of the important success stories of AI. This article reviews the past successes, current projects, and future research directions for AI using computer games as a research test bed.
---
paper_title: Case Acquisition Strategies for Case-Based Reasoning in Real-Time Strategy Games

paper_content:

Real-time Strategy (RTS) games are complex domains which are a significant challenge to both human and artificial intelligence (AI). For that reason, and although many AI approaches have been proposed for the RTS game AI problem, the AI of all commercial RTS games is scripted and offers a very static behavior subject to exploits. In this paper, we will focus on a case-based reasoning (CBR) approach to this problem, and concentrate on the process of case-acquisition. Specifically, we will describe 7 different techniques to automatically acquire plans by observing human demonstrations and compare their performance when using them in the Darmok 2 system in the context of an RTS game.
---
paper_title: A Bayesian model for RTS units control applied to StarCraft

paper_content:

In real-time strategy games (RTS), the player must reason about high-level strategy and planning while having effective tactics and even individual units micro-management. Enabling an artificial agent to deal with such a task entails breaking down the complexity of this environment. For that, we propose to control units locally in the Bayesian sensory motor robot fashion, with higher level orders integrated as perceptions. As complete inference encompassing global strategy down to individual unit needs is intractable, we embrace incompleteness through a hierarchical model able to deal with uncertainty. We developed and applied our approach on a StarCraft1 AI.
---
paper_title: Real-Time Strategy Game Competitions

paper_content:

In recent years, real-time strategy (RTS) games have gained attention in the AI research community for their multitude of challenging and relevant real-time decision problems that have to be solved in order to win against human experts or to effectively collaborate with other players in team-games. In this article we motivate research in this area, give an overview of past RTS game AI competitions, and discuss future directions.
---
paper_title: A Bayesian Model for Plan Recognition in RTS Games applied to StarCraft

paper_content:

The task of keyhole (unobtrusive) plan recognition is central to adaptive game AI. "Tech trees" or "build trees" are the core of real-time strategy (RTS) game strategic (long term) planning. This paper presents a generic and simple Bayesian model for RTS build tree prediction from noisy observations, which parameters are learned from replays (game logs). This unsupervised machine learning approach involves minimal work for the game developers as it leverage players' data (common in RTS). We applied it to StarCraft1 and showed that it yields high quality and robust predictions, that can feed an adaptive AI.
---
paper_title: Build order optimization in StarCraft

paper_content:

In recent years, real-time strategy (RTS) games have gained interest in the AI research community for their multitude of challenging subproblems — such as collaborative pathfinding, effective resource allocation and unit targeting, to name a few. In this paper we consider the build order problem in RTS games in which we need to find concurrent action sequences that, constrained by unit dependencies and resource availability, create a certain number of units and structures in the shortest possible time span. We present abstractions and heuristics that speed up the search for approximative solutions considerably in the game of StarCraft, and show the efficacy of our method by comparing its real-time performance with that of professional StarCraft players.
---
paper_title: Building Human-Level AI for Real-Time Strategy Games

paper_content:

Video games are complex simulation environments with many real-world properties that need to be addressed in order to build robust intelligence. In particular, realtime strategy games provide a multi-scale challenge which requires both deliberative and reactive reasoning processes. Experts approach this task by studying a corpus of games, building models for anticipating opponent actions, and practicing within the game environment. We motivate the need for integrating heterogeneous approaches by enumerating a range of competencies involved in gameplay and discuss how they are being implemented in EISBot, a reactive planning agent that we have applied to the task of playing real-time strategy games at the same granularity as humans.
---
paper_title: A Bayesian model for RTS units control applied to StarCraft

paper_content:

In real-time strategy games (RTS), the player must reason about high-level strategy and planning while having effective tactics and even individual units micro-management. Enabling an artificial agent to deal with such a task entails breaking down the complexity of this environment. For that, we propose to control units locally in the Bayesian sensory motor robot fashion, with higher level orders integrated as perceptions. As complete inference encompassing global strategy down to individual unit needs is intractable, we embrace incompleteness through a hierarchical model able to deal with uncertainty. We developed and applied our approach on a StarCraft1 AI.
---
paper_title: Terrain analysis in real-time strategy games: an integrated approach to choke point detection and region decomposition

paper_content:

Autonomous agents in real-time strategy (RTS) games lack an integrated framework for reasoning about choke points and regions of open space in their environment. This paper presents an algorithm which partitions the environment into a set of polygonal regions and computes optimal choke points between adjacent regions. This representation can be used as a component for AI agents to reason about terrain, plan multiple routes of attack, and make other tactical decisions. The algorithm is tested on a set of popular maps commonly used in international Starcraft competitions and evaluated against answers made by human participants. The algorithm identified 97% of the choke points that the participants found and also identified a number of bottlenecks that human participants did not recognize as choke points.
---
paper_title: A data mining approach to strategy prediction

paper_content:

We present a data mining approach to opponent modeling in strategy games. Expert gameplay is learned by applying machine learning techniques to large collections of game logs. This approach enables domain independent algorithms to acquire domain knowledge and perform opponent modeling. Machine learning algorithms are applied to the task of detecting an opponent's strategy before it is executed and predicting when an opponent will perform strategic actions. Our approach involves encoding game logs as a feature vector representation, where each feature describes when a unit or building type is first produced. We compare our representation to a state lattice representation in perfect and imperfect information environments and the results show that our representation has higher predictive capabilities and is more tolerant of noise. We also discuss how to incorporate our data mining approach into a full game playing agent.
---
paper_title: Strategies for Strategy Game AI

paper_content:

The biggest challenge in computer strategy games is the creation of a fun computer opponent. The hardest element of "fun" is "good". For the Artificial Intelligence to be good, it must do much of what a human player would do: situational & map analysis, resource allocation, and strategy execution. We present here the framework and some of the strategies we have developed at Activision in the development of games such as Dark Reign: The Future Of War, Battlezone, Civilization: Call To Power, and others.
---
paper_title: Reinforcement Learning: An Introduction

paper_content:

Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.
---
paper_title: Connectionist reinforcement learning for intelligent unit micro management in StarCraft

paper_content:

Real Time Strategy Games are one of the most popular game schemes in PC markets and offer a dynamic environment that involves several interacting agents. The core strategies that need to be developed in these games are unit micro management, building order, resource management, and the game main tactic. Unfortunately, current games only use scripted and fixed behaviors for their artificial intelligence (AI), and the player can easily learn the counter measures to defeat the AI. In this paper, we describe a system based on neural networks that controls a set of units of the same type in the popular game StarCraft. Using the neural networks, the units will either choose a unit to attack or evade from the battlefield. The system uses reinforcement learning combined with neural networks using online Sarsa and neural-fitted Sarsa, both with a short term memory reward function. We also present an incremental learning method for training the units for larger scenarios involving more units using trained neural networks on smaller scenarios. Additionally, we developed a novel sensing system to feed the environment data to the neural networks using separate vision grids. The simulation results show superior performance against the human-made AI scripts in StarCraft.
---
paper_title: Concurrent Hierarchical Reinforcement Learning

paper_content:

We consider applying hierarchical reinforcement learning techniques to problems in which an agent has several effectors to control simultaneously. We argue that the kind of prior knowledge one typically has about such problems is best expressed using a multithreaded partial program, and present concurrent ALisp, a language for specifying such partial programs. We describe algorithms for learning and acting with concurrent ALisp that can be efficient even when there are exponentially many joint choices at each decision point. Finally, we show results of applying these methods to a complex computer game domain.
---
paper_title: Reinforcement Learning via Practice and Critique Advice

paper_content:

We consider the problem of incorporating end-user advice into reinforcement learning (RL). In our setting, the learner alternates between practicing, where learning is based on actual world experience, and end-user critique sessions where advice is gathered. During each critique session the end-user is allowed to analyze a trajectory of the current policy and then label an arbitrary subset of the available actions as good or bad. Our main contribution is an approach for integrating all of the information gathered during practice and critiques in order to effectively optimize a parametric policy. The approach optimizes a loss function that linearly combines losses measured against the world experience and the critique data. We evaluate our approach using a prototype system for teaching tactical battle behavior in a real-time strategy game engine. Results are given for a significant evaluation involving ten end-users showing the promise of this approach and also highlighting challenges involved in inserting end-users into the RL loop.
---
paper_title: Learning continuous action models in a real-time strategy environment

paper_content:

Although several researchers have integrated methods for reinforcement learning (RL) with case-based reasoning (CBR) to model continuous action spaces, existing integrations typically employ discrete approximations of these models. This limits the set of actions that can be modeled, and may lead to non-optimal solutions. We introduce the Continuous Action and State Space Learner (CASSL), an integrated RL/CBR algorithm that uses continuous models directly. Our empirical study shows that CASSL significantly outperforms two baseline approaches for selecting actions on a task from a real-time strategy gaming environment.
---
paper_title: Transfer Learning in Real-Time Strategy Games Using Hybrid CBR/RL

paper_content:

The goal of transfer learning is to use the knowledge acquired in a set of source tasks to improve performance in a related but previously unseen target task. In this paper, we present a multilayered architecture named CAse-Based Reinforcement Learner (CARL). It uses a novel combination of Case-Based Reasoning (CBR) and Reinforcement Learning (RL) to achieve transfer while playing against the Game AI across a variety of scenarios in MadRTSTM, a commercial Real Time Strategy game. Our experiments demonstrate that CARL not only performs well on individual tasks but also exhibits significant performance gains when allowed to transfer knowledge from previous tasks.
---
paper_title: Connectionist reinforcement learning for intelligent unit micro management in StarCraft

paper_content:

Real Time Strategy Games are one of the most popular game schemes in PC markets and offer a dynamic environment that involves several interacting agents. The core strategies that need to be developed in these games are unit micro management, building order, resource management, and the game main tactic. Unfortunately, current games only use scripted and fixed behaviors for their artificial intelligence (AI), and the player can easily learn the counter measures to defeat the AI. In this paper, we describe a system based on neural networks that controls a set of units of the same type in the popular game StarCraft. Using the neural networks, the units will either choose a unit to attack or evade from the battlefield. The system uses reinforcement learning combined with neural networks using online Sarsa and neural-fitted Sarsa, both with a short term memory reward function. We also present an incremental learning method for training the units for larger scenarios involving more units using trained neural networks on smaller scenarios. Additionally, we developed a novel sensing system to feed the environment data to the neural networks using separate vision grids. The simulation results show superior performance against the human-made AI scripts in StarCraft.
---
paper_title: Adversarial Planning Through Strategy Simulation

paper_content:

Adversarial planning in highly complex decision domains, such as modern video games, has not yet received much attention from AI researchers. In this paper, we present a planning framework that uses strategy simulation in conjunction with Nash-equilibrium strategy approximation. We apply this framework to an army deployment problem in a real-time strategy game setting and present experimental results that indicate a performance gain over the scripted strategies that the system is built on. This technique provides an automated way of increasing the decision quality of scripted AI systems and is therefore ideally suited for video games and combat simulators
---
paper_title: Fast heuristic search for RTS game combat scenarios

paper_content:

Heuristic search has been very successful in abstract game domains such as Chess and Go. In video games, however, adoption has been slow due to the fact that state and move spaces are much larger, real-time constraints are harsher, and constraints on computational resources are tighter. In this paper we present a fast search method — Alpha-Beta search for durative moves — that can defeat commonly used AI scripts in RTS game combat scenarios of up to 8 vs. 8 units running on a single core in under 5ms per search episode. This performance is achieved by using standard search enhancements such as transposition tables and iterative deepening, and novel usage of combat AI scripts for sorting moves and state evaluation via playouts. We also present evidence that commonly used combat scripts are highly exploitable — opening the door for a promising line of research on opponent combat modelling.
---
paper_title: UCT for Tactical Assault Planning in Real-Time Strategy Games

paper_content:

We consider the problem of tactical assault planning in real-time strategy games where a team of friendly agents must launch an assault on an enemy. This problem offers many challenges including a highly dynamic and uncertain environment, multiple agents, durative actions, numeric attributes, and different optimization objectives. While the dynamics of this problem are quite complex, it is often possible to provide or learn a coarse simulation-based model of a tactical domain, which makes Monte-Carlo planning an attractive approach. In this paper, we investigate the use of UCT, a recent Monte-Carlo planning algorithm for this problem. UCT has recently shown impressive successes in the area of games, particularly Go, but has not yet been considered in the context of multiagent tactical planning. We discuss the challenges of adapting UCT to our domain and an implementation which allows for the optimization of user specified objective functions. We present an evaluation of our approach on a range of tactical assault problems with different objectives in the RTS game Wargus. The results indicate that our planner is able to generate superior plans compared to several baselines and a human player.
---
paper_title: Neuroevolution based multi-agent system for micromanagement in real-time strategy games

paper_content:

The main goal of this paper is the design of a multi-agent system (MAS) that handles unit micromanagement in real time strategy games and is able to adapt/learn during game play. To achieve this we adopted the rtNEAT approach in order to obtain customized neural network topologies, thus avoiding the generation of too complex architectures. Also by defining internal and external inputs for each agent we managed to create independent agents that are able to cooperate and form teams for their mutual benefit and at the same time eliminate unnecessary communication overhead. The MAS was implemented for the real time strategy game StarCraft using JADE multi-agent platform and BWAPI to ensure the interface with the game. We used as a baseline the in game AI and also tested it against other adapting AI systems in order to compare their performance against our system.
---
paper_title: A Bayesian model for RTS units control applied to StarCraft

paper_content:

In real-time strategy games (RTS), the player must reason about high-level strategy and planning while having effective tactics and even individual units micro-management. Enabling an artificial agent to deal with such a task entails breaking down the complexity of this environment. For that, we propose to control units locally in the Bayesian sensory motor robot fashion, with higher level orders integrated as perceptions. As complete inference encompassing global strategy down to individual unit needs is intractable, we embrace incompleteness through a hierarchical model able to deal with uncertainty. We developed and applied our approach on a StarCraft1 AI.
---
paper_title: Situation Assessment for Plan Retrieval in Real-Time Strategy Games

paper_content:

Case-Based Planning (CBP) is an effective technique for solving planning problems that has the potential to reduce the computational complexity of the generative planning approaches [8,3]. However, the success of plan execution using CBP depends highly on the selection of a correct plan; especially when the case-base of plans is extensive. In this paper we introduce the concept of a situationand explain a situation assessmentalgorithm which improves plan retrieval for CBP. We have applied situation assessment to our previous CBP system, Darmok [11], in the domain of real-time strategy games. During Darmok's execution using situation assessment, the high-level representation of the game state i.e. situation is predicted using a decision tree based Situation-Classification model. Situation predicted is further used for the selection of relevant knowledge intensive features, which are derived from the basic representation of the game state, to compute the similarity of cases with the current problem. The feature selection performed here is knowledge based and improves the performance of similarity measurements during plan retrieval. The instantiation of the situation assessment algorithm to Darmok gave us promising results for plan retrieval within the real-time constraints.
---
paper_title: A Behavior Language for Story-based Believable Agents

paper_content:

A behavior language is a reactive planning language, based on the Oz Project language Hap, designed specifically for authoring believable agents-characters that express rich personality, and that, in this case, play roles in an interactive story called Facade.
---
paper_title: Automatically acquiring domain knowledge for adaptive game AI using evolutionary learning

paper_content:

Game AI is the decision-making process of computer-controlled opponents in computer games. Adaptive game AI can improve the entertainment value of computer games. It allows computer-controlled opponents to automatically fix weaknesses in the game AI and respond to changes in human-player tactics. Dynamic scripting is a recently developed approach for adaptive game AI that learns which tactics (i.e., action sequences) an opponent should select to play effectively against the human player. In previous work, these tactics were manually generated. We introduce AKADS; it uses an evolutionary algorithm to automatically generate such tactics. Our experiments show that it improves dynamic scripting's performance on a real-time strategy (RTS) game. Therefore, we conclude that high-quality domain knowledge (i.e., tactics) can be automatically generated for strong adaptive AI opponents in RTS games. This reduces the time and effort required by game developers to create intelligent game AI, thus freeing them to focus on other important topics (e.g., storytelling, graphics).
---
paper_title: Learning to Win: Case-Based Plan Selection in a Real-Time Strategy Game

paper_content:

While several researchers have applied case-based reasoning techniques to games, only Ponsen and Spronck (2004) have addressed the challenging problem of learning to win real-time games. Focusing on Wargus, they report good results for a genetic algorithm that searches in plan space, and for a weighting algorithm (dynamic scripting) that biases subplan retrieval. However, both approaches assume a static opponent, and were not designed to transfer their learned knowledge to opponents with substantially different strategies. We introduce a plan retrieval algorithm that, by using three key sources of domain knowledge, removes the assumption of a static opponent. Our experiments show that its implementation in the Case-based Tactician (CaT) significantly outperforms the best among a set of genetically evolved plans when tested against random Wargus opponents. CaT communicates with Wargus through TIELT, a testbed for integrating and evaluating decision systems with simulators. This is the first application of TIELT. We describe this application, our lessons learned, and our motivations for future work.
---
paper_title: Combining expert knowledge and learning from demonstration in real-time strategy games

paper_content:

Case-based planning (CBP) is usually considered a good solution to solve the knowledge acquisition problem that arises when developing AIs for real-time strategy games. Unlike more classical approaches, such as state machines or rule-based systems, CBP allows experts to train AIs directly from games recorded by expert players. Unfortunately, this simple approach has also some drawbacks, for example it is not easy to refine an existing case base to learn specific strategies when a long game session is needed to create a new trace. Furthermore, CBP may be too reactive to small changes in the game state and, at the same time, do not respond fast enough to important changes in the opponent's strategy. We propose to alleviate these problems by letting experts to inject decision making knowledge into the system in the form of behavior trees, and we show promising results in some experiments using Starcraft.
---
paper_title: Fuzzy case-based reasoning for managing strategic and tactical reasoning in starcraft

paper_content:

We present the combination of Fuzzy sets and Case-Based Reasoning (FCBR) to deal with strategic and tactical management in the real-time strategy environment of StarCraft. Case-based reasoning is a problem solving AI approach that uses past experience to deal with actual problems. Fuzzy set theory is used in case representation to provide a characterization of imprecise and uncertain information. The results revealed that our system can successfully reason about strategies and tactics, defeating the built-in AI of StarCraft. The principal conclusion was that FCBR can reason with abstract information and a large space of actions. Moreover, the resulting system shows its potential to incorporate human knowledge and can effectively adapt to varying conditions of the map.
---
paper_title: Concurrent Hierarchical Reinforcement Learning

paper_content:

We consider applying hierarchical reinforcement learning techniques to problems in which an agent has several effectors to control simultaneously. We argue that the kind of prior knowledge one typically has about such problems is best expressed using a multithreaded partial program, and present concurrent ALisp, a language for specifying such partial programs. We describe algorithms for learning and acting with concurrent ALisp that can be efficient even when there are exponentially many joint choices at each decision point. Finally, we show results of applying these methods to a complex computer game domain.
---
paper_title: Learning hierarchical task networks by observation

paper_content:

Knowledge-based planning methods offer benefits over classical techniques, but they are time consuming and costly to construct. There has been research on learning plan knowledge from search, but this can take substantial computer time and may even fail to find solutions on complex tasks. Here we describe another approach that observes sequences of operators taken from expert solutions to problems and learns hierarchical task networks from them. The method has similarities to previous algorithms for explanation-based learning, but differs in its ability to acquire hierarchical structures and in the generality of learned conditions. These increase the method's capability to transfer learned knowledge to other problems and supports the acquisition of recursive procedures. After presenting the learning algorithm, we report experiments that compare its abilities to other techniques on two planning domains. In closing, we review related work and directions for future research.
---
paper_title: Combining expert knowledge and learning from demonstration in real-time strategy games

paper_content:

Case-based planning (CBP) is usually considered a good solution to solve the knowledge acquisition problem that arises when developing AIs for real-time strategy games. Unlike more classical approaches, such as state machines or rule-based systems, CBP allows experts to train AIs directly from games recorded by expert players. Unfortunately, this simple approach has also some drawbacks, for example it is not easy to refine an existing case base to learn specific strategies when a long game session is needed to create a new trace. Furthermore, CBP may be too reactive to small changes in the game state and, at the same time, do not respond fast enough to important changes in the opponent's strategy. We propose to alleviate these problems by letting experts to inject decision making knowledge into the system in the form of behavior trees, and we show promising results in some experiments using Starcraft.
---
paper_title: Combining expert knowledge and learning from demonstration in real-time strategy games

paper_content:

Case-based planning (CBP) is usually considered a good solution to solve the knowledge acquisition problem that arises when developing AIs for real-time strategy games. Unlike more classical approaches, such as state machines or rule-based systems, CBP allows experts to train AIs directly from games recorded by expert players. Unfortunately, this simple approach has also some drawbacks, for example it is not easy to refine an existing case base to learn specific strategies when a long game session is needed to create a new trace. Furthermore, CBP may be too reactive to small changes in the game state and, at the same time, do not respond fast enough to important changes in the opponent's strategy. We propose to alleviate these problems by letting experts to inject decision making knowledge into the system in the form of behavior trees, and we show promising results in some experiments using Starcraft.
---
paper_title: Learning from demonstration for goal-driven autonomy

paper_content:

Goal-driven autonomy (GDA) is a conceptual model for creating an autonomous agent that monitors a set of expectations during plan execution, detects when discrepancies occur, builds explanations for the cause of failures, and formulates new goals to pursue when planning failures arise. While this framework enables the development of agents that can operate in complex and dynamic environments, implementing the logic for each of the subtasks in the model requires substantial domain engineering.We present a method using case-based reasoning and intent recognition in order to build GDA agents that learn from demonstrations. Our approach reduces the amount of domain engineering necessary to implement GDA agents and learns expectations, explanations, and goals from expert demonstrations. We have applied this approach to build an agent for the real-time strategy game StarCraft. Our results show that integrating the GDA conceptual model into the agent greatly improves its win rate.
---
paper_title: Build order optimization in StarCraft

paper_content:

In recent years, real-time strategy (RTS) games have gained interest in the AI research community for their multitude of challenging subproblems — such as collaborative pathfinding, effective resource allocation and unit targeting, to name a few. In this paper we consider the build order problem in RTS games in which we need to find concurrent action sequences that, constrained by unit dependencies and resource availability, create a certain number of units and structures in the shortest possible time span. We present abstractions and heuristics that speed up the search for approximative solutions considerably in the game of StarCraft, and show the efficacy of our method by comparing its real-time performance with that of professional StarCraft players.
---
paper_title: Applying Goal-Oriented Action Planning to Games

paper_content:

A number of games have implemented characters with goal directed decision-making capabilities. A goal-directed character displays some measure of intelligence by autonomously deciding to activate the behavior that will satisfy the most relevant goal at any instance. Goal-Oriented Action Planning (GOAP) is a decision-making architecture that takes the next step, and allows characters to decide not only what to do, but how to do it. But why would we want to empower our characters with so much freedom?
---
paper_title: Online Planning for Resource Production in Real-Time Strategy Games

paper_content:

Planning in domains with temporal and numerical properties is an important research problem. One application of this is the resource production problem in real-time strategy (RTS) games, where players attempt to achieve the goal of producing a certain amount of resources as fast as possible. In this paper, we develop an online planner for resource production in the RTS game of Wargus, where the preconditions and effects of the actions obey many properties that are common across RTS games. Our planner is based on a computationally efficient action-selection mechanism, which at each decision epoch creates a possibly sub-optimal concurrent plan from the current state to the goal and then begins executing the initial set of actions. The plan is formed via a combination of means-ends analysis, scheduling, and a bounded search over sub-goals that are not required for goal achievement but may improve makespan. Experiments in the RTS game of Wargus show that the online planner is highly competitive with a human expert and often performs significantly better than state-of-the-art planning algorithms for this domain.
---
paper_title: Automatically acquiring domain knowledge for adaptive game AI using evolutionary learning

paper_content:

Game AI is the decision-making process of computer-controlled opponents in computer games. Adaptive game AI can improve the entertainment value of computer games. It allows computer-controlled opponents to automatically fix weaknesses in the game AI and respond to changes in human-player tactics. Dynamic scripting is a recently developed approach for adaptive game AI that learns which tactics (i.e., action sequences) an opponent should select to play effectively against the human player. In previous work, these tactics were manually generated. We introduce AKADS; it uses an evolutionary algorithm to automatically generate such tactics. Our experiments show that it improves dynamic scripting's performance on a real-time strategy (RTS) game. Therefore, we conclude that high-quality domain knowledge (i.e., tactics) can be automatically generated for strong adaptive AI opponents in RTS games. This reduces the time and effort required by game developers to create intelligent game AI, thus freeing them to focus on other important topics (e.g., storytelling, graphics).
---
paper_title: Learning to Win: Case-Based Plan Selection in a Real-Time Strategy Game

paper_content:

While several researchers have applied case-based reasoning techniques to games, only Ponsen and Spronck (2004) have addressed the challenging problem of learning to win real-time games. Focusing on Wargus, they report good results for a genetic algorithm that searches in plan space, and for a weighting algorithm (dynamic scripting) that biases subplan retrieval. However, both approaches assume a static opponent, and were not designed to transfer their learned knowledge to opponents with substantially different strategies. We introduce a plan retrieval algorithm that, by using three key sources of domain knowledge, removes the assumption of a static opponent. Our experiments show that its implementation in the Case-based Tactician (CaT) significantly outperforms the best among a set of genetically evolved plans when tested against random Wargus opponents. CaT communicates with Wargus through TIELT, a testbed for integrating and evaluating decision systems with simulators. This is the first application of TIELT. We describe this application, our lessons learned, and our motivations for future work.
---
paper_title: SORTS: A Human-Level Approach to Real-Time Strategy AI

paper_content:

We developed knowledge-rich agents to play real-time strategy games by interfacing the ORTS game engine to the Soar cognitive architecture. The middleware we developed supports grouping, attention, coordinated path finding, and FSM control of low-level unit behaviors. The middleware attempts to provide information humans use to reason about RTS games, and facilitates creating agent behaviors in Soar. Agents implemented with this system won two out of three categories in the AIIDE 2006 ORTS competition.
---
paper_title: Call for AI Research in RTS Games

paper_content:

This position paper discusses AI challenges in the area of real‐time strategy games and presents a research agenda aimed at improving AI performance in these popular multi‐ player computer games.
---
paper_title: The rise of potential fields in real time strategy bots

paper_content:

Bots for Real Time Strategy (RTS) games are challenging to implement. A bot controls a number of units that may have to navigate in a partially unknown environment, while at the same time search for enemies and coordinate attacks to fight them down. Potential fields is a technique originating from the area of robotics where it is used in controlling the navigation of robots in dynamic environments. We show that the use of potential fields for implementing a bot for a real time strategy game gives us a very competitive, configurable, and non-conventional solution.
---
paper_title: Opponent behaviour recognition for real-time strategy games

paper_content:

In Real-Time Strategy (RTS) video games, players (controlled by humans or computers) build structures and recruit armies, fight for space and resources in order to control strategic points, destroy the opposing force and ultimately win the game. Players need to predict where and how the opponents will strike in order to best defend themselves. Conversely, assessing how the opponents will defend themselves is crucial to mounting a successful attack while exploiting the vulnerabilities in the opponent's defence strategy. In this context, to be truly adaptable, computer-controlled players need to recognize their opponents' behaviour, their goals, and their plans to achieve those goals. In this paper we analyze the algorithmic challenges behind behaviour recognition in RTS games and discuss a generic RTS behaviour recognition system that we are developing to address those challenges. The application domain is that of RTS games, but many of the key points we discuss also apply to other video game genres such as multiplayer first person shooter (FPS) games.
---
paper_title: SORTS: A Human-Level Approach to Real-Time Strategy AI

paper_content:

We developed knowledge-rich agents to play real-time strategy games by interfacing the ORTS game engine to the Soar cognitive architecture. The middleware we developed supports grouping, attention, coordinated path finding, and FSM control of low-level unit behaviors. The middleware attempts to provide information humans use to reason about RTS games, and facilitates creating agent behaviors in Soar. Agents implemented with this system won two out of three categories in the AIIDE 2006 ORTS competition.
---
paper_title: A particle model for state estimation in real-time strategy games

paper_content:

A big challenge for creating human-level game AI is building agents capable of operating in imperfect information environments. In real-time strategy games the technological progress of an opponent and locations of enemy units are partially observable. To overcome this limitation, we explore a particle-based approach for estimating the location of enemy units that have been encountered. We represent state estimation as an optimization problem, and automatically learn parameters for the particle model by mining a corpus of expert StarCraft replays. The particle model tracks opponent units and provides conditions for activating tactical behaviors in our StarCraft bot. Our results show that incorporating a learned particle model improves the performance of EIS Bot by 10% over baseline approaches.
---
paper_title: Building Human-Level AI for Real-Time Strategy Games

paper_content:

Video games are complex simulation environments with many real-world properties that need to be addressed in order to build robust intelligence. In particular, realtime strategy games provide a multi-scale challenge which requires both deliberative and reactive reasoning processes. Experts approach this task by studying a corpus of games, building models for anticipating opponent actions, and practicing within the game environment. We motivate the need for integrating heterogeneous approaches by enumerating a range of competencies involved in gameplay and discuss how they are being implemented in EISBot, a reactive planning agent that we have applied to the task of playing real-time strategy games at the same granularity as humans.
---
paper_title: A Bayesian model for RTS units control applied to StarCraft

paper_content:

In real-time strategy games (RTS), the player must reason about high-level strategy and planning while having effective tactics and even individual units micro-management. Enabling an artificial agent to deal with such a task entails breaking down the complexity of this environment. For that, we propose to control units locally in the Bayesian sensory motor robot fashion, with higher level orders integrated as perceptions. As complete inference encompassing global strategy down to individual unit needs is intractable, we embrace incompleteness through a hierarchical model able to deal with uncertainty. We developed and applied our approach on a StarCraft1 AI.
---
paper_title: Terrain analysis in real-time strategy games: an integrated approach to choke point detection and region decomposition

paper_content:

Autonomous agents in real-time strategy (RTS) games lack an integrated framework for reasoning about choke points and regions of open space in their environment. This paper presents an algorithm which partitions the environment into a set of polygonal regions and computes optimal choke points between adjacent regions. This representation can be used as a component for AI agents to reason about terrain, plan multiple routes of attack, and make other tactical decisions. The algorithm is tested on a set of popular maps commonly used in international Starcraft competitions and evaluated against answers made by human participants. The algorithm identified 97% of the choke points that the participants found and also identified a number of bottlenecks that human participants did not recognize as choke points.
---
paper_title: CASE-BASED PLAN RECOGNITION FOR REAL-TIME STRATEGY GAMES

paper_content:

The current game industry around the world is one of the fastest growing industries. One gaming genre that is very popular is the real-time strategy games. However, current implementations of games apply extensive usage of FSM that makes them highly predictable and provides less replayability. Thus, this paper looks at the possibility of employing case-based plan recognition for NPCs so as to minimize their predictability. The paper also looks into possible representation adaptions to minimize the resource requirement to maintain the possibility of deployment in mobile devices.
---
paper_title: Building a player strategy model by analyzing replays of real-time strategy games

paper_content:

Developing computer-controlled groups to engage in combat, control the use of limited resources, and create units and buildings in real-time strategy (RTS) games is a novel application in game AI. However, tightly controlled online commercial game pose challenges to researchers interested in observing player activities, constructing player strategy models, and developing practical AI technology in them. Instead of setting up new programming environments or building a large amount of agentpsilas decision rules by playerpsilas experience for conducting real-time AI research, the authors use replays of the commercial RTS game StarCraft to evaluate human player behaviors and to construct an intelligent system to learn human-like decisions and behaviors. A case-based reasoning approach was applied for the purpose of training our system to learn and predict player strategies. Our analysis indicates that the proposed system is capable of learning and predicting individual player strategies, and that players provide evidence of their personal characteristics through their building construction order.
---
paper_title: Opponent behaviour recognition for real-time strategy games

paper_content:

In Real-Time Strategy (RTS) video games, players (controlled by humans or computers) build structures and recruit armies, fight for space and resources in order to control strategic points, destroy the opposing force and ultimately win the game. Players need to predict where and how the opponents will strike in order to best defend themselves. Conversely, assessing how the opponents will defend themselves is crucial to mounting a successful attack while exploiting the vulnerabilities in the opponent's defence strategy. In this context, to be truly adaptable, computer-controlled players need to recognize their opponents' behaviour, their goals, and their plans to achieve those goals. In this paper we analyze the algorithmic challenges behind behaviour recognition in RTS games and discuss a generic RTS behaviour recognition system that we are developing to address those challenges. The application domain is that of RTS games, but many of the key points we discuss also apply to other video game genres such as multiplayer first person shooter (FPS) games.
---
paper_title: A Bayesian Model for Plan Recognition in RTS Games applied to StarCraft

paper_content:

The task of keyhole (unobtrusive) plan recognition is central to adaptive game AI. "Tech trees" or "build trees" are the core of real-time strategy (RTS) game strategic (long term) planning. This paper presents a generic and simple Bayesian model for RTS build tree prediction from noisy observations, which parameters are learned from replays (game logs). This unsupervised machine learning approach involves minimal work for the game developers as it leverage players' data (common in RTS). We applied it to StarCraft1 and showed that it yields high quality and robust predictions, that can feed an adaptive AI.
---
paper_title: A data mining approach to strategy prediction

paper_content:

We present a data mining approach to opponent modeling in strategy games. Expert gameplay is learned by applying machine learning techniques to large collections of game logs. This approach enables domain independent algorithms to acquire domain knowledge and perform opponent modeling. Machine learning algorithms are applied to the task of detecting an opponent's strategy before it is executed and predicting when an opponent will perform strategic actions. Our approach involves encoding game logs as a feature vector representation, where each feature describes when a unit or building type is first produced. We compare our representation to a state lattice representation in perfect and imperfect information environments and the results show that our representation has higher predictive capabilities and is more tolerant of noise. We also discuss how to incorporate our data mining approach into a full game playing agent.
---
paper_title: Opponent behaviour recognition for real-time strategy games

paper_content:

In Real-Time Strategy (RTS) video games, players (controlled by humans or computers) build structures and recruit armies, fight for space and resources in order to control strategic points, destroy the opposing force and ultimately win the game. Players need to predict where and how the opponents will strike in order to best defend themselves. Conversely, assessing how the opponents will defend themselves is crucial to mounting a successful attack while exploiting the vulnerabilities in the opponent's defence strategy. In this context, to be truly adaptable, computer-controlled players need to recognize their opponents' behaviour, their goals, and their plans to achieve those goals. In this paper we analyze the algorithmic challenges behind behaviour recognition in RTS games and discuss a generic RTS behaviour recognition system that we are developing to address those challenges. The application domain is that of RTS games, but many of the key points we discuss also apply to other video game genres such as multiplayer first person shooter (FPS) games.
---
paper_title: Learning from demonstration for goal-driven autonomy

paper_content:

Goal-driven autonomy (GDA) is a conceptual model for creating an autonomous agent that monitors a set of expectations during plan execution, detects when discrepancies occur, builds explanations for the cause of failures, and formulates new goals to pursue when planning failures arise. While this framework enables the development of agents that can operate in complex and dynamic environments, implementing the logic for each of the subtasks in the model requires substantial domain engineering.We present a method using case-based reasoning and intent recognition in order to build GDA agents that learn from demonstrations. Our approach reduces the amount of domain engineering necessary to implement GDA agents and learns expectations, explanations, and goals from expert demonstrations. We have applied this approach to build an agent for the real-time strategy game StarCraft. Our results show that integrating the GDA conceptual model into the agent greatly improves its win rate.
---
paper_title: Opponent behaviour recognition for real-time strategy games

paper_content:

In Real-Time Strategy (RTS) video games, players (controlled by humans or computers) build structures and recruit armies, fight for space and resources in order to control strategic points, destroy the opposing force and ultimately win the game. Players need to predict where and how the opponents will strike in order to best defend themselves. Conversely, assessing how the opponents will defend themselves is crucial to mounting a successful attack while exploiting the vulnerabilities in the opponent's defence strategy. In this context, to be truly adaptable, computer-controlled players need to recognize their opponents' behaviour, their goals, and their plans to achieve those goals. In this paper we analyze the algorithmic challenges behind behaviour recognition in RTS games and discuss a generic RTS behaviour recognition system that we are developing to address those challenges. The application domain is that of RTS games, but many of the key points we discuss also apply to other video game genres such as multiplayer first person shooter (FPS) games.
---
paper_title: Learning from demonstration for goal-driven autonomy

paper_content:

Goal-driven autonomy (GDA) is a conceptual model for creating an autonomous agent that monitors a set of expectations during plan execution, detects when discrepancies occur, builds explanations for the cause of failures, and formulates new goals to pursue when planning failures arise. While this framework enables the development of agents that can operate in complex and dynamic environments, implementing the logic for each of the subtasks in the model requires substantial domain engineering.We present a method using case-based reasoning and intent recognition in order to build GDA agents that learn from demonstrations. Our approach reduces the amount of domain engineering necessary to implement GDA agents and learns expectations, explanations, and goals from expert demonstrations. We have applied this approach to build an agent for the real-time strategy game StarCraft. Our results show that integrating the GDA conceptual model into the agent greatly improves its win rate.
---
paper_title: Learning Probabilistic Behavior Models in Real-time Strategy Games

paper_content:

We study the problem of learning probabilistic models of high-level strategic behavior in the real-time strategy (RTS) game StarCraft. The models are automatically learned from sets of game logs and aim to capture the common strategic states and decision points that arise in those games. Unlike most work on behavior/strategy learning and prediction in RTS games, our data-centric approach is not biased by or limited to any set of preconceived strategic concepts. Further, since our behavior model is based on the well-developed and generic paradigm of hidden Markov models, it supports a variety of uses for the design of AI players and human assistants. For example, the learned models can be used to make probabilistic predictions of a player's future actions based on observations, to simulate possible future trajectories of a player, or to identify uncharacteristic or novel strategies in a game database. In addition, the learned qualitative structure of the model can be analyzed by humans in order to categorize common strategic elements. We demonstrate our approach by learning models from 331 expert-level games and provide both a qualitative and quantitative assessment of the learned model's utility.
---
paper_title: A Bayesian Model for Plan Recognition in RTS Games applied to StarCraft

paper_content:

The task of keyhole (unobtrusive) plan recognition is central to adaptive game AI. "Tech trees" or "build trees" are the core of real-time strategy (RTS) game strategic (long term) planning. This paper presents a generic and simple Bayesian model for RTS build tree prediction from noisy observations, which parameters are learned from replays (game logs). This unsupervised machine learning approach involves minimal work for the game developers as it leverage players' data (common in RTS). We applied it to StarCraft1 and showed that it yields high quality and robust predictions, that can feed an adaptive AI.
---
paper_title: Building a player strategy model by analyzing replays of real-time strategy games

paper_content:

Developing computer-controlled groups to engage in combat, control the use of limited resources, and create units and buildings in real-time strategy (RTS) games is a novel application in game AI. However, tightly controlled online commercial game pose challenges to researchers interested in observing player activities, constructing player strategy models, and developing practical AI technology in them. Instead of setting up new programming environments or building a large amount of agentpsilas decision rules by playerpsilas experience for conducting real-time AI research, the authors use replays of the commercial RTS game StarCraft to evaluate human player behaviors and to construct an intelligent system to learn human-like decisions and behaviors. A case-based reasoning approach was applied for the purpose of training our system to learn and predict player strategies. Our analysis indicates that the proposed system is capable of learning and predicting individual player strategies, and that players provide evidence of their personal characteristics through their building construction order.
---
paper_title: Building a player strategy model by analyzing replays of real-time strategy games

paper_content:

Developing computer-controlled groups to engage in combat, control the use of limited resources, and create units and buildings in real-time strategy (RTS) games is a novel application in game AI. However, tightly controlled online commercial game pose challenges to researchers interested in observing player activities, constructing player strategy models, and developing practical AI technology in them. Instead of setting up new programming environments or building a large amount of agentpsilas decision rules by playerpsilas experience for conducting real-time AI research, the authors use replays of the commercial RTS game StarCraft to evaluate human player behaviors and to construct an intelligent system to learn human-like decisions and behaviors. A case-based reasoning approach was applied for the purpose of training our system to learn and predict player strategies. Our analysis indicates that the proposed system is capable of learning and predicting individual player strategies, and that players provide evidence of their personal characteristics through their building construction order.
---
paper_title: Case Acquisition Strategies for Case-Based Reasoning in Real-Time Strategy Games

paper_content:

Real-time Strategy (RTS) games are complex domains which are a significant challenge to both human and artificial intelligence (AI). For that reason, and although many AI approaches have been proposed for the RTS game AI problem, the AI of all commercial RTS games is scripted and offers a very static behavior subject to exploits. In this paper, we will focus on a case-based reasoning (CBR) approach to this problem, and concentrate on the process of case-acquisition. Specifically, we will describe 7 different techniques to automatically acquire plans by observing human demonstrations and compare their performance when using them in the Darmok 2 system in the context of an RTS game.
---
paper_title: Learning robot soccer skills from demonstration

paper_content:

We seek to enable users to teach personal robots arbitrary tasks so that the robot can better perform as the user desires without explicit programming. Robot learning from demonstration is an approach well-suited to this paradigm, as a robot learns new tasks from observations of the task itself. Many current robot learning algorithms require the existence of basic behaviors that can be combined to perform the desired task. However, robots that exist in the world for long timeframes and learn many tasks over their lifetime may exhaust this basis set and need to move beyond it. In particular, we are interested in a robot that must learn to perform an unknown task for which its built in behaviors may not be appropriate. We demonstrate a learning paradigm that is capable of learning both low-level motion primitives (locomotion and manipulation) and high-level tasks built on top of them from interactive demonstration. We apply nonparametric regression within this framework towards learning a complete robot soccer player and successfully teach a robot dog to first walk, and then to seek and acquire a ball.
---
paper_title: Call for AI Research in RTS Games

paper_content:

This position paper discusses AI challenges in the area of real‐time strategy games and presents a research agenda aimed at improving AI performance in these popular multi‐ player computer games.
---
paper_title: Measuring player experience on runtime dynamic difficulty scaling in an RTS game

paper_content:

Do players find it more enjoyable to win, than to play even matches? We have made a study of what a number of players expressed after playing against computer opponents of different kinds in an RTS game. There were two static computer opponents, one that was easily beaten, and one that was hard to beat, and three dynamic ones that adapted their strength to that of the player. One of these three latter ones intentionally drops its performance in the end of the game to make it easy for the player to win. Our results indicate that the players found it more enjoyable to play an even game against an opponent that adapts to the performance of the player, than playing against an opponent with static difficulty. The results also show that when the computer player that dropped its performance to let the player win was the least enjoyable opponent of them all.
---
paper_title: Automatically acquiring domain knowledge for adaptive game AI using evolutionary learning

paper_content:

Game AI is the decision-making process of computer-controlled opponents in computer games. Adaptive game AI can improve the entertainment value of computer games. It allows computer-controlled opponents to automatically fix weaknesses in the game AI and respond to changes in human-player tactics. Dynamic scripting is a recently developed approach for adaptive game AI that learns which tactics (i.e., action sequences) an opponent should select to play effectively against the human player. In previous work, these tactics were manually generated. We introduce AKADS; it uses an evolutionary algorithm to automatically generate such tactics. Our experiments show that it improves dynamic scripting's performance on a real-time strategy (RTS) game. Therefore, we conclude that high-quality domain knowledge (i.e., tactics) can be automatically generated for strong adaptive AI opponents in RTS games. This reduces the time and effort required by game developers to create intelligent game AI, thus freeing them to focus on other important topics (e.g., storytelling, graphics).
---
paper_title: Case Acquisition Strategies for Case-Based Reasoning in Real-Time Strategy Games

paper_content:

Real-time Strategy (RTS) games are complex domains which are a significant challenge to both human and artificial intelligence (AI). For that reason, and although many AI approaches have been proposed for the RTS game AI problem, the AI of all commercial RTS games is scripted and offers a very static behavior subject to exploits. In this paper, we will focus on a case-based reasoning (CBR) approach to this problem, and concentrate on the process of case-acquisition. Specifically, we will describe 7 different techniques to automatically acquire plans by observing human demonstrations and compare their performance when using them in the Darmok 2 system in the context of an RTS game.
---
paper_title: Human-level AI's Killer Application: Interactive Computer Games

paper_content:

Although one of the fundamental goals of AI is to understand and develop intelligent systems that have all the capabilities of humans, there is little active research directly pursuing this goal. We propose that AI for interactive computer games is an emerging application area in which this goal of human-level AI can successfully be pursued. Interactive computer games have increasingly complex and realistic worlds and increasingly complex and intelligent computer-controlled characters. In this article, we further motivate our proposal of using interactive computer games for AI research, review previous research on AI and games, and present the different game genres and the roles that human-level AI could play within these genres. We then describe the research issues and AI techniques that are relevant to each of these roles. Our conclusion is that interactive computer games provide a rich environment for incremental research on human-level AI.
---
paper_title: Strategies for Strategy Game AI

paper_content:

The biggest challenge in computer strategy games is the creation of a fun computer opponent. The hardest element of "fun" is "good". For the Artificial Intelligence to be good, it must do much of what a human player would do: situational & map analysis, resource allocation, and strategy execution. We present here the framework and some of the strategies we have developed at Activision in the development of games such as Dark Reign: The Future Of War, Battlezone, Civilization: Call To Power, and others.
---
paper_title: Building Human-Level AI for Real-Time Strategy Games

paper_content:

Video games are complex simulation environments with many real-world properties that need to be addressed in order to build robust intelligence. In particular, realtime strategy games provide a multi-scale challenge which requires both deliberative and reactive reasoning processes. Experts approach this task by studying a corpus of games, building models for anticipating opponent actions, and practicing within the game environment. We motivate the need for integrating heterogeneous approaches by enumerating a range of competencies involved in gameplay and discuss how they are being implemented in EISBot, a reactive planning agent that we have applied to the task of playing real-time strategy games at the same granularity as humans.
---
paper_title: A Bayesian model for RTS units control applied to StarCraft

paper_content:

In real-time strategy games (RTS), the player must reason about high-level strategy and planning while having effective tactics and even individual units micro-management. Enabling an artificial agent to deal with such a task entails breaking down the complexity of this environment. For that, we propose to control units locally in the Bayesian sensory motor robot fashion, with higher level orders integrated as perceptions. As complete inference encompassing global strategy down to individual unit needs is intractable, we embrace incompleteness through a hierarchical model able to deal with uncertainty. We developed and applied our approach on a StarCraft1 AI.
---
paper_title: A Bayesian Model for Plan Recognition in RTS Games applied to StarCraft

paper_content:

The task of keyhole (unobtrusive) plan recognition is central to adaptive game AI. "Tech trees" or "build trees" are the core of real-time strategy (RTS) game strategic (long term) planning. This paper presents a generic and simple Bayesian model for RTS build tree prediction from noisy observations, which parameters are learned from replays (game logs). This unsupervised machine learning approach involves minimal work for the game developers as it leverage players' data (common in RTS). We applied it to StarCraft1 and showed that it yields high quality and robust predictions, that can feed an adaptive AI.
---
paper_title: The RoboCup Synthetic Agent Challenge 97

paper_content:

RoboCup Challenge offers a set of challenges for intelligent agent researchers using a friendly competition in a dynamic, real-time, multiagent domain. While RoboCup in general envisions longer range challenges over the next few decades, RoboCup Challenge presents three specific challenges for the next two years: (i) learning of individual agents and teams; (ii) multi-agent team planning and plan-execution in service of teamwork; and (iii) opponent modeling. RoboCup Challenge provides a novel opportunity for machine learning, planning, and multi-agent researchers it not only supplies a concrete domain to evaluate their techniques, but also challenges researchers to evolve these techniques to face key constraints fundamental to this domain: real-time, uncertainty, and teamwork.
---
paper_title: Integrating Learning in Interactive Gaming Simulators

paper_content:

Many developers of simulations for computer-generated forces and real-time strategy games seek to incorporate learning or learned behaviors in their systems. Likewise, many researchers seek to evaluate their learning systems in these simulators. However, these integrations require great effort. We describe our initial work on a testbed, named TIELT that we are designing to facilitate these integrations.
---

Title: A Review of Real-Time Strategy Game AI

Section 1: Introduction
Description 1: Introduce the significance of AI in games, particularly in video games, and outline the scope and purpose of the literature review.

Section 2: RTS Games
Description 2: Describe the fundamental characteristics of real-time strategy (RTS) games and explain their significance as a domain for AI research.

Section 3: StarCraft
Description 3: Explain why StarCraft is a central focus for RTS AI research and detail its relevance in the context of AI competitions and research communities.

Section 4: Tactical Decision Making
Description 4: Discuss AI techniques used for tactical and micromanagement decisions in RTS games, focusing on methods and examples from StarCraft.

Section 5: Reinforcement Learning
Description 5: Explain the application of reinforcement learning in tactical decision-making within RTS games, with a focus on specific studies and examples.

Section 6: Game-Tree Search
Description 6: Describe the application of search techniques in RTS games, particularly for tactical decisions, and discuss the challenges and solutions.

Section 7: Monte Carlo Planning
Description 7: Detail the use and potential of Monte Carlo planning in RTS games, including its benefits and challenges.

Section 8: Other Techniques
Description 8: Explore other AI techniques employed for tactical decision making, such as Bayesian models, case-based reasoning, and neuroevolution.

Section 9: Strategic Decision Making
Description 9: Cover AI methods for making long-term strategic decisions in RTS games, discussing planning systems and their applications.

Section 10: Case-Based Planning
Description 10: Explain case-based planning in the context of RTS games and provide examples of its use in AI systems.

Section 11: Hierarchical Planning
Description 11: Describe hierarchical planning techniques and their relevance to strategic decision making in RTS games.

Section 12: Behavior Trees and Goal-Driven Autonomy
Description 12: Discuss the use of behavior trees and goal-driven autonomy in strategic decision making for RTS games, with examples from StarCraft.

Section 13: State Space Planning
Description 13: Explain heuristic state space planning techniques and their applications in RTS AI systems.

Section 14: Evolutionary Algorithms
Description 14: Examine the use of evolutionary algorithms for strategic decision making in RTS games, with examples from StarCraft and other titles.

Section 15: Cognitive Architectures
Description 15: Explore the application of cognitive architectures in modeling human-like reasoning for RTS game AI.

Section 16: Spatial Reasoning
Description 16: Discuss the importance of spatial reasoning in RTS games and the techniques employed to enable it in AI systems.

Section 17: Plan Recognition and Learning
Description 17: Cover AI methods for recognizing and learning strategic-level gameplay, focusing on plan recognition and case-based techniques.

Section 18: Learning by Observation
Description 18: Describe techniques for AI systems to learn gameplay by observing human players, using traces and replays.

Section 19: Learning from Demonstration
Description 19: Discuss the concept of learning from demonstration in RTS AI, where systems learn from annotated examples of desired behavior.

Section 20: Open Research Areas
Description 20: Identify and describe research areas in RTS AI that are currently underexplored or have potential for significant contributions.

Section 21: Game AI in Industry
Description 21: Contrast academic research with industry practices in game AI, discussing the reasons for low adoption of advanced AI techniques in commercial games.

Section 22: Multiscale AI
Description 22: Explain the concept of multiscale AI and the challenges it presents for concurrent and coordinated goal pursuit in RTS games.

Section 23: Cooperation
Description 23: Highlight the importance of cooperative AI in RTS games and the challenges of enabling effective collaboration between AI and human players.

Section 24: Standardized Evaluation
Description 24: Address the need for standardized evaluation methods in RTS AI research to enable fair and consistent comparison of techniques and results.

Section 25: Conclusion
Description 25: Summarize the findings of the review, emphasizing the key techniques discussed, gaps identified, and future directions for RTS AI research.