CNN-based Game State Detection for a Foosball Table
David Hagens Jan Knaup Elke Hergenröther Andreas Weinmann
The automation of games using Deep Reinforcement Learning Strategies (DRL) is
a well-known challenge in AI research. While for feature extraction in a video
game typically the whole image is used, this is hardly practical for many real
world games. Instead, using a smaller game state reducing the dimension of the
parameter space to include essential parameters only seems to be a promising
approach. In the game of Foosball, a compact and comprehensive game state
description consists of the positional shifts and rotations of the figures and
the position of the ball over time. In particular, velocities and accelerations
can be derived from consecutive time samples of the game state. In this paper,
a figure detection system to determine the game state in Foosball is presented.
We capture a dataset containing the rotations of the rods which were measured
using accelerometers and the positional shifts were derived using traditional
Computer Vision techniques (in a laboratory setting). This dataset is utilized
to train Convolutional Neural Network (CNN) based end-to-end regression models
to predict the rotations and shifts of each rod. We present an evaluation of
our system using different state-of-the-art CNNs as base architectures for the
regression model. We show that our system is able to predict the game state
with high accuracy. By providing data for both black and white teams, the
presented system is intended to provide the required data for future
developments of Imitation Learning techniques w.r.t. to observing human
players.