Proving the Potential of Skeleton Based Action Recognition to Automate
  the Analysis of Manual Processes
Marlin Berger Frederik Cloppenburg Jens Eufinger Thomas Gries
In manufacturing sectors such as textiles and electronics, manual processes
are a fundamental part of production. The analysis and monitoring of the
processes is necessary for efficient production design. Traditional methods for
analyzing manual processes are complex, expensive, and inflexible. Compared to
established approaches such as Methods-Time-Measurement (MTM), machine learning
(ML) methods promise: Higher flexibility, self-sufficient & permanent use,
lower costs. In this work, based on a video stream, the current motion class in
a manual assembly process is detected. With information on the current motion,
Key-Performance-Indicators (KPIs) can be derived easily. A skeleton-based
action recognition approach is taken, as this field recently shows major
success in machine vision tasks. For skeleton-based action recognition in
manual assembly, no sufficient pre-work could be found. Therefore, a ML
pipeline is developed, to enable extensive research on different (pre-)
processing methods and neural nets. Suitable well generalizing approaches are
found, proving the potential of ML to enhance analyzation of manual processes.
Models detect the current motion, performed by an operator in manual assembly,
but the results can be transferred to all kinds of manual processes.