IndustReal: A Dataset for Procedure Step Recognition Handling Execution
  Errors in Egocentric Videos in an Industrial-Like Setting
Tim J. Schoonbeek Tim Houben Hans Onvlee Peter H. N. de With Fons van der Sommen
Although action recognition for procedural tasks has received notable
attention, it has a fundamental flaw in that no measure of success for actions
is provided. This limits the applicability of such systems especially within
the industrial domain, since the outcome of procedural actions is often
significantly more important than the mere execution. To address this
limitation, we define the novel task of procedure step recognition (PSR),
focusing on recognizing the correct completion and order of procedural steps.
Alongside the new task, we also present the multi-modal IndustReal dataset.
Unlike currently available datasets, IndustReal contains procedural errors
(such as omissions) as well as execution errors. A significant part of these
errors are exclusively present in the validation and test sets, making
IndustReal suitable to evaluate robustness of algorithms to new, unseen
mistakes. Additionally, to encourage reproducibility and allow for scalable
approaches trained on synthetic data, the 3D models of all parts are publicly
available. Annotations and benchmark performance are provided for action
recognition and assembly state detection, as well as the new PSR task.
IndustReal, along with the code and model weights, is available at:
https://github.com/TimSchoonbeek/IndustReal .