Metrics Matter in Surgical Phase Recognition
Isabel Funke Dominik Rivoir Stefanie Speidel
Surgical phase recognition is a basic component for different context-aware
applications in computer- and robot-assisted surgery. In recent years, several
methods for automatic surgical phase recognition have been proposed, showing
promising results. However, a meaningful comparison of these methods is
difficult due to differences in the evaluation process and incomplete reporting
of evaluation details. In particular, the details of metric computation can
vary widely between different studies. To raise awareness of potential
inconsistencies, this paper summarizes common deviations in the evaluation of
phase recognition algorithms on the Cholec80 benchmark. In addition, a
structured overview of previously reported evaluation results on Cholec80 is
provided, taking known differences in evaluation protocols into account.
Greater attention to evaluation details could help achieve more consistent and
comparable results on the surgical phase recognition task, leading to more
reliable conclusions about advancements in the field and, finally, translation
into clinical practice.