Action Recognition in Video Recordings from Gynecologic Laparoscopy
Sahar Nasirihaghighi Negin Ghamsarian Daniela Stefanics Klaus Schoeffmann Heinrich Husslein
Action recognition is a prerequisite for many applications in laparoscopic
video analysis including but not limited to surgical training, operation room
planning, follow-up surgery preparation, post-operative surgical assessment,
and surgical outcome estimation. However, automatic action recognition in
laparoscopic surgeries involves numerous challenges such as (I) cross-action
and intra-action duration variation, (II) relevant content distortion due to
smoke, blood accumulation, fast camera motions, organ movements, object
occlusion, and (III) surgical scene variations due to different illuminations
and viewpoints. Besides, action annotations in laparoscopy surgeries are
limited and expensive due to requiring expert knowledge. In this study, we
design and evaluate a CNN-RNN architecture as well as a customized
training-inference framework to deal with the mentioned challenges in
laparoscopic surgery action recognition. Using stacked recurrent layers, our
proposed network takes advantage of inter-frame dependencies to negate the
negative effect of content distortion and variation in action recognition.
Furthermore, our proposed frame sampling strategy effectively manages the
duration variations in surgical actions to enable action recognition with high
temporal resolution. Our extensive experiments confirm the superiority of our
proposed method in action recognition compared to static CNNs.