Self-Supervised Video Desmoking for Laparoscopic Surgery
Renlong Wu Zhilu Zhang Shuohao Zhang Longfei Gou Haobin Chen Lei Zhang Hao Chen Wangmeng Zuo
Due to the difficulty of collecting real paired data, most existing desmoking
methods train the models by synthesizing smoke, generalizing poorly to real
surgical scenarios. Although a few works have explored single-image real-world
desmoking in unpaired learning manners, they still encounter challenges in
handling dense smoke. In this work, we address these issues together by
introducing the self-supervised surgery video desmoking (SelfSVD). On the one
hand, we observe that the frame captured before the activation of high-energy
devices is generally clear (named pre-smoke frame, PS frame), thus it can serve
as supervision for other smoky frames, making real-world self-supervised video
desmoking practically feasible. On the other hand, in order to enhance the
desmoking performance, we further feed the valuable information from PS frame
into models, where a masking strategy and a regularization term are presented
to avoid trivial solutions. In addition, we construct a real surgery video
dataset for desmoking, which covers a variety of smoky scenes. Extensive
experiments on the dataset show that our SelfSVD can remove smoke more
effectively and efficiently while recovering more photo-realistic details than
the state-of-the-art methods. The dataset, codes, and pre-trained models are
available at \url{https://github.com/ZcsrenlongZ/SelfSVD}.