RT-GAN: Recurrent Temporal GAN for Adding Lightweight Temporal
  Consistency to Frame-Based Domain Translation Approaches
Shawn Mathew Saad Nadeem Alvin C. Goh Arie Kaufman
While developing new unsupervised domain translation methods for endoscopy
videos, it is typical to start with approaches that initially work for
individual frames without temporal consistency. Once an individual-frame model
has been finalized, additional contiguous frames are added with a modified deep
learning architecture to train a new model for temporal consistency. This
transition to temporally-consistent deep learning models, however, requires
significantly more computational and memory resources for training. In this
paper, we present a lightweight solution with a tunable temporal parameter,
RT-GAN (Recurrent Temporal GAN), for adding temporal consistency to individual
frame-based approaches that reduces training requirements by a factor of 5. We
demonstrate the effectiveness of our approach on two challenging use cases in
colonoscopy: haustral fold segmentation (indicative of missed surface) and
realistic colonoscopy simulator video generation. The datasets, accompanying
code, and pretrained models will be made available at
\url{https://github.com/nadeemlab/CEP}.