Joint one-sided synthetic unpaired image translation and segmentation
  for colorectal cancer prevention
Enric Moreu Eric Arazo Kevin McGuinness Noel E. O'Connor
Deep learning has shown excellent performance in analysing medical images.
However, datasets are difficult to obtain due privacy issues, standardization
problems, and lack of annotations. We address these problems by producing
realistic synthetic images using a combination of 3D technologies and
generative adversarial networks. We propose CUT-seg, a joint training where a
segmentation model and a generative model are jointly trained to produce
realistic images while learning to segment polyps. We take advantage of recent
one-sided translation models because they use significantly less memory,
allowing us to add a segmentation model in the training loop. CUT-seg performs
better, is computationally less expensive, and requires less real images than
other memory-intensive image translation approaches that require two stage
training. Promising results are achieved on five real polyp segmentation
datasets using only one real image and zero real annotations. As a part of this
study we release Synth-Colon, an entirely synthetic dataset that includes 20000
realistic colon images and additional details about depth and 3D geometry:
https://enric1994.github.io/synth-colon