Multi-Branch Generative Models for Multichannel Imaging with an
  Application to PET/CT Joint Reconstruction
Noel Jeffrey Pinton Alexandre Bousse Catherine Cheze-Le-Rest Dimitris Visvikis
This paper presents a proof-of-concept approach for learned synergistic
reconstruction of medical images using multi-branch generative models.
Leveraging variational autoencoders (VAEs) and generative adversarial networks
(GANs), our models learn from pairs of images simultaneously, enabling
effective denoising and reconstruction. Synergistic image reconstruction is
achieved by incorporating the trained models in a regularizer that evaluates
the distance between the images and the model, in a similar fashion to
multichannel dictionary learning (DiL). We demonstrate the efficacy of our
approach on both Modified National Institute of Standards and Technology
(MNIST) and positron emission tomography (PET)/computed tomography (CT)
datasets, showcasing improved image quality and information sharing between
modalities. Despite challenges such as patch decomposition and model
limitations, our results underscore the potential of generative models for
enhancing medical imaging reconstruction.