Reconstructing Retinal Visual Images from 3T fMRI Data Enhanced by
  Unsupervised Learning
Yujian Xiong Wenhui Zhu Zhong-Lin Lu Yalin Wang
The reconstruction of human visual inputs from brain activity, particularly
through functional Magnetic Resonance Imaging (fMRI), holds promising avenues
for unraveling the mechanisms of the human visual system. Despite the
significant strides made by deep learning methods in improving the quality and
interpretability of visual reconstruction, there remains a substantial demand
for high-quality, long-duration, subject-specific 7-Tesla fMRI experiments. The
challenge arises in integrating diverse smaller 3-Tesla datasets or
accommodating new subjects with brief and low-quality fMRI scans. In response
to these constraints, we propose a novel framework that generates enhanced 3T
fMRI data through an unsupervised Generative Adversarial Network (GAN),
leveraging unpaired training across two distinct fMRI datasets in 7T and 3T,
respectively. This approach aims to overcome the limitations of the scarcity of
high-quality 7-Tesla data and the challenges associated with brief and
low-quality scans in 3-Tesla experiments. In this paper, we demonstrate the
reconstruction capabilities of the enhanced 3T fMRI data, highlighting its
proficiency in generating superior input visual images compared to
data-intensive methods trained and tested on a single subject.