Unidirectional brain-computer interface: Artificial neural network
  encoding natural images to fMRI response in the visual cortex
Ruixing Liang Xiangyu Zhang Qiong Li Lai Wei Hexin Liu Avisha Kumar Kelley M. Kempski Leadingham Joshua Punnoose Leibny Paola Garcia Amir Manbachi
While significant advancements in artificial intelligence (AI) have catalyzed
progress across various domains, its full potential in understanding visual
perception remains underexplored. We propose an artificial neural network
dubbed VISION, an acronym for "Visual Interface System for Imaging Output of
Neural activity," to mimic the human brain and show how it can foster
neuroscientific inquiries. Using visual and contextual inputs, this multimodal
model predicts the brain's functional magnetic resonance imaging (fMRI) scan
response to natural images. VISION successfully predicts human hemodynamic
responses as fMRI voxel values to visual inputs with an accuracy exceeding
state-of-the-art performance by 45%. We further probe the trained networks to
reveal representational biases in different visual areas, generate
experimentally testable hypotheses, and formulate an interpretable metric to
associate these hypotheses with cortical functions. With both a model and
evaluation metric, the cost and time burdens associated with designing and
implementing functional analysis on the visual cortex could be reduced. Our
work suggests that the evolution of computational models may shed light on our
fundamental understanding of the visual cortex and provide a viable approach
toward reliable brain-machine interfaces.