Cortex Inspired Learning to Recover Damaged Signal Modality with ReD-SOM
  Model
Artem Muliukov Laurent Rodriguez Benoit Miramond
Recent progress in the fields of AI and cognitive sciences opens up new
challenges that were previously inaccessible to study. One of such modern tasks
is recovering lost data of one modality by using the data from another one. A
similar effect (called the McGurk Effect) has been found in the functioning of
the human brain. Observing this effect, one modality of information interferes
with another, changing its perception. In this paper, we propose a way to
simulate such an effect and use it to reconstruct lost data modalities by
combining Variational Auto-Encoders, Self-Organizing Maps, and Hebb connections
in a unified ReD-SOM (Reentering Deep Self-organizing Map) model. We are
inspired by human's capability to use different zones of the brain in different
modalities, in case of having a lack of information in one of the modalities.
This new approach not only improves the analysis of ambiguous data but also
restores the intended signal! The results obtained on the multimodal dataset
demonstrate an increase of quality of the signal reconstruction. The effect is
remarkable both visually and quantitatively, specifically in presence of a
significant degree of signal's distortion.