Self-Supervised Interpretable Sensorimotor Learning via Latent
  Functional Modularity
Hyunki Seong David Hyunchul Shim
We introduce MoNet, a novel method that combines end-to-end learning with
modular network architectures for self-supervised and interpretable
sensorimotor learning. MoNet is composed of three functionally distinct neural
modules: Perception, Planning, and Control. Leveraging its inherent modularity
through a cognition-guided contrastive loss function, MoNet efficiently learns
task-specific decision-making processes in latent space, without requiring
task-level supervision. Moreover, our method incorporates an online post-hoc
explainability approach, which enhances the interpretability of the end-to-end
inferences without a trade-off in sensorimotor performance. In real-world
indoor environments, MoNet demonstrates effective visual autonomous navigation,
surpassing baseline models by 11% to 47% in task specificity analysis. We
further delve into the interpretability of our network through the post-hoc
analysis of perceptual saliency maps and latent decision vectors. This offers
insights into the incorporation of explainable artificial intelligence within
the realm of robotic learning, encompassing both perceptual and behavioral
perspectives.