Improving Compositional Generalization Using Iterated Learning and
  Simplicial Embeddings
Yi Ren Samuel Lavoie Mikhail Galkin Danica J. Sutherland Aaron Courville
Compositional generalization, the ability of an agent to generalize to unseen
combinations of latent factors, is easy for humans but hard for deep neural
networks. A line of research in cognitive science has hypothesized a process,
``iterated learning,'' to help explain how human language developed this
ability; the theory rests on simultaneous pressures towards compressibility
(when an ignorant agent learns from an informed one) and expressivity (when it
uses the representation for downstream tasks). Inspired by this process, we
propose to improve the compositional generalization of deep networks by using
iterated learning on models with simplicial embeddings, which can approximately
discretize representations. This approach is further motivated by an analysis
of compositionality based on Kolmogorov complexity. We show that this
combination of changes improves compositional generalization over other
approaches, demonstrating these improvements both on vision tasks with
well-understood latent factors and on real molecular graph prediction tasks
where the latent structure is unknown.