Learning disentangling representations of the independent factors of variations that explain the data in an unsupervised setting is still a major challenge. In the following paper we address the task of disentanglement and introduce a new state-of-the-art approach called Non-synergistic variational Autoencoder (Non-Syn VAE). Our model draws inspiration from population coding, where the notion of synergy arises when we describe the encoded information by neurons in the form of responses from the stimuli. If those responses convey more information together than separate as independent sources of encoding information, they are acting synergetically. By penalizing the synergistic mutual information within the latents we encourage information independence and by doing that disentangle the latent factors. Notably, our approach could be added to the VAE framework easily, where the new ELBO function is still a lower bound on the log likelihood. In addition, we qualitatively compare our model with Factor VAE and show that this one implicitly minimises the synergy of the latents. Our world is hierarchical and compositional, humans can generalise better since we use primitive concepts that allow us to create complex representations BID11 ). Towards the creation of truly intelligent systems, they should learn in a similar way resulting in an increase of their performance since they would capture the underlying factors of variation of the data BID2 ; ; ). In addition, good representations improve the performance for tasks involving transfer learning and multi-task learning; since it will capture the explanatory factors.According to BID18 , a compositional representation should create new elements from the combination of primitive concepts resulting in a infinite number of new representations. For example if our model is trained with images of white wall and then is presented a boy with a white shirt, it should identify the color white as a primitive element. Intuitively, our model will be able to construct different and multiple representations from the primitives.Furthermore, a disentangled representation has been interpreted in different ways, for instance BID2 define it as one where single latent variables are sensitive to changes in generative factors, while being invariant to changes in other factors. In addition, we agree with BID12 , which mentions that a disentangle representation should be factorised and interpretable. Intuitevely, the model could learn generative factors such as position, scale or colour; if it is disentangle it should be able to traverse along the position variable without changing the scale or the colour. It's worth noting that disentangled representations have been useful for a variety of downstream tasks such as domain adaptation by training a Reinforcement Learning agent that uses a disentangled representation of its environment BID13 ; or for learning disentangled primitives grounded in the visual domain discovered in an unsupervised manner BID14 .
Minimising the synergistic mutual information within the latents and the data for the task of disentanglement using the VAE framework. Proposes a new objective function for learning dientangled representations in a variational framework by minimizing the synergy of the information provided. The authors aim at training a VAE that has disentangled latent representations in a "synergistically" maximal way.  This paper proposes a new approach to enforcing disentanglement in VAEs using a term that penalizes the synergistic mutual information between the latent variables.