Good representations facilitate transfer learning and few-shot learning. Motivated by theories of language and communication that explain why communities with large number of speakers have, on average, simpler languages with more regularity, we cast the representation learning problem in terms of learning to communicate. Our starting  point sees traditional autoencoders as  a single encoder with a fixed decoder partner that must learn to communicate. Generalizing from there, we introduce community-based autoencoders in which multiple encoders and decoders collectively learn representations by being randomly paired up on successive training iterations. Our experiments show that increasing community sizes reduce idiosyncrasies in the learned codes, resulting in more invariant representations with increased reusability and structure. The importance of representation learning lies in two dimensions. First and foremost, representation learning is a crucial building block of a neural model being trained to perform well on a particular task, i.e., representation learning that induces the "right" manifold structure can lead to models that generalize better, and even extrapolate. Another property of representation learning, and arguably the most important one, is that it can facilitate transfer of knowledge across different tasks , essential for transfer learning and few-shot learning among others BID0 . With this second point in mind, we can define good representations as the ones that are reusable, induce the abstractions that capture the "right" type of invariances and can allow for generalizing very quickly to a new task. Significant efforts have been made to learn representations with these properties; one frequently explored direction involves trying to learn disentangled representations BID12 BID6 BID5 BID17 ), while others focus on general regularization methods BID15 BID18 . In this work, we take a different approach to representation learning, inspired by successful abstraction mechanisms found in nature, to wit human language and communication.Human languages and their properties are greatly affected by the size of their linguistic community BID11 BID19 BID16 BID9 . Small linguistic communities of speakers tend to develop more structurally complex languages, while larger communities give rise to simpler languages (Dryer & Haspelmath, 2013) . Moreover, we even observe structural simplification as the effective number of speakers grows, as in the example of English language BID10 . A similar relation between number of speakers and linguistic complexity can also be observed during linguistic communication. Speakers, aiming at maximizing communication effectiveness, adapt and shape their conceptualizations to account for the needs of their specific partners, a phenomenon often termed in dialogue research as partner specificity BID2 ). As such, speakers form conceptual pacts with their listeners BID1 , and in some extreme cases, these pacts are so ad-hoc and idiosyncratic that overhearers cannot follow the discussion BID13 )! But how are all these linguistic situations related to representation learning? We start by drawing an analogy between language and representations induced by the traditional and extensively used framework of autonencoders (AE). In the traditional AE set-up, there is a fixed pair of a single encoder and a single decoder that are trained to maximize a reconstruction loss. However, encoders and decoders co-adapt to one another, yielding idiosyncratic representations. The encoders spend repre-sentational capacity modeling any kind of information about the data that could allow the decoder to successfully reconstruct the input; as long as the encoder and the decoder agree on a representation protocol, this information need not be abstract or systematic. This has a negative impact on the reusability of the representations, something that afterall is a key objective of representation learning. Evidence of this co-adaption is found in the above-mentioned efforts targeting generalization. The human language analogy of the traditional AE setup would be an extreme version of the conceptual pact experiments from BID13 , where two people never communicate with anybody else: the resulting language would be very hard to understand for any outsider.In this work we test whether removing this co-adaptation between encoders and decoders can yield better generalization, much as dropout removes co-adaptation between activations and thereby yields better generalization in general neural networks. We hypothesize that machines that communicate not with a specific partner but with a multitude of partners, will shape the representations they communicate to be simpler in nature. We introduce a simple framework that we term communitybased autoencoders (CbAEs), in which there exist multiple encoders and decoders, and at every training iteration one of each is randomly sampled to perform a traditional autoencoder (AE) training step. Given that the identity of the decoder is not revealed to the encoder during the encoding of the input, the induced representation should be such that all decoders can use it to successfully reconstruct the input. A similar argument holds for the decoder, which at reconstruction time does not have access to the identity of the encoder. We conjecture that this process will reduce the level of idiosyncrasy, resulting in representations that are invariant to the diverse encoders and decoders.We apply CbAEs to two standard computer vision datasets and probe their representations along two axes; their reusability and their structural properties. We find that in contrast to representations induced within a traditional AE framework 1) the CbAE-induced representations encode abstract information that is more easily extracted and re-used for a different task 2) CbAE representations provide an interface that is easier to learn for new users 3) and the underlying topology of the CbAE representations is more aligned to human perceptual data that are disentangled and structured. We have presented Community-based AutoEncoders, a framework in which multiple encoders and decoders collectively learn representations by being randomly paired up on successive training iterations, encouraging a similar lack of co-adaptation that dropout does at the activation level, at model level. Analogous to the structural simplicity found in languages with many speakers, we find that the latent representations induced in this scheme are easier to use and more structured. This result is philosophically interesting in that it suggests that the community size effects found in human languages are general properties of any representation learning system, opening avenues to potential synergies between representation learning linguistics.The price for obtaining these representations is the increase in computational requirements, which is linear in the community size. Due to the reusability of the resulting representations, this cost may be amortized over a number of applications trained on top of the encoders. Furthermore, the community-based training procedure is highly parallelizable, since only the latents and corresponding backpropagated errors need to be sent between the encoders and decoders.
Motivated by theories of language and communication, we introduce community-based autoencoders, in which multiple encoders and decoders collectively learn structured and reusable representations. The authors tackle the problem of representation learning, aim to build reusable and structured represenation, argue co-adaptation between encoder and decoder in traditional AE yields poor representation, and introduce community based auto-encoders. The paper presents a community based autoencoder framework to address co-adaptation of encoders and decoders and aims at constructing better representations.