The development of high-dimensional generative models has recently gained a great surge of interest with the introduction of variational auto-encoders and generative adversarial neural networks. Different variants have been proposed where the underlying latent space is structured, for example, based on attributes describing the data to generate. We focus on a particular problem where one aims at generating samples corresponding to a number of objects under various views. We assume that the distribution of the data is driven by two independent latent factors: the content, which represents the intrinsic features of an object, and the view, which stands for the settings of a particular observation of that object. Therefore, we propose a generative model and a conditional variant built on such a disentangled latent space. This approach allows us to generate realistic samples corresponding to various objects in a high variety of views. Unlike many multi-view approaches, our model doesn't need any supervision on the views but only on the content. Compared to other conditional generation approaches that are mostly based on binary or categorical attributes, we make no such assumption about the factors of variations. Our model can be used on problems with a huge, potentially infinite, number of categories. We experiment it on four images datasets on which we demonstrate the effectiveness of the model and its ability to generalize. Multi-view learning aims at developing models that are trained over datasets composed of multiple views over different objects. The problem of handling multi-view inputs has mainly been studied from the predictive point of view where one wants, for example, to learn a model able to predict/classify over multiple views of the same object BID25 ; BID23 ). For example, using deep learning approaches, different strategies have been explored to aggregate multiple views but a common general idea is based on the (early or late) fusion of the different views at a particular level of a deep architecture. Few other studies have proposed to predict missing views from one or multiple remaining views as in Arsalan BID0 .Recent research has focused on identifying factors of variations from multiview datasets. The underlying idea is to consider that a particular data sample may be thought as the mix of a content information (e.g. related to its class label like a given person in a face dataset) and of a side information, the view, which accounts for factors of variability (e.g. exposure, viewpoint, with/wo glasses...). All samples of a given class share the same content information while they differ on the view information. A number of approaches have been proposed to disentangle the content from the view, also referred as the style in some papers BID19 ; BID4 ). For instance, different models have been built to extract from a single photo of any object both the characteristics of the object but also the camera position. Once such a disentanglement is learned, one may build various applications like predicting how an object looks like under different viewpoints BID19 ; BID28 ). In the generative domain, models with a disentangled latent space BID18 ; BID5 ) have been recently proposed with applications to image editing, where one wants to modify an input sample by preserving its content while changing its view BID13 ; BID10 ) (see Section 6).Yet most existing controlled generative approaches have two strong limitations: (i) they usually consider discrete views that are characterized by a domain or a set of discrete (binary/categorical) attributes (e.g. face with/wo glasses, the color of the hair, etc.) and could not easily scale to a large number of attributes or to continuous views. (ii) most models are trained using view supervision (e.g. the view attributes), which of course greatly helps learning such model, yet prevents their use on many datasets where this information is not available. Recently, some attempts have been made to learn such models without any supervision ; BID7 ), but they cannot disentangle high level concepts as only simple features can be reliably captured without any guidance.In this paper, we are interested in learning generative models that build and make use of a disentangled latent space where the content and the view are encoded separately. We propose to take an original approach by learning such models from multi-view datasets, where (i) samples are labeled based on their content, and without any view information, and (ii) where the generated views are not restricted to be one view in a subset of possible views. Following with our same example above, it means first, learning from a face dataset including multiple photos of multiple persons taken in various conditions related to exposure, viewpoint etc. and second, being able to generate an infinity of views of an imaginary person (or the same views of an infinity of imaginary persons) -see FIG0 . This contrast with most current approaches that use information about the style, and cannot generate multiple possible outputs.More precisely, we propose two models to tackle this particularly difficult setting: a generative model (GMV -Generative Multi-view Model) that generates objects under various views (multiview generation), and a conditional extension (C-GMV) of this model that generates a large number of views of any input object (conditional multi-view generation). These two models are based on the adversarial training schema of Generative Adversarial Networks (GAN) proposed in BID6 ). The simple but strong idea is to focus on distributions over pairs of examples (e.g. images representing a same object in different views) rather than distribution on single examples as we will explain later.Our contributions are the following: (i) We propose a new generative model able to generate data with various content and high view diversity using a supervision on the content information only.(ii) We extend the model to a conditional model that allows generating new views over any input sample. (iii) We report experimental results on four different images datasets that show the ability of our models to generate realistic samples and to capture (and generate with) the diversity of views.The paper is organized as follows. We first remind useful background on GANs and their conditional variant (Section 2). Then we successively detail our proposal for a generative multiview model (Section 3.2) and for its conditional extension (Section 4). Finally, we report experimental results on the various generative tasks allowed by our models in Section 5.2. We have proposed a generative model operating on a disentangled latent space which may be learned from multiview data without any view supervision, allowing its application to many multiview dataset. Our model allows generating realistic data with a rich view diversity. We also proposed a conditional version of this model which allows generating new views of an input image which may again be learned without view supervision. Our experimental results show the quality of the produced outputs, and the ability of the model to capture content and view factors. They also illustrate the ability of GMV and CGMV to capture the diversity over the CelebA dataset, and to generate more realistic samples than baseline models. In a near future, we plan to investigate the use of such an approach for data augmentation. Indeed, when only a few training data are available, one elegant solution for learning a model could be to generate new views of the existing data in order to increase the size of the training set. This solution will be explored in both semi-supervised and one-shot/few-shot learning settings. APPENDIX Figure 9 : Additional results on GMV: All images in a row have been generated with the same content vector, and all images in a column have been generated with the same view vector TAB7 : Distribution of the different attributes over generated samples. For example, 3.8 % of the samples generated by the GMV model exhibit the "Eyeglasses" attribute. 23
We describe a novel multi-view generative model that can generate multiple views of the same object, or multiple objects in the same view with no need of label on views. This paper presents a GAN-based method for image generation that attempts to separate latent variables describing image content from those describing properties of view. This paper proposes a GAN architecture that aims at decomposing the underlying distribution of a particular class into "content" and "view". Proposes a new generative model based on the Generative Adversarial Network (GAN) that disentangles the content and the view of objects without view supervision and extends GMV into a conditional generative model that takes an input image and generates different views of the object in the input image. 