The goal of unpaired cross-domain translation is to learn useful mappings between two domains, given unpaired sets of datapoints from these domains. While this formulation is highly underconstrained, recent work has shown that it is possible to learn mappings useful for downstream tasks by encouraging approximate cycle consistency in the mappings between the two domains [Zhu et al., 2017]. In this work, we propose AlignFlow, a framework for unpaired cross-domain translation that ensures exact cycle consistency in the learned mappings. Our framework uses a normalizing flow model to specify a single invertible mapping between the two domains. In contrast to prior works in cycle-consistent translations, we can learn AlignFlow via adversarial training, maximum likelihood estimation, or a hybrid of the two methods. Theoretically, we derive consistency results for AlignFlow which guarantee recovery of desirable mappings under suitable assumptions. Empirically, AlignFlow demonstrates significant improvements over relevant baselines on image-to-image translation and unsupervised domain adaptation tasks on benchmark datasets. Given data from two domains, cross-domain translation refers to the task of learning a mapping from one domain to another, such as translating text across two languages or image colorization. This ability to learn a meaningful alignment between two domains has a broad range of applications across machine learning, including relational learning BID1 , domain adaptation BID2 BID4 , image and video translation for computer vision BID6 , and machine translation for natural language processing BID7 .Broadly , there are two learning paradigms for cross-domain translation: paired and unpaired. In paired cross-domain translation, we assume access to pairs of datapoints across the two domains, e.g., black and white images and their respective colorizations. However, paired data can be expensive to obtain or may not even exist, as in neural style transfer BID8 where the goal is to translate across the works of two artists that typically do not exhibit a direct correspondence.Unpaired cross-domain translation tackles this regime where paired data is not available and learns an alignment between two domains given only unpaired sets of datapoints from the domains. Formally , we seek to learn a joint distribution over two domains, say A and B, given samples only from the marginal distributions over A and B. CycleGAN BID0 , a highly successful approach to this problem, learns a pair of conditional generative models, say G A→B and G B→A , to match the marginal distributions over A and B via an adversarial objective BID9 . The marginal matching constraints alone are insufficient to learn the desired joint distribution, both in theory and practice. To further constrain the problem, an additional desideratum is imposed in the form of cycle-consistency. That is, given any datapoint A = a, the cycle-consistency term in the learning objective prefers mappings G A→B and G B→A such that G B→A (G A→B (a)) ≈ a. Symmetrically, cycle-consistency in the reverse direction implies G A→B (G B→A (b)) ≈ b for all datapoints B = b. Intuitively , this encourages the learning of approximately bijective mappings.While empirically effective, the CycleGAN objective only imposes a soft cycle-consistency penalty and provides no guarantee that G A→B and G B→A are true inverses of each other. A natural question, then, is whether the cycle-consistency objective can be replaced with a single, invertible model G A→B . Drawing inspiration from the literature on invertible generative models (Rezende and BID10 BID11 BID13 , we propose AlignFlow, a learning framework for cross-domain translations which uses normalizing flow models to represent the mappings. In AlignFlow, we compose a pair of invertible flow models G Z→A and G Z→B , to represent the mapping G A→B = G Z→B • G −1 Z→A . Here, Z is a shared latent space between the two domains. Since composition of invertible mappings preserves invertibility , the mapping G A→B is invertible and the reverse mapping from B → A is simply given as G B→A = G −1 A→B . Hence, AlignFlow guarantees exact cycle-consistency by design and simplifies the standard CycleGAN learning objective by learning a single, invertible mapping.Furthermore, AlignFlow provides flexibility in specifying the training objective. In addition to adversarial training, we can also specify a prior distribution over the latent variables Z and train the two component models G Z→B and G Z→A via maximum likelihood estimation (MLE). MLE is statistically efficient, exhibits stable training dynamics , and can have a regularizing effect when used in conjunction with adversarial training of invertible generative models BID14 . In this work, we presented AlignFlow, a learning framework for cross-domain translations based on normalizing flow models. The use of normalizing flow models is an attractive choice for several reasons we highlight: it guarantees exact cycle-consistency via a single cross-domain mapping, learns a shared latent space across two domains, and permits a flexible training objective which is a hybrid of terms corresponding to adversarial training and exact maximum likelihood estimation. Theoretically, we derived conditions under which the AlignFlow model learns marginals that are consistent with the underlying data distributions. Finally, our empirical evaluation demonstrated significant gains on the tasks of image-to-image translation and unsupervised domain adaptation, along with an increase in inference capabilities due to the use of invertible models, e.g., paired interpolations in the latent space for two domains.In the future, we would like to consider extensions of AlignFlow to learning stochastic, multimodal mappings BID37 and translations across more than two domains BID38 . In spite of strong empirical results in domain alignments in the last few years, a well-established theory explaining such results is lacking. With a handle on model likelihoods and exact invertibility for inference, we are optimistic that AlignFlow can potentially aid the development of such a theory and characterize structure that leads to provably identifiable recovery of cross-domain mappings. Exploring the latent space of AlignFlow from a manifold learning perspective to domain alignment BID44 is also an interesting direction for future research.
We propose a learning framework for cross-domain translations which is exactly cycle-consistent and can be learned via adversarial training, maximum likelihood estimation, or a hybrid. Proposes AlignFlow, an efficient way of implementing cycle consistency principle using invertible flows. Flow models for unpaired image to image translation