Controlling the Latent Space of GANs through Reinforcement Learning: A
  Case Study on Task-based Image-to-Image Translation
Mahyar Abbasian Taha Rajabzadeh Ahmadreza Moradipari Seyed Amir Hossein Aqajari Hongsheng Lu Amir Rahmani
Generative Adversarial Networks (GAN) have emerged as a formidable AI tool to
generate realistic outputs based on training datasets. However, the challenge
of exerting control over the generation process of GANs remains a significant
hurdle. In this paper, we propose a novel methodology to address this issue by
integrating a reinforcement learning (RL) agent with a latent-space GAN
(l-GAN), thereby facilitating the generation of desired outputs. More
specifically, we have developed an actor-critic RL agent with a meticulously
designed reward policy, enabling it to acquire proficiency in navigating the
latent space of the l-GAN and generating outputs based on specified tasks. To
substantiate the efficacy of our approach, we have conducted a series of
experiments employing the MNIST dataset, including arithmetic addition as an
illustrative task. The outcomes of these experiments serve to validate our
methodology. Our pioneering integration of an RL agent with a GAN model
represents a novel advancement, holding great potential for enhancing
generative networks in the future.