♻️ CycleGAN

CycleGAN modifies the 🖼️ Generative Adversarial Network game to facilitate domain transfer, pairing samples from one distribution to samples from another. To do this, we learn two generative models and ; the former maps from distribution to distribution , and the latter goes backwards. We also train two discriminators and .

The key idea is that if we go from to , we should be able to go back from to . This defines our cycle consistency loss

which we add onto the standard GAN losses for and .

StarGAN §

If we want to go between multiple domains, it would be inefficient to train so many generators and discriminators based on the CycleGAN structure. StarGAN addresses this limitation by training a single generator that can translate between any two domains.

To do this, our generator takes in both the source image and the target main label to generate output . Our discriminator will predict and , the former for discriminating between real and fake images and the latter to predict which domain belongs to.

Our overall loss then consists of three parts: adversarial, domain classification (split into real and fake examples), and reconstruction. 1.

The discriminator is trained to minimize a combination of negative and , and our generator is trained to minimize a combination of , , and .