I am re-writing a GAN (cGAN) into a Wasserstein GAN.
My original generator is trained both with adversarial loss from the discriminator but also with L1-loss between the generated fake and the target (I am also experimenting with VGG-loss and L2-loss).
My Wasserstein GAN works as expected when only using an adversarial loss but since it uses Wasserstein distance, the critic outputs losses which can range between 1e-5 to 1e6, shifting throughout the training. Combining other loss functions which generally have ranges from 0-1 feels next to impossible even with scaling factors.
I have therefore currently added a Tanh activation function for my discriminator, but I wonder if this is the way to go. It is not the true Wasserstein distance but the loss is “standardized”.
If you know of any project using WGAN loss in combination with other loss functions, please let me know.