Why is the discriminator trained more than the generator in GAN training?

In the original GAN paper, in section 4, the authors have trained the generator once for every k times of the discriminator.
Why does the discriminator require more training than the generator?

You can think of the discriminator as a teacher. Do you want the teacher more or less experienced than the student?

Hence, the discriminator needs to stay ahead of the curve. Additionally, once the discriminator has a high rate of being fooled, such that the generator produces images indistinguishable from real images, you’ll know training is finished.

1 Like

Thank you for your answer. That was a very clear and simple analogy.
Also, when you talk about the training being finished, do you mean perceptually verifying the produced images to see how good they are or is there any particular trend of values we need to observe for the adversarial loss?

You can decide how long you wish to train, but as you continue training, you’ll see diminishing returns. What I mean is the discriminator, being more sufficiently trained than the generator, will give you a way to gauge with some certainty how well the generator is doing and where you may want to stop training.

Suppose you had been training your GAN several weeks. And you establish a save and sample every 1,000 batches. And you have a batch size of 16. So a total of 16,000 samples per epoch. And let’s suppose the discriminator guessed 15,900 of those samples were real. So 99.375% generator accuracy. Keeping in mind, the discriminator is only useful during training and later disgarded, once the generator is sufficiently producing realistic images.

But you might decide to keep training a few more weeks in hopes of getting another 0.1-0.2% improvement.

1 Like