'Good' Discriminator -> no Learning for Generator

When training GANs, a ‘bad’ Discriminator D is the one that correctly classifies real and fake, generated by Generator G data, yielding small derivatives. Therefore, a D that outputs values ~0.5 reagrdless of the input is supposed to be ‘good’, yielding large derivatives.

In one of the models I’m training an error D(1-G(z)) quickly reduces to ~0.69 (i.e. -log(0.5)), which should be good, because we are fooling D, but G actually doesn’t learn anything from it quickly converging to outputting all 0s (for example).

  1. What is the reason for this convergence?
  2. Why doesn’t G learn from the errors?

Therefore, a D that outputs values ~0.5 regardless of the input is supposed to be ‘good’

This isn’t necessarily a flag that your generator and discriminators are amazing. If your Discriminator refuses to adjust itself, then your discriminator fell into a local optimum that it’s unable to escape from.
There’s can be many reasons for this to occur, GANs are notoriously tricky to train. I believe this might be affecting both behaviors you’re noticing.

I would suggest exploring the learning rates you have set, different batch sizes, loss functions, and/or optimizers. I would also suggest logging your losses (both gen and discrim) over time to help provide more insight. You shouldn’t see one model getting better than the other, it should be a consistent switch in the “winner” or a steady chase after the winner. If you see one model slowly winning over time, the training is diverging.

I’m not sure I understand this bit. By ‘winner’ do you mean G or D?

Either one really.

When I say consistent switch, I mean, the generator wins, then loses some iterations later, wins some iterations later, and so on.

When I say a steady chase, it means either G or D is contentiously winning BUT isn’t leaving the other behind.

From experience, I normally see the consistent switch behavior. Depending on your task, it’s hard to determine the performance of your GAN (other than if a generated sample looks realistic)