When training GANs, a ‘bad’ Discriminator D is the one that correctly classifies real and fake, generated by Generator G data, yielding small derivatives. Therefore, a D that outputs values ~0.5 reagrdless of the input is supposed to be ‘good’, yielding large derivatives.
In one of the models I’m training an error D(1-G(z)) quickly reduces to ~0.69 (i.e. -log(0.5)), which should be good, because we are fooling D, but G actually doesn’t learn anything from it quickly converging to outputting all 0s (for example).
Therefore, a D that outputs values ~0.5 regardless of the input is supposed to be ‘good’
This isn’t necessarily a flag that your generator and discriminators are amazing. If your Discriminator refuses to adjust itself, then your discriminator fell into a local optimum that it’s unable to escape from.
There’s can be many reasons for this to occur, GANs are notoriously tricky to train. I believe this might be affecting both behaviors you’re noticing.
I would suggest exploring the learning rates you have set, different batch sizes, loss functions, and/or optimizers. I would also suggest logging your losses (both gen and discrim) over time to help provide more insight. You shouldn’t see one model getting better than the other, it should be a consistent switch in the “winner” or a steady chase after the winner. If you see one model slowly winning over time, the training is diverging.
When I say consistent switch, I mean, the generator wins, then loses some iterations later, wins some iterations later, and so on.
When I say a steady chase, it means either G or D is contentiously winning BUT isn’t leaving the other behind.
From experience, I normally see the consistent switch behavior. Depending on your task, it’s hard to determine the performance of your GAN (other than if a generated sample looks realistic)