Hi, everyone. I don’t have much experience in training a GAN. Looking for help. Welcome discussion.
I am trying the tutorial dcgan codes from here https://github.com/pytorch/examples/tree/master/dcgan
the training logs suddenly turned into this. As we can see, the discriminator fails to recognize the real images as real. D(x) drops to 0 suddenly which represents discriminator recognizes the real images as fake. I cant figure out why. Anyone have occurred this phenomenon when training a GAN? Please help.
The logs I posted here is based on MNIST dataset, and the setting of the codes is:
--dataset mnist --dataroot /data --workers 2 --cuda --niter 25 --batchSize 64 --imageSize 64 --ngf 64 --ndf 64 --outf output```
Is this due to the batchsize, or optimizer or learning rate or something? I am running the codes on lsun dataset again now, and this strange phenomenon have not appeared yet.
[3/25][2300/7500] Loss_D: 0.0575 Loss_G: 4.4713 D(x): 0.9529 D(G(z)): 0.0024 / 0.0136
[3/25][2400/7500] Loss_D: 0.0026 Loss_G: 8.2796 D(x): 0.9977 D(G(z)): 0.0003 / 0.0003
[3/25][2500/7500] Loss_D: 0.0025 Loss_G: 6.8174 D(x): 1.0000 D(G(z)): 0.0025 / 0.0011
[3/25][2600/7500] Loss_D: 0.0030 Loss_G: 7.3201 D(x): 0.9988 D(G(z)): 0.0018 / 0.0008
[3/25][2700/7500] Loss_D: 63.1907 Loss_G: 65.5239 D(x): 0.0000 D(G(z)): 0.0000 / 0.0000
[3/25][2800/7500] Loss_D: 65.7721 Loss_G: 50.2691 D(x): 0.0000 D(G(z)): 0.0000 / 0.0000
[3/25][2900/7500] Loss_D: 62.2274 Loss_G: 65.5300 D(x): 0.0000 D(G(z)): 0.0000 / 0.0000
[3/25][3000/7500] Loss_D: 63.2673 Loss_G: 65.6175 D(x): 0.0000 D(G(z)): 0.0000 / 0.0000
[3/25][3100/7500] Loss_D: 58.2727 Loss_G: 57.9211 D(x): 0.0000 D(G(z)): 0.0000 / 0.0000
[3/25][3200/7500] Loss_D: 62.8068 Loss_G: 65.5562 D(x): 0.0000 D(G(z)): 0.0000 / 0.0000
[3/25][3300/7500] Loss_D: 63.3889 Loss_G: 65.6232 D(x): 0.0000 D(G(z)): 0.0000 / 0.0000
[3/25][3400/7500] Loss_D: 65.0035 Loss_G: 65.4272 D(x): 0.0000 D(G(z)): 0.0000 / 0.0000
[3/25][3500/7500] Loss_D: 62.5094 Loss_G: 42.7007 D(x): 0.0000 D(G(z)): 0.0000 / 0.0000
[3/25][3600/7500] Loss_D: 61.7205 Loss_G: 65.4317 D(x): 0.0000 D(G(z)): 0.0000 / 0.0000
I also found that the logs suddenly turns into this later. However, when I do the visual check, the generative images are all wrong.
[8/25][3300/7500] Loss_D: 58.5593 Loss_G: 87.9837 D(x): 0.0000 D(G(z)): 0.0000 / 0.0000
[8/25][3400/7500] Loss_D: 58.9293 Loss_G: 80.2016 D(x): 0.0000 D(G(z)): 0.0000 / 0.0000
[8/25][3500/7500] Loss_D: 61.1501 Loss_G: 87.7365 D(x): 0.0000 D(G(z)): 0.0000 / 0.0000
[8/25][3600/7500] Loss_D: 58.7557 Loss_G: 87.5904 D(x): 0.0000 D(G(z)): 0.0000 / 0.0000
[8/25][3700/7500] Loss_D: 27.6310 Loss_G: 0.0000 D(x): 1.0000 D(G(z)): 1.0000 / 1.0000
[8/25][3800/7500] Loss_D: 27.6310 Loss_G: 0.0000 D(x): 1.0000 D(G(z)): 1.0000 / 1.0000
[8/25][3900/7500] Loss_D: 27.6310 Loss_G: 0.0000 D(x): 1.0000 D(G(z)): 1.0000 / 1.0000
[8/25][4000/7500] Loss_D: 27.6310 Loss_G: 0.0000 D(x): 1.0000 D(G(z)): 1.0000 / 1.0000
[8/25][4100/7500] Loss_D: 27.6310 Loss_G: 0.0000 D(x): 1.0000 D(G(z)): 1.0000 / 1.0000