I use Batchnorm2d in my WGAN-GP model. I have heard that it is preferred for GAN models to stay on training mode because eval mode is highly unstable, which is particularly the case when you have batchnorm in generator. Is this true?
I already did a bit of testing and it does seem like the model on training more generated more realistic and diverse samples while the model on eval mode outputted very low quality images that almost looked like mode collapse. I originally thought eval mode would give improved results, but it seems like my WGAN-GP model does way worse on eval mode. Is this normal, and should I just use my model on training mode for evaluation?