I have a simple GAN architecture with a nn.BCEWithLogitsLoss() as a loss function. My final layer is linear and I am using the default weight initialization.
I want to validate my weight initialisation and expect to get an initial loss of log(2) =0.6931 though when I generate random inputs I get a loss value around 0.66 - 0.67
I believe the difference is significant as repeating the experiments still gives loss values in the same range.
Was looking forward to help to find ways to further debug this?
Thanks!