I have a combined network with generator and discriminator.
The joint training of this network collapses due to the over-fitting of discriminator.
So I try to update discriminator occasionally,
i.e., generator will be updated in every time step but in case of discriminator it will update only once in a hundred step.
The pseudo-code of the model is
def __ init__():
def forward(self, desired, predicted)
loss_discriminator = nn.crossEntropyLoss(desired, predicted)
loss_generator = nn.MSELoss(desired, predicted)
return aloss_discriminator + bloss_generator
predicted = model( input )
loss = custom_loss( desired, predicted )
(where a, b are constant)