Hi all, I have two distinct network with same loss and two optimizers for each network. How can I backprop the loss to the networks?
Any help would be appreciated.
This probably means the common loss is a function of the parameters of both the models.
You could then just create a single optimizer containing the parameters of both the models, like so:
optimizer = torch.optim.Adam([modelA.parameters()] + [modelB.parameteres()])
Let me know if this gives any errors.
Hi @srishti-git1110, thanks for your tip, but my optimizers are two distinct optimizers with different hyperparameters.
In that case, is this code not working?
There’s also an option to create parameter groups within the same optimizer but I think that is equivalent to creating two different optimizers with different learning rates etc. (even for optimization algorithms like Adam that calculate running stats.)
Is it true? I mean what is the acceptable code for this problem
Your code should work fine. Please post if you face any errors.