Simultaneously train two model in each epoch

def U_closure(self):
    loss = self.cost_function()
    return loss
def E_closure(self):
    loss = self.cost_function()
    return loss
# Training function
def train(self, epochs, opt_func=torch.optim.Adam):
    self.optimizer_U_model = opt_func(self.U_model.parameters())
    self.optimizer_E_model = opt_func(self.E_model.parameters())
    for epoch in range(epochs):

There are two models U_model and E_model which are needed to be trained in each epoch. But when sending closure function to optimizer it is calculating loss two times. I have same loss for both model. Should I have only one optimizer? If I have only one optimizer then can it take parameters of both modes?

You can use one optimiser that has access to both models’ parameters. Concerning the loss, just sum the two, the gradient won’t change.

What should be the parameter for line

self.optimizer = opt_func(self.U_model.parameters())

,i.e., how can I give two weights parameter in single opttimiser?

I was just wondering if you have tried this.

I guess that @mxahan has provided you with the answer. In any case, the PyTorch documentation already provides you with examples that I think can be extended to multiple models.

In particular I am referring to this:

                {'params': model.base.parameters()},
                {'params': model.classifier.parameters(), 'lr': 1e-3}
            ], lr=1e-2, momentum=0.9)