I am trying to implement a method which has three sub networks. An encoder, discriminator and a regressor.
The outputs of the encoder are inputs to both the discriminator as well as the regressor.
We also define 3 optimizers. They are as follows.
optimizer = optim.Adam(list(encoder.parameters()) + list(regressor.parameters()), lr=1e-4) optimizer_conf = optim.Adam(list(encoder.parameters()), lr=1e-4) optimizer_dm = optim.Adam(list(domain_predictor.parameters()), lr=1e-4)
Here’s my code.
# First update the encoder and regressor optimizer.zero_grad() features = encoder(data) output_pred = regressor(features) loss_total = criteron(output_pred, target) loss_total.backward(retain_graph=True) optimizer.step() # Now update just the domain_predictor optimizer_dm.zero_grad() output_dm = domain_predictor(features.detach()) loss_dm = domain_criterion(output_dm, domain_target) loss_dm.backward(retain_graph=False) optimizer_dm.step() # Now update just the encoder using the domain loss optimizer_conf.zero_grad() output_dm_conf = domain_predictor(features) loss_conf = beta * conf_criterion(output_dm_conf, domain_target) loss_conf.backward(retain_graph=False) optimizer_conf.step()
This is the error I am getting, I have also put anomaly detection:
***one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor ] is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!*** File "/home/new_user/DA/Brain-MR-Segmentation-Playground/methods/unlearn.py", line 303, in train_unlearn loss_conf.backward(retain_graph=False) File "/home/new_user/DA/Brain-MR-Segmentation-Playground/methods/unlearn.py", line 524, in cmd_train loss, acc, dm_loss, conf_loss = train_unlearn(ctx, models, train_dataloaders, optimizers, criterions, File "/home/new_user/DA/Brain-MR-Segmentation-Playground/methods/runner.py", line 31, in run_main selected_method.cmd_train(ctx) File "/home/new_user/DA/Brain-MR-Segmentation-Playground/methods/runner.py", line 35, in <module> run_main()