Gradients do not update when I replace the loss value in this code

autograd
1

I would like to use weighted multi-task loss in my experiment. For the same, I found a script (here).

The only change I made in this script was to replace the squared loss with another loss value. However, this does not seem to be causing the problem where the self.log_vars.grad results in None. The standalone script works fine, but when I use it with my setup, it does not update grads. Yes, list(model.parameters()) lists: tensor([0., 0.], device='cuda:0', requires_grad=True)

  • another change from the script is that my model setup is on GPU where self.log_vars.grad = None, but in the orignal script, it runs well on the CPU.
2

The problem is solved. Thanks to this post and discussion between @ljj7975 & @ptrblck for hinting me towards checking if the computation graph was broken somewhere. It was exactly that. The computation graph was broken while calling loss.backward().

2 Likes
3

I had similar problem, but it was solved as I mentioned here: