I would like to use weighted multi-task loss in my experiment. For the same, I found a script (here).
The only change I made in this script was to replace the squared loss with another loss value. However, this does not seem to be causing the problem where the
self.log_vars.grad results in
None. The standalone script works fine, but when I use it with my setup, it does not update grads. Yes,
tensor([0., 0.], device='cuda:0', requires_grad=True)
- another change from the script is that my model setup is on GPU where
self.log_vars.grad = None, but in the orignal script, it runs well on the CPU.