I have a custom torch.autograd.Function where I print out grad_in directly before returning it. The grad_in has non-zero values but when I look at convLayer.weight.grad it sometimes, seemingly randomly, has is all zeros. Additionally I am comparing this with old code from PyTorch 0.3 trying to transition to PyTorch 1.0. As far as I can tell the inputs, outputs, weights, and all values throughout the entire network are exactly the same. However with the same input in PyTorch 0.3 and the same grad_in, Pytorch 0.3 assigns a value to convLayer.weight.grad while in Pytorch 1.0 it is all 0’s.
I have downloaded pytorch from source and am using gdb to walk through python_engine.cpp and engine.cpp. However I am not having any luck actually finding where my tensors.grad values are being assigned and the values going into the calculation. Looking for help as to why this might be happening in python or where I should look in the c++ code for where the values of all the .grad’s are assigned.