[resolved] Gradient is None on GPU in gradcheck.py

Hello everyone,

I am implementing multivariate normal distribution with auto gradient, I manually wrote backward pass. When using gradcheck.py to check if I calculate the gradients right, I found out when the inputs were on CPU, the test cases always passed while on GPU, the test cases failed. Later, I looked into the gradcheck.py, and found out the analytical gradient became all zeros which means after doing .backward(), the gradients in my variables are all None. However, in my last test cases, it shows that the backward function of this self-defined operation works fine with cudaTensor.

I have fixed this problem. This seems to be the same problem with Gradient computed on CPU but nor computed on GPU?.

Although I still don’t quite get the reason of the solution given in the discussion above.

Besides, in my original test cases, I seem to use the first given solution, but it turned out to fail.