Thanks a lot for suggesting hook!!
I tried to use hook to print the mean of the grad tensor, and I find that there is an F.interpolate in my code which brings different gradients each time. Do I have any method to eliminate this randomness? With this randomness, the result fluctuates violently and I cannot tune the hyperparameters of my model.