out1 = module1(input)
out2 = module2(out1)
out3 = module3(out2)
If there is code like above, How can I know backpropagation is well done?
It there any technical method to check it?
Thank you for reading it and happy new year!
out1 = module1(input)
out2 = module2(out1)
out3 = module3(out2)
If there is code like above, How can I know backpropagation is well done?
It there any technical method to check it?
Thank you for reading it and happy new year!
You could call backward
on out3
(or the mean/sum of it) and see if all parameters you care about have valid .grad
attributes. Since .grad
will be None
after initialization, this should give you some feedback on the proper gradient calculation.
thank you for help
and happy new year!