Hello,
Is there any way to print all of the graph leaves of a model? Otherwise (/preferably), is there any way to see which autograd.Variables will have their gradients updated during the backward() call on the root node?
For context, I’m having a problem where my backward passes are getting slower as training progresses, and I’m trying to diagnose what’s causing this. I saw this problem earlier when I had mistakenly set requires_grad=True for all of my inputs, so the backward pass was unnecessarily computing the gradients for all previously inputs. When I set all of these back to requires_grad=False, the backward passes sped up considerably; however, they are still slowing down over time. I’m assuming a similar mechanism is causing this where loss.backward() is computing the gradient with respect to past variables, but unfortunately I can’t track them down in my code.
Any help is appreciated! Thanks.