Performance of `.backward()` vs `torch.autograd.grad()`?

Is there any performance difference between Variable.backward() and torch.autograd.grad()?

quoting Adam from slack:

grad has to traverse the graph twice, but it shouldn’t be noticeable unless you’re doing a loooot of tiny ops. Apart from this no. It’s almost the same code path

2 Likes