How to identify slowest step in backward computation?

I want to know if there is any principled way of identifying which step is taking maximum amount of time when performing loss.backward().

You could try out torch.utils.bottleneck to find bottlenecks in your autograd part.

1 Like

This is exactly what I wanted thanks.