How to time the running time of each layer of a model?

autograd
1

Now I have to time the running time of each layer of a model, including forward, update and backward. Does pytorch provide such tools? Thanks!

2

Maybe torch.autograd.profiler can help, but I haven’t used torch.autograd. But you can do it yourself with the time library.
For the forward pass, you can manually place time.start() and time.end(). And for the backward pass you can use hooks. Refer to this post for reference link

3

Thanks for your reply. But I want to measure the running time of each layer, and I don’t understand how to use time library to do that, could you explain more?

4

I don’t understand. Why you don’t want to ise time lib? It can do what you want. But if you still use hooks maybe.

5

I have the same issue for profiling the runtime of each layer of a model, including forward, backward and update during training. Do you find some pytorch tools? Or give me some detailed implement hints. Thank you.