How to obtain the corresponding relationship between forward and backward from a profile file

vision
1

I profile a model with train model ()and get the profile file xxx.json

with torch.autograd.profiler.profile(enabled=True, use_cuda=False, record_shapes=False, profile_memory=False) as prof:

I get the file, in the backward part, there are many kernels, but I don’t know the corresponding forward operator for it.

image
image2707×1179 279 KB

2

Alex Settle integrated sequence IDs into the profile as described in this RFE, which landed in this PR. He also followed up for AOTAutoGrad here, which might be useful for your workflow.

3

Much thanks! ! that loosk like which I was finding, I will try.