Is it possible for me to pinpoint an undetached tensor by plotting the computation graph?

I am facing the following error:

RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.

I am 90% percent sure this is because that I have some variables are used across different batches without being detached. Can I pinpoint those variables by looking at the computation graph using tensorboard? If so, what parameters should I pass to add_graph()?

Based on my understanding, this error occurs mostly when some input tensor with grad_fn that was created in previous computation is fed to the model. Normally, I can pinpoint the problem by checking the grad_fn of each input tensor, however, this time this error occurs when I am using Allennlp command line, in which a lot of data feeding operations are hided from us. I guess I can still to the same thing as previous, but with more effort to dissect the source code of Allennlp.

You are still bound to have written some glue code that deals with tensors?
Generally, those graphs get unwieldy much faster than just the inputs.

One of the things that easily happens is to not keep the leaf tensors:
x = torch.randn(1, 2, 3, requires_grad=True).cuda() is not a leaf tensor, so computing something with x, doing backward, computing something with x and then doing backward again, it will try to backward twice through the .cuda() step.

Best regards

Thomas

Sorry, I didn’t understand you. Are you just trying to give me an example that can lead to the error I run into?
Are there any general principles to locate the undetached tensors when running into this kind of error?

Yeah, sorry.
Most people start by looking at what they feed into the models, but you said it didn’t apply to you.

Best regards

Thomas