Transforming a tree to a dag in pytorch efficiently and without backward passes errors

I want to transform my tree neural net to a dag neural net. The forward pass is simple because I can simply detect if I’ve gone through that node/subtree already by caching a unique name for the node and the tensor. However, in the backward pass it is an issue because even if I reuse those tensors the auto grad engine complains that I’ve already done the backward pass through those tensors (and correctly so). Is there a way to modify the backward pass in a simple way so to re-use the already computed gradients?

Or is it just easier to abandon my old code and implement these models in Pytorch Geometric (PyG) or with the Deep Graph Library (dgl)?

For the sake of completeness here is the error message:

    train_loss, train_acc = agent.train(n_epoch)
  File "/home/miranda9/ML4Coq/ml4coq-proj-src/", line 102, in train
    loss.backward()  # each process synchronizes it's gradients in the backward pass
  File "/home/miranda9/miniconda3/envs/metalearningpy1.7.1c10.2/lib/python3.8/site-packages/torch/", line 221, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/miranda9/miniconda3/envs/metalearningpy1.7.1c10.2/lib/python3.8/site-packages/torch/autograd/", line 130, in backward
RuntimeError: Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling backward the first time.

cross posted: machine learning - How doe some transforms a Tree Neural Net to a Dag Neural Net in Pytorch efficiently and without backward passes errors? - Stack Overflow