Update internal variable across different forward pass

How can i update an existing variable across different inferences in a computational graph (a memory variable that gets updated for every input during a forward pass) making sure that the gradients are computed correctly for that variable?

For example, I want to do self.x = self.x + input every inference, where self.x is an internal variable for a layer.

I get an error that says the following:

RuntimeError: Trying to backward through the graph a second time (or directly access
saved tensors after they have already been freed). Saved intermediate values of the
graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=
True if you need to backward through the graph a second time or if you need to acces
s saved tensors after calling backward.

Upon setting retain_graph to True, I get the following error:

RuntimeError: one of the variables needed for gradient computation has been modified
by an inplace operation: [torch.cuda.FloatTensor [70, 32, 4, 4]] is at version 2; e
xpected version 1 instead. Hint: enable anomaly detection to find the operation that
failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

As a workaround, is it functionally equivalent if I detach() and reassign value to the same variable?

This looks like some kind of backpropgation through time. Could you consider doing a single backward step for multiple forward steps? Correct way to do backpropagation through time? - PyTorch Forums

I guess another question is what it would mean to compute a gradient for that variable across multiple forward passes. It’s not really a parameter as it is being overwritten during every forward pass.