I am trying to implement an RNN-based model for time-series data and any help would be much appreciated! I have a reward signal I would like to utilize to backpropagate a loss through the RNN every n steps. I cannot seem to find a way to backpropagate anything without detaching the hidden state, but I don’t think that is a good approach in this case.
Let’s start from a simple example, straight from the PyTorch documentation, but amended such that one can step through the optimizer within the time loop:
import torch import torch.nn as nn rnn = nn.GRUCell(10, 20) optimizer = torch.optim.SGD(rnn.parameters(), lr=1e-3) input = torch.randn(6, 3, 10) hx = torch.randn(3, 20) output =  for i in range(6): hx = rnn(input[i], hx) output.append(hx) hx.mean().backward(retain_graph=retain_graph) optimizer.step() optimizer.zero_grad()
In the above script, if
retain_graph is False, I get a “RuntimeError: Trying to backward through the graph a second time”. When
retain_graph is True, I get “RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [20, 60]]”.
I’ve seen some time-series RNN implementations online that don’t seem to bother with detaching the hidden state, so I suppose this used to work well in previous PyTorch versions.