Hi, I’ve been trying to get Tensorboard working for some recurrent models. I want to visualise the graph for a model, but I have this issue where when I try and use .add_graph, TensorBoard returns:
‘RuntimeError: Cannot insert a Tensor that requires grad as a constant. Consider making it a parameter or input, or detaching the gradient’
As far as I can tell, its complaining about the hidden state of the Recurrent unit. I can make the error go away if I tell the hidden state to detach during ‘forward()’, but then the network is unable to train. Besides, the network is able to run and update parameters properly, so I don’t really see why Tensorboard should be unable to create a graph from it. Thanks! Code is below.
import torch
from torch.utils.tensorboard import SummaryWriter
import torch.nn as nn
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.rec1 = nn.RNN(1, 10, 1, bias=False)
self.lin1 = nn.Linear(10,1)
self.hidden = None
def forward(self, x):
x, self.hidden = self.rec1(x, self.hidden)
#self.hidden = self.hidden.detach()
return self.lin1(x)
if __name__ == '__main__':
in_data = torch.empty([100, 10, 1])
tgt_data = torch.empty([100, 10, 1])
network = Net()
loss_fcn = nn.MSELoss()
optimizer = torch.optim.Adam(network.parameters(), 0.0005)
network.hidden = torch.empty([1, 10, 10])
out = network(in_data)
loss = loss_fcn(out, tgt_data)
loss.backward()
optimizer.step()
writer = SummaryWriter('runs/test_icicles')
writer.add_graph(network, in_data)