Hi. I’m trying to use a HyperNetwork, i.e. a network that generates weights for another network. However, I’m finding that the HyperNetwork weights are not being updated when running the opt.step() . Is there a way to trace the gradient graph, back to the input, to make sure the HyperNetwork weights are on there? It would be great if there was a way to visualise that but just being able to trace the gradients would be extremely helpful.
a = f1(input)
b = f2(a)
c = f3(b)
loss = MSE(c, truth)
From here, I’d like to be able to trace gradients back to make sure parameters of f3 have grad graph edge to params of f2 which in turn have grad graph edge to params of f1.