Computing gradients of cloned parameters

[Rouarez]

I think I found an answer to my question:

Loading a state dict seems to erase grad

Hi Ya’ll, I am experiencing a somewhat spooky error, in which the simple act of loading a new state dictionary into my model seems to erase the graph tying parameters to an external matrix. Here is the code, and I will explain below. for grad, (name, param) in zip(recovered, model.named_parameters()): updated_param = param - stepsize * grad # This is 100% dependent on M new_state_dict[name] = updated_param model.load_state_dict(new_state_dict) l = list(model.para…

WHen loading the graph, we don’t load the computational graph. Therefore the gradients are cannot be backpropagated to the dict object loaded into it.