I have a instance of a neural network called myNet (that inherits from nn.Module). I would like to Perform a deep copy of this network (including gradient values of all parameters).
To perform an initial copy of the parameter values, I performed a deepcopy as follows:
myCopy = copy.deepcopy( myNet )
However, this does not copy the gradient values. Thus, I performed the following:
optimizerCopy = optim.SGD( netCopy.parameters(), lr=0.1, momentum=0.0 ) optimizerCopy.zero_grad() for paramName, paramValue, in net.named_parameters(): for netCopyName, netCopyValue, in netCopy.named_parameters(): if paramName == netCopyName: netCopyValue.grad = paramValue.grad
This only does a shallow copy, though, of the gradient values.
How can I perform a deep copy of gradient values from myNet to myCopy?