Hi, Thanks a lot.
So this means, when I do clone() on tensors, their clones will still be on the graph and any operations on them will be reflected in the graph right? for example changing the values or attributes will also change the original tensor as well? or affect the graph computation when doing backward pass?
In the case of the following two code snippets, what does happen in each case?
I think, deepcopy disregards any graph related information and just copies the data
as if it is a simple object while the clone, will create a new tensor which any operations on it will be reflected in the graph and in order to prevent this I need to use detach as well. am I right?
weights_encoder = sae_model.encoder[0].weight.data.clone()
weights_decoder = sae_model.decoder[0].weight.data.clone()
or
weights_encoder = copy.deepcopy(sae_model.encoder[0].weight.data)
weights_decoder = copy.deepcopy(sae_model.decoder[0].weight.data)