# Exclude a variable from a change through a computation

Hi this is a beginner question and probably really quickly answered. But I could nto find it online, or probably to know the proper vocabulary to find what I am looking for.

I am trying to save the gradients into the variable “grads”, as in the code below.

``````grads=[0,0]
for idx,p in enumerate(model.parameters()):

``````

However, `p.grad.zero_` always not only sets the `p.grad` but also `grads` to zero.

I understand that it has todo with the way these two variables are connected through a graph. But how do I stop `p.grad.zero_` form setting also `grads` to zero.

I tried using `detach`, but that did not work.

``````grads=[0,0]
for idx,p in enumerate(model.parameters()):

``````

Both variables are pointing to the same memory address, you have to do a deep copy of p.grad in order not to be modified. You probably can do that by using the clone() tensor method.

1 Like

Hi, thanks for the quick reply. It worked! yey.

However, I dont understand why.

In this post
the difference between clone and detach is explained as follows:

You should use `detach()` when attempting to remove a tensor from a computation graph, and `clone` as a way to copy the tensor while still keeping the copy as a part of the computation graph it came from.

Maybe I am confusing the computatoinal graph and the memory adress. In my case I would like to remoe the `grads` object form my graph or not?

Hi,
As far as I understand you want to save gradients.
That’s not really related to computational graphs or pytorch stuff rather than python’s.

When I told you to use clone on grads because gradients are tensors, thus, you can use tensor’s methods. The reason why is clone does a kind of deep copy of the tensor (You can google python’s deepcopy and shallow copy).

In short, when you append a tensor into a list, that element of the list keeps pointing to the original tensor, thus, if the original tensor is modified, the one in the list is modified too.
(Example here copied from https://stackoverflow.com/questions/17873384/how-to-deep-copy-a-list)

``````>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = list(a)
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a = 10
>>> a
[[1, 10, 3], [4, 5, 6]]
>>> b   # b changes too -> Not a deepcopy.
[[1, 10, 3], [4, 5, 6]]
``````

About the computational graph it’s a really long story. In short, when you detach a tensor you are breaking the computational graph, whereas when you clone a tensor you are creating a “new branch” in the graph.