Grad remains None for attributes, not for loose variables

autograd
1

When I make a Tensor that requires_grad an attribute of an object and return a clone of it with an @property method, it does not obtain any gradient for backpropagation. However, the same computation does give gradients for other Tensors (v in the example below), eventhough they go through the same operations. Removing the clone() operation does work from the my_parameter() method does work. However, according to the documentation clone() should backpropagate changes to the original, and it does so for v. Could someone please hint me what I am missing?

import torch
import torch.nn as nn
class MyModule(nn.Module):
    def __init__(self):
        super().__init__()
        self.__my_parameter = nn.Parameter(torch.tensor([1.0], requires_grad=True))

    @property
    def my_parameter(self) -> torch.Tensor:
        return self.__my_parameter.clone()

v = torch.tensor([1.0], requires_grad=True)
my_module = MyModule()
loss = torch.sum(v.clone() + my_module.my_parameter)
loss.backward()
print(v.grad, my_module.my_parameter.grad)

Output:

tensor([1.]) None
2

you’re calling the property getter here, my_module._my_parameter.grad would work (use single underscore to avoid name mangling, otherwise you parameter is only accessible as my_module._parameters[‘_MyModule__my_parameter’])

1 Like
3

Thank you, single underscores do indeed seem to fix it.

But with double underscores the variable would still be accessible through my_module._MyModule__my_parameter.grad. However, in the example above, printing my_module._MyModule__my_parameter.grad leads to exactly the same result as just calling my_module.my_parameter.grad.

Does the name mangling impact autograd?

4

idk, works for me:

my_module._MyModule__my_parameter.grad
Out[10]: tensor(cpu,(1,)[1.])
id(my_module._MyModule__my_parameter)
Out[11]: 158271096
id(my_module._parameters['_MyModule__my_parameter'])
Out[12]: 158271096
1 Like
5

Indeed it does. Hmm strange, I am quite sure I got a different result a moment ago. :man_shrugging: (Perhaps I made a typo somewhere? Or maybe Jupyter didn’t load the newest version properly?)

Either way, I can continue my work, thanks!