I have declared two nn.Parameter() variables with requires_grad=True and I am using those in a different function that’s being called inside the init method of the class where variables are declared. lparam and rparam are not getting updated
My question is am I doing it the right way?
if not how it should be done?
here is the code example:
def __init__(self, opt):
self.opt = opt
self.lparam = nn.Parameter(torch.zeros(1), requires_grad=True).cuda(device=opt.gpu_ids)
self.rparam = nn.Parameter(torch.zeros(1), requires_grad=True).cuda(device=opt.gpu_ids)
def foo(self, a, b, k=1.0, lparam=0, rparam=0):
t = bar(a, b, k=k, lparam=lparam, rparam=rparam)
def forward(self, a, b):
x = self.foo(a, b, k=self.opt.k, lparam=self.lparam, rparam=self.rparam)
BaseNetwork is just initializing functions and uses nn.Module
The .cuda() operation on the nn.Parameter is differentiable and will create a non-leaf tensor.
Remove the .cuda() operation and call it on the nn.Module instead or alternatively call it on the tensor before wrapping it into the nn.Parameter.
Yes, since the cuda() and to() operations are differentiable as explained before with recommendations of a proper usage (call these ops on the module or on the tensor before wrapping it into an nn.Parameter):