Optimizer Error in non leaf tensor

1

hello, I have an error in my code, can you help me to solve.
I have initialized my weights
weight_1 = nn.Parameter(torch.Tensor(128,128)).cuda()
weight_2 = nn.Parameter(torch.Tensor(128,128)).cuda()

optim.Adam([{‘params’:model.parameters()}, {‘params’: [weight_1, weight_2]}], lr=1e-5) . It is giving me a value error: cannot optimize a non-leaf tensor.

2

I think you can get the desired behavior by moving the .cuda() call inside:

>>> import torch
>>> w1 = torch.nn.Parameter(torch.Tensor(1)).cuda()
>>> w2 = torch.nn.Parameter(torch.Tensor(1).cuda())
>>> w1.is_leaf
False
>>> w2.is_leaf
True
>>>