After first round optimization success, raise ValueError: can't optimize a non-leaf Tensor

1

Hi all,

I need a bit of help with my PyTorch code. I am trying to generate a random image, and then optimize its cell values in order to be classified as a targeted class by an image classifier, the optimization seems to work for the first iteration, and then raise the ValueError: can’t optimize a non-leaf Tensor.

I am quite confused why the optimizer seems to be working for the first time but not for the following iterations. Any help would be highly appreciated.

This is my code:

# set the target class
y_target = Variable(torch.LongTensor([0]), requires_grad=False)
y_target = y_target.to(device)

# Generate a random image
created_image = torch.normal(mean=mean[0], std=std[0], size=(1,1,28,28), requires_grad=True, device="cuda")

for i in range(1, 200):
    # Define optimizer for the image
    optimizer = optim.SGD([created_image],lr=1e-2)
    print("111")
    #clamp created img
    created_image = torch.clamp(created_image, -0.4241,2.8214)
    zero_gradients(created_image)
    # Forward
    output_created_image = net.forward(created_image)
    # Get confidence from softmax
    created_image_confidence = F.softmax(output_created_image)[0][y_target]
    print("222")
    print(created_image_confidence)
    created_image_confidence_probs =  round((created_image_confidence.item()) * 100,2)
    print("333")
    print(created_image_confidence_probs)

    if created_image_confidence_probs > 80.00:
        print("444")
        created_image = created_image.squeeze().detach().cpu().numpy()
        created_image = (created_image * std[0] + mean[0]) * 255.0
        np.save(f"./adv_images/{y_target.item()}/{i}_gaf_t_c80.npy",adv_data)
        break

    # Target specific class
    class_loss = -output_created_image[0, y_target]
    print("555")
    # Zero all existing gradients
    net.zero_grad()
    print("666")
    # Backward
    class_loss.backward
    print("777")
    # Update image
    optimizer.step()
    print("888")

And this is the result I get:

111
222
tensor([0.0891], device='cuda:0', grad_fn=<IndexBackward>)
333
8.91
555
666
777
888
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
 in 
     16     # created_image = created_image.requires_grad_()
     17     # Define optimizer for the image
---> 18     optimizer = optim.SGD([created_image],lr=6)
     19     print("111")
     20     #clamp created img

C:\ProgramData\Anaconda3\lib\site-packages\torch\optim\sgd.py in __init__(self, params, lr, momentum, dampening, weight_decay, nesterov)
     62         if nesterov and (momentum <= 0 or dampening != 0):
     63             raise ValueError("Nesterov momentum requires a momentum and zero dampening")
---> 64         super(SGD, self).__init__(params, defaults)
     65 
     66     def __setstate__(self, state):

C:\ProgramData\Anaconda3\lib\site-packages\torch\optim\optimizer.py in __init__(self, params, defaults)
     49 
     50         for param_group in param_groups:
---> 51             self.add_param_group(param_group)
     52 
     53     def __getstate__(self):

C:\ProgramData\Anaconda3\lib\site-packages\torch\optim\optimizer.py in add_param_group(self, param_group)
    200                                 "but one of the params is " + torch.typename(param))
    201             if not param.is_leaf:
--> 202                 raise ValueError("can't optimize a non-leaf Tensor")
    203 
    204         for name, default in self.defaults.items():

ValueError: can't optimize a non-leaf Tensor
2

You are creating a non-leaf tensor with the torch.clamp operation:

created_image = torch.normal(mean=1, std=1, size=(1,1,28,28), requires_grad=True)
print(created_image.is_leaf)
> True
created_image = torch.clamp(created_image, -1, 1)
print(created_image.is_leaf)
> False

Try to change the output of torch.clamp to another variable name.

Also, there are some minor issues in the code:

  • Variables are deprecated since 0.4.0, so you can use tensors directly
  • Don’t call net.forward, as this will not call into registered hooks. Instead call the model as net(input)
  • class_loss.backward is probably a typo and should call the method as class_loss.backward()
3

Thanks a bunch, ptrblck! All issues solved. Save my day :smiley: