Some confusion about the grad of torch.sign()

autograd
1

Here’ s my demo and why x_binary 's grad is always None ?

I do understand the grad of torch.sign() does not exist mathematically, but F.conv2d should give x_binary grad even though it won’t be passed to x.

import torch
import torch.nn.functional as F

x = torch.randn(2,3,4,4, requires_grad=True)
# print x
x_binary = x.sign().requires_grad_()
# print x_binary

weight = torch.ones(3,3,1,1, requires_grad=True)
# print weight

out = F.conv2d(x_binary, weight, None, 1, 0, 1, 1)
# out = F.conv2d(x, weight, None, 1, 0, 1, 1)
# print out

loss = out.mean()
print loss

tensor(-0.3750)

print x.grad, x_binary.grad

None None

loss.backward()

print x.grad, x_binary.grad

tensor([[[[ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.]],

         [[ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.]],

         [[ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.]]],


        [[[ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.]],

         [[ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.]],

         [[ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.]]]]) None

print weight.grad

tensor([[[[-6.2500e-02]],

         [[-6.2500e-02]],

         [[ 1.8626e-09]]],


        [[[-6.2500e-02]],

         [[-6.2500e-02]],

         [[ 1.8626e-09]]],


        [[[-6.2500e-02]],

         [[-6.2500e-02]],

         [[ 1.8626e-09]]]])
2

x_binary = x.sign().detach().requires_grad_() works.

Looks like a bug to me.

3

Any plan to report this?

4

I think you should report this.

5

see the discussion here

1 Like
6

I use your method in the code such as:

symbol_alpha = relation_alpha.sign().detach().requires_grad_()

but when the loss backward, the parameters of relation_alpha is none , my pytorch version is 1.0.
why?