2: RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

autograd
1

Hi -

I’m trying to implement a custom loss function. Not differentiable, requires I implement gradients. The interpreter doesn’t like the current foo model:

class my_loss_fn(Function):

    def __init__(self, scalar=None):
        super(my_loss_fn, self).__init__()
        self.scalar = scalar
    
    @staticmethod
    def forward(self, i):
        out = (i - i).pow(2).sum()
        self.mark_non_differentiable(out)
        return out
    
    @staticmethod
    def backward(self, grad_output):
        grad_input = grad_output.clone()
        grad_input = 2.0*(grad_output)
        return grad_input, None

model = nn.Linear(2, 2)
x = torch.randn(1, 2)
target = torch.randn(1, 2)
output = model(x)
loss = my_loss_fn.apply(output)
loss.backward()
print(model.weight.grad)

I get error: RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

I’ve tried throwing requires_grad = True in some places, per some suggestions in other posts, with no luck.

What am I missing?

I don’t care if the model does anything meaningful, for now.

2

Hi,

You can find more details on how to implement custom functions here.
In particular, you don’t need the __init__ method.

And also, because you’re writing the custom Function, you only need to mark an output as non-differentiable if you don’t need gradient for it. Here you do, so you should remove the self.mark_non_differentiable(out) line.