Non Differentiable Function

autograd
1

Hi all,

Can someone please tell me how pytorch deals with non differentiable functions like torch.abs and L1 loss function?

2

For most non-differential function including L1, I think it uses sub-gradients of the function. Also, You can always use proximal gradient to calculate non-differential function, if you want to use some custom optimization method on the non-differential function

3

Will the sub-gradient be applied implicitly?

4

When we try to do autodiff of torch.min(x,abs(x)) to calculate the subgradient, it calculates as 0.5 which is not true. Why is this the case?

5

Can you share a minimal reproducible example of this incorrect behavior?