Functions like abs() are non-differentiable at some certain points. How does Autograd deal with that?
Mostly some (more or less) arbitrary extension from the intervals is used.
One thing that people seem to like - and PyTorch mostly does - is to have zero derivative if it is zero in a neighbourhood - eg for relu at zero.
OIC. Thank you for helping!