The short story is that torch.minimum() and torch.maximum() are not
differentiable at the special points where the arguments are equal.

The function abs (x) offers a simpler example. When x > 0, the derivative
is +1, while when x < 0, itβs -1. When x = 0, mathematically speaking,
the derivative is not defined. (One could try to play some game where you
define the derivative to be 0 when x = 0 β and pytorch does do this for the abs() function β but doing so is problematic, and not really worth the bother.)

Here is an example script that probes your loss function with values where
the arguments are close, but not exactly equal:

First, I do not get the same gradients using your new version of iou_loss().

Second, while your new iou_loss() still returns zero for the input parameters
given in your first post, this is no longer the minimum as iou_loss() can now
become negative.

If you believe otherwise please post a complete, runnable script, together with
its output, that demonstrates your result.

Hi K.Frank, you are right. the loss becomes 1 - num_pred / num_target now and it can be negative. The signs of the gradients switched, too. Sorry it was a stupid question. Thank you for your help.