Hello! I need to write a slightly modified percentage error loss function, with a threshold for the denominator. The part of code relevant to it looks like this:
I have a similar question about adding a constant in the loss function: let’s suppose I want to multiply the final loss by a factor, and that factor is computed based on the inputs or the outputs within a batch.
In my case I’m working with triplet networks and a custom loss, and now I would like to add a scale factor based on the amount of ‘easy negatives’, for example. This is part of my loss function:
If I remove the part where it computes the scale factor I get normal gradients after calling .backward(), but if I leave that section the gradients become nan.
How could I do that withouth breaking the computation graph? How should I deal with that constant (scale_factor)? Should I use .detach()?
Hi,
If you are getting nan, it means that you’re most likely dividing by 0 somewhere in your function. Can you check the different values to confirm that?
You can use .detach() but that will have a different effect: No gradient will flow back from the scale_factor. It depends on what you want to do here.