[SOLVED] Should I use torch.clamp after torch.sigmoid?

[KFrank]

Hello Andrei!

@tzeny

…
Tracing it back I reached the conclusion that sometimes my model outputs very small numbers, for example -136. This leads to:

torch.sigmoid(torch.tensor([-136.])) -> tensor([0.]) which leads to -inf in torch.log().

Is it ok to use torch.clamp(torch.sigmoid(torch.tensor([-136.])), min=1e-8, max=1 - 1e-8) ?

The short (narrow) answer – use LogSigmoid.

Please see my reply to your “Weighted Binary Crossentropy”
post:

[SOLVED] Class Weighed Binary Crossentropy not working, even with equal weights

Hi Andrei! First, to answer the question I think you’re asking: You should be using (as in the comment in your code) BCEWithLogitsLoss. BCEWithLogitsLoss supports sample weights, which you can use for class weights. Let’s say you have class weight w_1 for class 1, and w_0 for class 0. Let w_n be the sample weight for sample n. Simply set w_n = w_1 if y_n = 1, and w_n = w_0 if y_n = 0. (This assumes that the y_n are either 0 or 1, as they should be if they are binary class labels.) N…

Best.

K. Frank