Hi I am using torch.nn.BCELoss to train my model. The targets are either 0 or 1 (segmentation application, so the targets are a matrix of 0’s and 1’s). But I saw in the description of this loss ‘Note that the targets yy should be numbers between 0 and 1’. I am curious if its necessary to scale my targets to lie between 0 and 1?
Hi Samra!
More precisely, the BCELoss documentation should read:
Note that the targets y should be numbers between 0 and 1,
inclusive.
That is, the y should range over the (so-called closed) interval
[0.0, 1.0], including taking on, potentially, the values 0.0 and 1.0.
The target y can be understood to be the “ground truth” probability of
the sample being in the “1” state. But this means that it also works as
a binary label: y = 1 means that the sample has a 100% probability
of being in the “1” state, so y = 1 means the “1” state, while y = 0
means 0% probability of the “1” state, hence 100% probability of the
“0” state, so y = 0 means the “0” state (when understood as a binary
label).
As a side note, for numerical reasons you will be better off using
BCEWithLogitsLoss (and using the output of the final Linear
layer of your model as the model’s output, without passing it through
a Sigmoid.
Best.
K. Frank