Question about BCEWithLogitsLoss

[Nikronic (Nikan Doosti)]

Hi,

It seems you misunderstood the BCEWithLogitsLoss. It uses sigmoid function on its inputs not on outputs. Here is pipeline: x->BCEWithLogitsLoss = x-> sigmoid -> BCELoss (Note that BCELoss is a standalone function in PyTorch too.)

If you look at the documentation of torch.nn.BCEWithLogitsLoss, it says “This loss combines a Sigmoid layer and the BCELoss in one single class. This version is more numerically stable than using a plain Sigmoid followed by a BCELoss as…”.

Also, this post may help you too.

BCELoss vs BCEWithLogitsLoss

@ptrblck Isn’t it the other way around? I thought BCELoss needs to receive the outputs of Sigmoid activation as its input, but the other-one BCEWithLogitsLoss will need the logits as inputs instead of outputs of Sigmoid, since it will apply sigmoid internally. Although, the example in the docs do not apply Sigmoid function prior to BCELoss: ### Example from pytorch-docs: >>> m = nn.Sigmoid() >>> loss = nn.BCELoss() >>> input = torch.randn(3, requires_grad=True) >>> target = torch.empty(3).ran…

By the way, sigmoid maps input to [0,1] by default.

bests