Is torch.nn.BCELoss `reduction` applied for "pixel" or "batch"?

1

Hi,

I am wondering how the result would be calculated if I set reduction=mean in torch.nn.BCELoss when I use auto-encoder model.
Do I average the loss only for “batch” dimension or both “pixel” and “batch”?

If I average for both “pixel” and “batch”, is there no way to sum loss for all “pixel” and to average for “batch” dimension?

Thanks

2

Hello,

reduction=mean in BCELoss means average in pixel dimension, the result you get is a scalar.
If you want to sum loss in pixel dimention and perform averaging in batch dim, you could set reduction='none' and manually add sum and average operation.

criterion = nn.BCEWithLogitsLoss(reduction='none')
...
loss = criterion(logits, target).sum(dim=(1,2,3)).mean(dim=0)
loss.backward()
1 Like
3

It is clear. Thanks a lot :slight_smile: !!!

4

If I use reduction=mean in BCELoss, you said that the result is a scalar.
and does it mean that it is reduced(mean) in both batch dimension and pixel dimension?