Behavior of BCEWithLogitsLoss pos_weight

[whoab]

Let’s say we have a nB x 2048 prediction tensor and nB x 2048 ground truth/label tensor.

Let’s say we plug both of these into BCEWithLogitsLoss.

The docs says:
* **pos_weight** ([*Tensor*](https://pytorch.org/docs/stable/tensors.html#torch.Tensor)*,* *optional*) – a weight of positive examples. Must be a vector with length equal to the number of classes.

But, if you make a pos_weight of size: nB x 2048 instead of the required 2048, the BCEWithLogitsLoss still accepts the pos_weight. Why is that? What happens when you do the former; does it just weight every single scalar?

[KFrank]

Hi Whoab!

@whoab

But, if you make a pos_weight of size: nB x 2048 instead of the required 2048, the BCEWithLogitsLoss still accepts the pos_weight.

Please see this post for an explanation:

"pos_weight" and "weight" parameters in BCEWithLogitsLoss

Hi Ali! Let me answer a couple of your specific questions first and then explain how I look at it. In this case the current (1.9.0) documentation for BCEWithLogitsLoss is wrong (or at least misleading). Quoting: weight (Tensor, optional) – a manual rescaling weight given to the loss of each batch element. If given, has to be a Tensor of size nbatch. On the contrary, weight can have other shapes – for example, it can have the same shape as input. (As near as I can tell, the precise r…

That post discusses primarily BCEWithLogitsLoss’s weight parameter
but the same comments apply almost unchanged to the pos_weight
parameter.

Best.

K. Frank