BCEWithLogitsLoss

1

I have multilabel onehot encoded targets. As definition each of my examples can fall in more classes but it can fall also in None of them. Should I create a new class corresponding to the examples that fall in no class or is it ok to have example with all 0 vector target.

Some example:
‘text’–>8 element target

‘i do not like this movie’ → [0,0,0,0,0,1,0,0]
‘i do not like this movie because i hate thriller’ → [1,0,0,0,0,1,0,0]
‘hi I’m tommy’ → [0,0,0,0,0,0,0,0]

My question referring to the example is if it’s, in general, better to change everything to:
‘text’–>9 element target

‘i do not like this movie’ → [0,0,0,0,0,1,0,0,0]
‘i do not like this movie because i hate thriller’ → [1,0,0,0,0,1,0,0,0]
‘hi I’m tommy’ → [0,0,0,0,0,0,0,0,1]

2

My understanding is that your additional “other” logit would not bring in any additional information in the sense that saying it is the 9th element is equivalent to say it is none of the others.

1 Like
3

I agree with you that it brings no additional information, in my training set the NoClass examples are around 70% of the dataset and I wonder which of the two explained situation are better in order to add the “pos_weight” parameter of BCEWithLogitsLoss. Also the other classes are unbalanced, which is in the 30% of the dataset the classes are not equally distributed.

4

The pos_weight parameter is specified per-class. e.g. if the 0th class only has one positive occurrence to a thousand negative occurrences then you would set it to 1000/1, 1000, regardless of the relative occurrences of other classes. It only concerns the relative appearance of a positive value for that class in relation to the number of times it’s 0.

All that adding a class for the no-class case would do for pos_weight is add another column where the weight is approximately 30/70 (i.e. weighting the negative cases more strongly to offset how many positive values there are).

All this is assuming that you wish to weight the classes as if they’re all equally likely to occur within the dataset.

1 Like