I’m working on a classification problem which can have a variable number of classes as the ground truth. And for each data point, I’d like to have k possible targets.
This problem can be modelled as a single-label, multi-class problem with probabilistic (“soft”) labels. The soft targets can be configured to have a probability of 0.7 for class 3, a probability of 0.2 for class 2, and a probability of 0.1 for class 5 (and zero for everything else). Ref
But for certain examples, I want the model to predict more than 1 class for a given data point. Should I use BCEWithLogitsLoss, CrossEntropyLoss or some combination of both?
This just means that one data point can fall in multiple categories (rather than just one) – a multi-label multi-class problem, essentially.
You should be fine using BCEWithLogitsLoss. Please note from the docs -
This loss combines a Sigmoid layer and the BCELoss in one single class.
which means you do not have to explicitly include a sigmoid layer.
@srishti-git1110
I had a follow-up question on this. Say my ground truth classes were [3,4,5,6] and their probabilities were [0.7, 0.7, 0.3, 0.3] and I wanted my model to pick one of [3,4] and another of [5,6], can the loss function be modelled accordingly?
Because the way I say it, it’ll always try to converge into predicting [3,4] as it has the highest weight but turns out, [5,6] are the next best options to choose from.