How to ignore certain samples when employing CrossEntropyLoss?

The detailed problem is as follows:
I have a mixture of Labelled Sample Group and Unlabeled Sample Group. For some special reason, I have to mix those samples in a same mini-batch to train a classification network, with CrossEntropyLoss. So I need to ignore these unlabeled samples when computing the CrossEntropyLoss.

I tried to assign a pseudo label to these unlabeled samples and set the option weight in CrossEntropyLoss to be (zero) corresponding to the pseudo label , but found that it wouldn’t satisfy my demand. First the output turns to be C+1 vector. Second and most importantly, though the loss on the pseudo class is ignored, if the prediction of an unlabeled data have large values on other indexes, it still generate large loss. I need those samples contribute exactly 0 to the CrossEntropyLoss.

So will someone kindly help me with this issue? Thx a lot.

Define a custom loss. I had an example where I wanted to treat different samples in batch differently. I’m assuming you want them to go through the same forward prop i.e. forward() function in the model file.

Write your data loader such that you know the positions of the labelled samples and unlabelled samples. This won’t be hard. Then at the time of your loss calculation, simply re-write a custom version of CrossEntropyLoss based on Pytorch docs and for the unlabelled samples you can neglect them by not adding them to the equation.

Writing custom loss functions is not that hard so don’t worry :slight_smile:

Yes, I need the unlabeled samples to go through the model.forward() and generate an unsupervised loss together with the labeled samples in the mini batch. Thank you for the suggestion. I will try it.

1 Like