Is there any way to completely ignore samples with a particular label when calculating loss?

Hi there,
my problem is as follows:
I have a mixture of Labelled Sample Group and Unlabeled Sample Group. For some special reason, I have to mix those samples in a same mini-batch to train a classification network, with CrossEntropyLoss . So I need to ignore these unlabeled samples when computing the CrossEntropyLoss.

I tried to assign a pseudo label to these unlabeled samples and set the option weight in CrossEntropyLoss to be ( zero ) corresponding to the pseudo label , but found that it wouldn’t satisfy my demand. Although the loss on the pseudo class is ignored, if the prediction of an unlabeled data have large values on other indexes, it still generate large loss. I need those samples contribute exactly 0 to the CrossEntropyLoss.

So will someone kindly help me with this issue?
Millions of thanks in advance.

Hello,

Indeed setting the weight in CrossEntropyLoss set weights with respects to the classes which seems not to be what you need.
Not sure my solution could help you but in my opinion you could create 3 datasets, one with labelled, one with unlabelelled and one that return the 2. You can create them via module Dataset of Pytorch, you can make a call only on the labelled data to compute the crossentropyloss, something like:

mixed_dataset = MixedDataset() #the __getitem__() function return ( labeleddata, unlabeleddata)
labeled_dataset = LabeledDataset()
unlabelled_dataset = UnlabeledDataset()

for data in mixed_dataset:
     whatever you need
     for labeled in labeled_dataset:
             compute loss
    whatever you need

just be sure to not mix the index and always return the corresponding data

Thanks for your reply.

I’m a little confused that there are two for loops to get 2 batch of data. Can you explain more about that?

In my case, both the labeled data and unlabeled data are mixed and shuffled during training, forming a minibatch to do forward propagation.

Hi SoucheChapich, thanks for your reply.

I’m a little confused that there are two for loops to get two batches of data. Can you explain more about that?

In my case, both the labeled data and unlabeled data are mixed and shuffled during training

I notice that there exists a parameter called ignore_index in nn.CrossEntropyLoss(), but it seems that ignore_index works similarly to zero weight and cannot solve the problem that misclassified
unlabeled data still contribute to the final loss

Hello,

I was just thinking about a solution for your case, I think that something like that is possible.

How did you define your dataset ? Are you using data.Dataset class from pytorch ? If yes, you should be able to specify whatever you want. I mean, you can do somehing like:

class mixedDataset(data.Dataset):
     def __getitem__(self, index):
           return your combined_data

    def __getlabeleddata_(self,index):
           return your labeled data 

and after you can iterate through your mixedDataset and calling your getlabeldata function to get the labeled data.

Yes, I use Dataset and dataloader class from pytorch. You can viewed the model comprising a feature extractor and a classifier which works in an E2E manner. For both labeled and unlabeled data mixed together to do forward propagation, however, you can view it as we use the whole data for feature extraction, but only labeled data was used for classificaion.