I have this dataset which contains some 10000 shirts and their color (blue, black,red…) and pattern (checks, stripes, solid…). There are total 12 different classes in color and 8 in pattern. I am able to make a model of resnet18 (training all layers) to identify color and another resnet18 to identify pattern. For these two models I use Cross-entropy loss and the results are good. There are 12 outputs for color-resnet and 8 outputs for pattern-resnet.
But when I want to do both things using single resnet (12+8 outputs), using BCEwithlogits loss, the network is not giving proper results (infact the results are very bad) . Any idea as why is this happening?
You should CrossEntropyLoss for the other case also. In general, whenever you have a classification problem you should use CrossEntropyLoss due to various statistical reasons. BCEWithLogitsLoss is not suitable for that purpose. It is generally used in auto-encoders when you want to compare the reconstruction to the original image.
Using two different heads for both modalities and nn.CrossEntropyLoss sounds like a good idea.
However, for the sake of completeness, nn.BCELoss might also be used in a multi-label classification use case, i.e. when multiple classes are active for a single sample. In this particular case, @Kushaj’s suggestion could yield better results, so let us know if you were successful. 
1 Like