Hello, I am doing textclassification (3 classes) with huggigface library and Would like to know which loss function should use ? I used crossentropyloss but since reading forums , I found categorical cross entropy.

So which one is better ?

Hello, I am doing textclassification (3 classes) with huggigface library and Would like to know which loss function should use ? I used crossentropyloss but since reading forums , I found categorical cross entropy.

So which one is better ?

Hi Emmanuelle!

The choice will depend on how your data is â€śannotated.â€ť

If your ground-truth labels (annotations) are integer class labels, then

you would use what I believe you are calling â€ścategorical cross entropy.â€ť

That is, each sample in your data set would be labelled with a single

integer â€“ `0`

for class-0, `1`

for class-1, and `2`

for class-2.

If your labels are probabilistic â€śsoftâ€ť labels, then you would use the

more general full cross entropy.

In this case, each sample would be labelled with three floating-point

probabilities, `P[c]`

, where `P[0]`

would be the probability of that sample

being in class-0, `P[1]`

the probability of being in class-1, and so on.

As of the latest stable release, 1.10.0, pytorchâ€™s `CrossEntropyLoss`

supports both categorical and probabilistic labels.

But again, the kind of labels you have determines which version of

cross-entropy loss you should use.

Best.

K. Frank

1 Like