Cross Entropy Loss between 3d tensors

[KFrank]

Hello Matheus!

@mfcs

But how the Cross entropy Loss perform the computation between a probability distribution and a class value instead of the probability of the class?

I won’t comment on whether KL-divergence or cross-entropy is
likely to be the better loss function for your use case.

But (assuming I understand what you are asking), no, you can’t
use pytorch’s built-in CrossEntropyLoss with probabilities for
targets (sometimes called soft labels, a term I don’t much like).
It requires integer class labels (even though cross-entropy makes
perfect sense for targets that are probabilities).

However, you can write your own without much difficulty (or loss
of performance). See this post:

Soft Cross Entropy Loss (TF has it does Pytorch have it)

Hello Raaj! I do not believe that pytorch has a “soft” cross-entropy function built in. But you can implement it using pytorch tensor operations, so you should get the full benefit of autograd and gpu acceleration. See this (pytorch version 0.3.0) script: import torch torch.__version__ # define "soft" cross-entropy with pytorch tensor operations def softXEnt (input, target): logprobs = torch.nn.functional.log_softmax (input, dim = 1) return -(target * logprobs).sum() / input.shap…

Best.

K. Frank