Penalize some classes more than others in cross entropy (but only for a specific class!)

[KFrank]

Hi Rohit!

@steeltrooper17

How would I achieve this?

If I understand your question, you will need a custom loss function.

I believe that your use case is addressed by this post (and its entire thread):

Create a Variation of Cross Entropy Loss with Per Pair of Classes Weights

Hi David! Cross entropy (by definition) doesn’t work this way. But you can write some other loss function that does have “per-pair” penalties. I give a couple of possibilities in the example script, below. misclassA() just weights the probability for an incorrect prediction by the per-pair weight given in your matrix D. My guess is that such a loss function won’t train very well. In my experience, the logarithmic divergence for bad predictions in cross entropy seems to be very helpfu…

Best.

K. Frank