Custom Loss penalizing similar categegories more strong

Hi everyone,

I’m designing at the moment my network, which shall classify images relating how many people are in there. So my output dimension is 6 - 0 … 5 people. I’m observing now that the most errors are happening in cases where 4 people are on the image but the network classifies it as 3 or 5. As one idea I would like to define my loss (at the moment I’m using CrossEntropyLoss) in a manner that these cases are more weighted in compare to other cases.

How can you do this?

Thanks and best regards
Jonas