Weight parameter in nn.CrossEntropyLoss

vision kornia
#1

  1. If the accuracy for each class of my model is [0.7, 0.8, 0.2], can I improve the performance of the last class by applying the weight as [1,1,2] in cross-entropy ?

  2. When I applied the weight as [1,1,2], I thought the loss would be different from that of ces1. But it gives the same value. Why is that?

ces1 = nn.CrossEntropyLoss(reduction='none')

class_weights = [1,1,2]
class_weights = torch.FloatTensor(class_weights).cuda()
ces2 = nn.CrossEntropyLoss(weight=class_weights, reduction='none')
Compute with ces1 : [0.3, 0.4, 0.4]
Compute with ces2 : [0.3, 0.4, 0.4]
#2

  1. The weighting should balance the losses and might thus also balance the accuracy. I.e. while the class2 accuracy might increase the other two might decrease.

  2. Your current target might not use the class2 index. This code shows the difference:

class_weights = torch.tensor([1., 1., 2.])
ces1 = nn.CrossEntropyLoss(reduction='none')
ces2 = nn.CrossEntropyLoss(weight=class_weights, reduction='none')

x = torch.randn(2, 3)
target = torch.tensor([0, 2])

loss1 = ces1(x, target)
> tensor([0.2583, 0.1567])
loss2 = ces2(x, target)
> tensor([0.2583, 0.3133])
1 Like