I’m missing something. Loss (Error on rare class) same as Loss (Error on common class)?

weight_inbalance=torch.tensor([1000.0,1.0]) # very rare first class

loss_inbalance = torch.nn.CrossEntropyLoss(weight=weight_inbalance,reduction=‘mean’)

- batch of 2 samples, toggle only first sample

inbal_err_on_rare = loss_inbalance(torch.tensor([[-1.,-1],[-1,1]]), torch.tensor([0,1]))

inbal_err_on_com = loss_inbalance(torch.tensor([[1.,1],[-1,1]]), torch.tensor([0,1]))

print(inbal_err_on_rare, inbal_err_on_com)

tensor(0.6926) tensor(0.6926)

The same experiment on a ballance class yeilds the same equivalence, just offset in value.

weight_balance=torch.tensor([1.0,1.0])

loss_balance = torch.nn.CrossEntropyLoss(weight=weight_balance,reduction=‘mean’)

bal_err_on_not_rare = loss_balance(torch.tensor([[-1.,-1],[-1,1]]), torch.tensor([0,1]))

bal_err_on_com = loss_balance(torch.tensor([[1.,1],[-1,1]]), torch.tensor([0,1]))

print(bal_err_on_not_rare, bal_err_on_com)

tensor(0.4100) tensor(0.4100)

I generated the contours, and they look the same, just shifted.

weight_inbalance=torch.tensor([1000.0,1.0]) # very rare first class

weight_balance=torch.tensor([1.0,1.0])

loss_inbalance = torch.nn.CrossEntropyLoss(weight=weight_inbalance,reduction=‘mean’)

loss_balance = torch.nn.CrossEntropyLoss(weight=weight_balance,reduction=‘mean’)

X = np.arange(-1., 1.1, 0.1)

Y = np.arange(-1., 1.1, 0.1)

Z_inbalance = np.zeros((X.shape[0],Y.shape[0]))

Z_balance = np.zeros((X.shape[0],Y.shape[0]))

#print(X.shape, Y.shape, Z.shape)i = -1

for x in X:

i+=1

j=-1

for y in Y:

j+=1

batch=torch.tensor([[x, y], [-1, 1]], dtype=torch.float32)`X[i], Y[j], Z_inbalance[i, j] =x,y,loss_inbalance(batch, torch.tensor([0,1])) X[i], Y[j], Z_balance[i, j] =x,y,loss_balance(batch, torch.tensor([0,1]))`

fig, ax = plt.subplots(2)

CS0 = ax[0].contour(Y,X,Z_inbalance,cmap=cm.coolwarm)

ax[0].clabel(CS0, inline=1, fontsize=14)

#ax[0].set_title(‘Inbalanced Classes’)

ax[0].set_ylabel(‘true class is rare’)

#ax[0].set_xlabel(“not selected class is Common”)CS1 = ax[1].contour(Y,X,Z_balance,cmap=cm.coolwarm)

ax[1].clabel(CS1, inline=1, fontsize=14)

#ax[1].set_title(‘Balanced Classes’)

ax[1].set_ylabel(‘true class is common’)

ax[1].set_xlabel(“not selected class is Common”)

So either I am missing the point of weights, or made a newbie error. Either way, I would appreciate some comments.