Hi, everybody, I met a strange problem when trying to define a weighted cross entropy myself.
My loss func is simple:
def weighted_BCE(sigmoid_x, targets): #this is a weighted Binary Cross Entropy assert sigmoid_x.size() == targets.size() #make sure input and label have same shape count_p = (targets == 1.0).sum() + 1 count_n = (targets == 0.0).sum() + 1 #count 1(positive) and 0(negative), add 1 to avoid 0 loss = -((targets * sigmoid_x.log()) * (1 / count_p)) - (((1 - targets) * (1 - sigmoid_x).log()) * (1 / count_n)) #divide the positive part and negative part with their count respectively return loss.mean()
then try it with a random input and all 0 label
if __name__ == '__main__': a = np.random.uniform(low = 0.0, high = 1.0, size = (3, 3)) a = torch.from_numpy(a).float().to('cuda:0') a.requires_grad = True b = np.zeros((3, 3)) b = torch.from_numpy(b).float().to('cuda:0') b.requires_grad = False loss = weighted_BCE(a, b) loss2 = torch.mean(-((b * a.log()) * (1 / 1)) - (((1 - b) * (1 - a).log()) * (1 / 10))) # calculate the loss in the same way without definition print(loss.item()) print(loss2.item()) print((loss - loss2).item())
# here is the result 0.0 0.15303052961826324 -0.15303052961826324 #this is the supposed to be zero
It may be a silly question, but I can’t figure out the reason why the loss func returns 0.0 in a definition and different result when calculate it in the main func.
My environment is
Os : Ubuntu 18.04.2 LTS GPU: GTX 1080ti pytorch: 1.2.0