So, I run the same code with a 50/50 split of 0 and 1 label, I get aboyt 70% accuracy on val set and my val preds are not stuck at 0.
However, when I run the code on a dataset with 84/16 % split of labels 0 and 1, all my val preds end up being 0. I used both cross-entropy loss as well as BCEWithLogitLoss with a weight vector (though I am not sure if I set the weight vector correctly). Also, is weight = torch.tensor([0.84, 0.16])
correct?
How can I fix this problem?
loss_type = 'BCEWithLogitsLoss'
if loss_type == 'BCEWithLogitsLoss':
self.criterion = nn.BCEWithLogitsLoss(reduction='none') # weighted loss for imbalanced dataset
#self.criterion = nn.BCEWithLogitsLoss()
elif loss_type == 'CrossEntropyLoss':
self.criterion = nn.CrossEntropyLoss() # this should work for binary classification
if loss_type == 'BCEWithLogitsLoss':
labels = torch.as_tensor(labels, dtype=torch.float32) # we need float labels for BCEWithLogitsLoss
weight = torch.tensor([0.84, 0.16]) # how to decide on this weights?
#weight = torch.tensor([0.5, 0.5])
weight_ = weight[labels.data.view(-1).long()].view_as(labels)
m = nn.Sigmoid()
with torch.cuda.amp.autocast():
loss = self.criterion(m(out[:,1]-out[:,0]), labels.cuda())
loss_class_weighted = loss * weight_.cuda()
loss_class_weighted = loss_class_weighted.mean()
loss = loss_class_weighted
elif loss_type == 'CrossEntropyLoss':
labels = torch.as_tensor(labels)
with torch.cuda.amp.autocast():
loss = self.criterion(out, labels.cuda())
pred_labels = out.data.max(1)[1]
#pred_labels = out.argmax(dim=1)
labels = labels.int()
return pred_labels, labels, loss