Penalize FN in binary cross enthropy

vision
1

Hi all,

I am trying to implement a weighted binary cross entropy loss function with dice loss, basically:

total_loss = Weighted_bce_loss + dice_loss

I am using the code below: (SR - segmentation result, GT - ground truth)

def bce_dice_loss(GT, SR):

SR.flatten()
GT.flatten()
criterion = torch.nn.BCELoss()

loss = criterion(GT, SR) + dice_coef_loss(GT, SR)
return loss

def dice_coef_loss(GT, SR):
SR.flatten()
GT.flatten()
loss = 1 - dice_coeff(GT, SR)
return loss

def dice_coeff(GT, SR):

smooth = 1.
GT.flatten()
SR.flatten()
production = GT * SR
intersection = production.sum()

return (2. * intersection + smooth) / (GT.sum() + SR.sum() + smooth)

I am doing an medical image segmentation and I deal with a highly imbalanced dataset. In my error loss function (in the BCE part) I want to penalize False Negatives in order to get a higher Recall value.

Does anyone has an idea how to proceed with this?

Thank you in advance :slight_smile:

2

You could use nn.BCEWithLogitsLoss instead of nn.BCELoss (remove the sigmoid therefore) and set the pos_weight > 1 to increase the recall.

3

This is my train loop:

def per_epoch_train(self,epoch):

avg_meter = defaultdict(float)
pbar = tqdm(enumerate(self.train_loader), total=len(self.train_loader), desc="Epoch {}".format(epoch), ncols=0)
self.attUnet.train(True)
meter = {}

acc_detection = 0.
recall_detection = 0.
precision_detection = 0.
DC_detection = 0.
    
length = 0
avg_loss = 0

for i, data in pbar:
  images = data[0]
  GT = data[1]
  
  images, GT = Variable(images.cuda()), Variable(GT.cuda())
  
  SR = self.attUnet(images)
  SR_probs = torch.nn.functional.sigmoid(SR)

  length += images.size(0)
  
  loss = bce_dice_loss(SR_probs, GT)
  avg_loss += loss.item()
  
  self.optimizer.zero_grad()
  loss.backward()
  self.optimizer.step()

Does this mean I have to get rid of this line to remove Sigmoid function:

  SR_probs = torch.nn.functional.sigmoid(SR)

And also by set the pos_weight > 1, is this the right implementation:
pos_weight = torch.FloatTensor([1, 1.2])
criterion = torch.nn.BCEWithLogitsLoss(pos_weight)
loss = criterion(GT, SR)

Thank you a lot for your help!!

4

HI!

I am getting an error in the pos_weight declaration. How i am supposed to use it?

5

If you are working with a binary classification task, you should pass a single value:

criterion = nn.BCEWithLogitsLoss(pos_weight=torch.tensor(5.))
x = torch.randn(10, 1)
y = torch.randint(0, 2, (10, 1)).float()

loss = criterion(x, y)
6

I am doing pixel wise classification for medical image segmentation. How should I use pos_weight in this case?