BCELoss for MultiClass problem

vision
1

Is it a possibility to calculate the Multiclass crossentropy loss by successively using the nn.BCELoss() implementation

This is what I have tried.

# Implementation of the Multiclass Cross Entropy classification
def SegLossFn(predictions,targets):
    _, c, _, _ = predictions.size()
    loss=0
    m=nn.Sigmoid()
    loss_fn=nn.BCELoss()
    #BCE-> MCE by adding for each of the classes BCE
    for i in range(c):
        loss+=loss_fn(m(predictions[0][i]),Variable(targets[i][0]).cuda())
            
    return loss
2

For multi-class classification you would usually just use nn.CrossEntropyLoss, and I don’t think you’ll end up with the same result, as you are calling torch.sigmoid on each prediction.

For multi-label classification, you might use nn.BCELoss with hot-encoded targets and won’t need a for loop.

Could you explain your use case a bit as I’m currently not sure to understand it properly?

3 Likes
3

My question is about the the addition operation and how it is handled. What happens the grad_fn? how do they get added?

4

The losses will get accumulated and your loss tensor will get grad_fn=<AddBackward0> as its grad_fn.

1 Like
5

The problem for not using nn.CrossEntropyLosswas my predicted output is of size [N,C,H,W] and the target is of [N,C,H,W]. [H,W] in the C channel of the target is 0 or 1 based on presence or absence of class C. Is their a work around?

6

nn.CrossEntropyLoss expects a torch.LongTensor containing the class indices without the channel dimension. In your case, you could simply use:

targets = torch.argmax(targets, 1)

to create your target tensor.

1 Like
7

I’m not sure if there is so called “Multiclass Cross Entropy”.
But doing binary classification for each class makes sense, so I think what you have tried is correct.

8

It’s usually called multi-category cross entropy but yeah, the CrossEntropyLoss is essentially that. Just be careful, the CrossEntropyLoss takes the logits as inputs (before softmax) and the BCELoss takes the probabilities as input (after logistic sigmoid)

1 Like
9

Thanks, I think I confuse multi-class with multi-label, where they do multiple BCE like that

10

I don’t think you confused anything, because both multi-label cross entropy and binary cross entropy work for dealing with multi-class problems. The difference though is that in binary cross entropy, mathematically, you assume that the classes are independent.

11

i think this particular implementation has some problem beause when I do loss.backward() improvement is seen only in the last class. The other previous channels just have complement of the last class.

For example:
Expected Output

[[1, 0],             [[0, 1],           [[0, 0],
 [0, 0]]              [0, 0]]            [1, 1]]

Got Output

[[1, 1],             [[1, 1],           [[0, 0],
 [0, 0]]              [0, 0]]            [1, 1]]
12

how can i use MSEloss for multi class?

13

You could try to apply a softmax as the last activation function in your model and use nn.MSELoss directly on these outputs and the one-hot encoded targets.
Alternatively, you could also use a single output unit, which could be used to predict the class “index” as a floating point number and calculate the loss using the targets containing class indices.

However, note that nn.MSELoss is not the usual loss function you would use for a classification use case.

14

tnx
if I do soft-max and one-hot, the final loss is a vector with length equal to the classes count
and in this case I cant do backwards because it works on single value:

def weighted_mse_loss(input,target,weights):
    out = (input-target)**2
    out = out * weights.expand_as(out)
    loss = out.sum(0)
    return loss
15

You could either pass the gradients in the same shape as the output (torch.ones_like(loss) would be the default) or reduce the loss to a scalar.

By default the loss functions will calculate the loss average and will return a scalar.