KLDivLoss for binary classification

vision
1

Hi, I have two outputs from two models of form logit, and I’m using BCEWithLogitsLoss for loss#1 for each output, and KLDivLoss for loss#2 between the two outputs. The following code for KLDivLoss gives me the following error. I was wondering what the problem is? Thanks!

The code:

loss = KLDivLoss(torch.sigmoid(output1), torch.sigmoid(output2))

The error:

if size_average and reduce:
RuntimeError: Boolean value of Tensor with more than one value is ambiguous
2

You would need to create the criterion object first before calling it, while both tensors are used as input arguments to the KLDivLoss.__init__ method in your code.

loss = nn.KLDivLoss()(input, target)
# or 
criterion = nn.KLDivLoss()
loss = criterion(input, target)

should work.
Also note this section of the docs:

As with NLLLoss, the input given is expected to contain log-probabilities and is not restricted to a 2D Tensor. The targets are interpreted as probabilities by default, but could be considered as log-probabilities with log_target set to True.

as you were using torch.sigmoid for both tensors in your code.

1 Like
3

Thank you @ptrblck! For the second part of your answer, my understanding is that I should use LogSigmoid(output1) and LogSigmoid(output2) instead of sigmoid(output1) and sigmoid(output2). Am I right?

4

The targets should contain probabilities by default unless you use log_target=True.

1 Like