KLDivLoss for binary classification

Hi, I have two outputs from two models of form logit, and I’m using BCEWithLogitsLoss for loss#1 for each output, and KLDivLoss for loss#2 between the two outputs. The following code for KLDivLoss gives me the following error. I was wondering what the problem is? Thanks!

The code:

loss = KLDivLoss(torch.sigmoid(output1), torch.sigmoid(output2))

The error:

if size_average and reduce:
RuntimeError: Boolean value of Tensor with more than one value is ambiguous

You would need to create the criterion object first before calling it, while both tensors are used as input arguments to the KLDivLoss.__init__ method in your code.

loss = nn.KLDivLoss()(input, target)
# or 
criterion = nn.KLDivLoss()
loss = criterion(input, target)

should work.
Also note this section of the docs:

As with NLLLoss, the input given is expected to contain log-probabilities and is not restricted to a 2D Tensor. The targets are interpreted as probabilities by default, but could be considered as log-probabilities with log_target set to True.

as you were using torch.sigmoid for both tensors in your code.

1 Like

Thank you @ptrblck! For the second part of your answer, my understanding is that I should use LogSigmoid(output1) and LogSigmoid(output2) instead of sigmoid(output1) and sigmoid(output2). Am I right?

The targets should contain probabilities by default unless you use log_target=True.

1 Like