Info about binary cross entropy with logits

nivter · April 1, 2019, 9:45am

The function torch.nn.functional.binary_cross_entropy_with_logits actually returns a call to the function torch.binary_cross_entropy_with_logits. But I can’t find any information about it. It doesn’t have any docstring either. What is the actual code that is called and how is it called?

tenzo · July 18, 2019, 10:31am

Hi, this is pretty old question but I think I found its definition:

github.com

pytorch/pytorch/blob/35fed93b1ef05175143f883c6f89f06c6dd9429b/aten/src/ATen/native/Loss.cpp#L96-L112


Tensor binary_cross_entropy_with_logits(const Tensor& input, const Tensor& target, const Tensor& weight, const Tensor& pos_weight, int64_t reduction) {
    Tensor loss;
    auto max_val = (-input).clamp_min_(0);
    if (pos_weight.defined()) {
        // pos_weight need to be broadcasted, thus mul(target) is not inplace.
        auto log_weight = (pos_weight - 1).mul(target).add_(1);
        loss = (1 - target).mul_(input).add_(log_weight.mul_(((-max_val).exp_().add_((-input - max_val).exp_())).log_().add_(max_val)));
    } else {
        loss = (1 - target).mul_(input).add_(max_val).add_((-max_val).exp_().add_((-input -max_val).exp_()).log_());
    }


    if (weight.defined()) {
        loss.mul_(weight);
    }


    return apply_loss_reduction(loss, reduction);
}

solsol · January 13, 2020, 6:39am

Hi,

My problem with using binary cross entropy is due to input and target shape.
input size is [n,m] however target size is [n]. By upsampling target size to [n,m], I got “zero-Devision-Error-scaled” error in backward. Any help please?