# Loss function for binary classification with Pytorch

**URL:** <https://discuss.pytorch.org/t/loss-function-for-binary-classification-with-pytorch/26460>\
**Category:** nlp\
**Created:** [October 3, 2018, 11:38am UTC](https://discuss.pytorch.org/t/loss-function-for-binary-classification-with-pytorch/26460 "2018-10-03T11:38:41Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![coyote](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/d6d6ee/32.png) [@coyote](https://discuss.pytorch.org/u/coyote)\
**Post date:** [October 3, 2018, 11:38am UTC](https://discuss.pytorch.org/t/loss-function-for-binary-classification-with-pytorch/26460/1 "2018-10-03T11:38:41Z")

</div>

Hi everyone,

I am trying to implement a model for binary classification problem. Up to now, I was using softmax function (at the output layer) together with torch.NLLLoss function to calculate the loss. However, now I want to use the sigmoid function (instead of softmax) at the output layer. If I do that, should I also change the loss function or may I still use torch.NLLLoss function?

---

<div class="post-metadata">

**Author:** ![kenmikanmi](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kenmikanmi/32/4699_2.png) [@kenmikanmi](https://discuss.pytorch.org/u/kenmikanmi)\
**Post date:** [October 4, 2018, 1:27am UTC](https://discuss.pytorch.org/t/loss-function-for-binary-classification-with-pytorch/26460/2 "2018-10-04T01:27:33Z")

</div>

Suppose following simple model:

```auto
class Net(nn.Module):
    def __init__ (self):
        super(Net, self). __init__ ()
        self.linear1 = nn.Linear(10, 10)
        self.linear2 = nn.Linear(10, 2)

    def forward(self, x):
        x = linear1(x)
        x = linear2(x)
        return x

```

you can add [`nn.LogSigmoid()`](https://pytorch.org/docs/stable/nn.html#logsigmoid) layer to get sigmoid(x):

```auto
class Net(nn.Module):
    def __init__ (self):
        super(Net, self). __init__ ()
        self.linear1 = nn.Linear(10, 10)
        self.linear2 = nn.Linear(10, 2)
        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        x = linear1(x)
        x = linear2(x)
        x = sigmoid(x)
        return x

```

If you use binary cross entropy loss, you can compute loss as:

```auto
model = Net()
y = model.forward(input)
loss = - t*log(y) - (1-t)*log(1-y) 

```

* * *

**postscript**  
Modified the LogSigmoid to Sigmoid

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [October 4, 2018, 5:41am UTC](https://discuss.pytorch.org/t/loss-function-for-binary-classification-with-pytorch/26460/3 "2018-10-04T05:41:13Z")

</div>

For the sake of completeness: you can also use `nn.Sigmoid` as the output layer and [nn.BCELoss](https://pytorch.org/docs/stable/nn.html#torch.nn.BCELoss) in case you don’t want to write the formula yourself.

---

<div class="post-metadata">

**Author:** ![coyote](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/d6d6ee/32.png) [@coyote](https://discuss.pytorch.org/u/coyote)\
**Post date:** [October 4, 2018, 2:23pm UTC](https://discuss.pytorch.org/t/loss-function-for-binary-classification-with-pytorch/26460/4 "2018-10-04T14:23:14Z")

</div>

The above comment confused me a little bit. If I want to use nn.BCELoss, should I take the log of nn.Sigmoid inside the forward function ?

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [October 4, 2018, 4:45pm UTC](https://discuss.pytorch.org/t/loss-function-for-binary-classification-with-pytorch/26460/5 "2018-10-04T16:45:59Z")

</div>

Sorry for the confusion. No, you should just use a sigmoid on your output, if you are using `nn.BCELoss`.

Also, I’m not sure @kenmikanmi’s approach will work, as the second term seems to have a small mistake.  
The second term should look like: `(1 - t) * log(1 - sigmoid(x))`, while currently the formula uses `(1 - t) * (1 - logsigmoid(x))`.

---

<div class="post-metadata">

**Author:** ![kenmikanmi](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kenmikanmi/32/4699_2.png) [@kenmikanmi](https://discuss.pytorch.org/u/kenmikanmi)\
**Post date:** [October 5, 2018, 2:17am UTC](https://discuss.pytorch.org/t/loss-function-for-binary-classification-with-pytorch/26460/6 "2018-10-05T02:17:44Z")

</div>

@coyote I’m sorry for mistake and modified the first comment.(please use nn.Sigmoid() instead)

As @ptrblck first mentioned, I also consider using nn.BCELoss is better way when you don’t use custom loss function.