# Neural network binary classification softmax logsofmax and loss function

**URL:** <https://discuss.pytorch.org/t/neural-network-binary-classification-softmax-logsofmax-and-loss-function/145532>\
**Category:** Uncategorized\
**Created:** [March 3, 2022, 4:01pm UTC](https://discuss.pytorch.org/t/neural-network-binary-classification-softmax-logsofmax-and-loss-function/145532 "2022-03-03T16:01:02Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![ni\_tempe](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/n/e9c0ed/32.png) [@ni\_tempe](https://discuss.pytorch.org/u/ni_tempe)\
**Post date:** [March 3, 2022, 4:01pm UTC](https://discuss.pytorch.org/t/neural-network-binary-classification-softmax-logsofmax-and-loss-function/145532/1 "2022-03-03T16:01:03Z")

</div>

I am building a binary classification where the class I want to predict is present only \<2% of times.

The last layer could be `logosftmax` or `softmax`.

`self.softmax = nn.Softmax(dim=1)` or `self.softmax = nn.LogSoftmax(dim=1)`

my questions

1. I should use `softmax` as it will provide outputs that sum up to 1 and I can check performance for various prob thresholds. is that understanding correct?

2. if I use `softmax` then can I use `cross_entropy` loss? [This](https://machinelearningmastery.com/how-to-choose-loss-functions-when-training-deep-learning-neural-networks/) seems to suggest that it is okay to use

3. if i use `logsoftmax` then can I use `cross_entropy` loss? [This](https://stackoverflow.com/questions/64494819/use-crossentropyloss-with-logsoftmax) seems to suggest that I shouldnt.

4. if I use `softmax` then is there any better option than `cross_entropy` loss?

---

<div class="post-metadata">

**Author:** ![KFrank](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/32.png) [@KFrank](https://discuss.pytorch.org/u/KFrank)\
**Post date:** [March 3, 2022, 4:49pm UTC](https://discuss.pytorch.org/t/neural-network-binary-classification-softmax-logsofmax-and-loss-function/145532/2 "2022-03-03T16:49:00Z")

</div>

Hi Ni!

> [@ni\_tempe](#):
>
> I am building a binary classification

Build a model that outputs a single value (per sample in a batch),  
typically by using a `Linear` with `out_features = 1` as the final  
layer.

This value will be a raw-score _logit._ Use `BCEWithLogitsLoss` as your  
loss criterion (and do not use a final “activation” such as `sigmoid()` or  
`softmax()` or `log_softmax()`).

> [@](#):
>
> the class I want to predict is present only \<2% of times.

Either sample your underrepresented class more heavily when training,  
e.g., about fifty times more heavily, or weight the underrepresented class  
in your loss computation by using `BCEWithLogitsLoss`’s `pos_weight`  
constructor argument with something like:

```python
criterion = torch.nn.BCEWithLogitsLoss (pos_weight = torch.tensor ([50.0]))

```

Best.

K. Frank

---

<div class="post-metadata">

**Author:** ![ni\_tempe](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/n/e9c0ed/32.png) [@ni\_tempe](https://discuss.pytorch.org/u/ni_tempe)\
**Post date:** [March 3, 2022, 5:02pm UTC](https://discuss.pytorch.org/t/neural-network-binary-classification-softmax-logsofmax-and-loss-function/145532/3 "2022-03-03T17:02:11Z")

</div>

could you answer my 4 questions? just yes or no would suffice…  
I will also look into your reply and try

Few additional questions:  
I understand your suggestion " and do not use a final “activation” such as `sigmoid()` or  
`softmax()` or `log_softmax()` )." But what should be my final activation? i looked at linear and it doesnt do anything. it is just a pass through. Could you point the exact function?