How to interpret the output of a single neuron?

Hi, I’m currently using CrossEntropyLoss for a model with a single neuron as ouptut.

My predictions have the following pattern:

        [ 0.0445],
        [ 0.2206],

To get my classes, do I have to do:

      pred_batch = torch.sigmoid(pred_batch)
      pred_batch[pred_batch >= 0.5] = 1
      pred_batch[pred_batch < 0.5] = 0


      pred_batch[pred_batch >= 0] = 1
      pred_batch[pred_batch < 0] = 0

Also, I’m getting some negative loss for my training set, is that normal?

Firstly, nn.CrossEntropyLoss() is mostly used when you need multple predictions.
Since you have just one output neuron, you must use nn.Sigmoid() followed by nn.BCELoss() or simply nn.BCELossWithLogits().

Just need a clarification,

  1. You have single output neuron.
  2. Your target spans between 0 and 1.

Consider a batch-size of 7.

  1. pred_batch will be of shape (7,1) since u have single neuron.
  2. target shape will be (7,) with minimum value of 0 and maximum value of 1.
    Right ?

How come you are able to pass these inside CrossEntropyLoss() without getting any error.?

Good question, I have no idea.

When I use BCELossWithLogits() I had to do a
label_batch = label_batch.view(-1, 1)

Didn’t BCELosswithLogits() do the purpose then ?
If you were using this bce-with-logits, then torch.sigmoid() is the right way as you have posted in the question.
If you’ve used nn.CrossEntropyLoss(), then u apply Softmax(), and pick the class which has the highest value.
In your case, picking the class with highest value would always result in the same class ( zeroth class ), since you have just one neuron.

Here’s my code:

    model = models.resnet50(pretrained=pretrain_status)
    num_ftrs = model.fc.in_features
    model.fc = nn.Sequential(
      nn.Linear(num_ftrs, 1))
  criterion = nn.BCELossWithLogits() 
  optimizer = torch.optim.SGD(model.parameters(), lr =lr)
  for image_batch, label_batch in tqdm(train_dl):
    image_batch, label_batch =,
    pred_batch = model(image_batch)
    label_batch = label_batch.view(-1, 1)

    batch_loss = criterion(pred_batch.double(), label_batch.double())
    total_train_loss += batch_loss.item()


    pred_batch[pred_batch >= 0] = 1
    pred_batch[pred_batch < 0] = 0

I know there’s a mistake in there but I can’t seem to find it

Hi @alx,
the code is perfectly fine upto optimizer.step()

Please add these steps after that line

probabilities  = torch.sigmoid(pred_batch)
pred_batch[probabilities>=0.5] = 1
pred_batch[probabilities<0.5]  = 0

This is the first case which you have asked in your question.!