MSELoss always produces 0 - what am I doing wrong?

I’m a pytorch neophyte so please be gentle. :grin:

I have trained a model to predict a 5-element output from an input image. The 5 elements must sum to 1.0 so I am using softmax on the output from the last fully connected layer, and using the MSELoss to assess discrepancy between predicted and actual values.

The training process seems to be working well, with the loss gradually declining as expected. However, when I try to validate using new images, the MSELoss is always 0, even though I can see that the predictions are far from perfect. The code is very similar to the training so I am quite perplexed.

Here’s the relevant code:


criterion = nn.MSELoss()
with torch.no_grad():
    for i, data in enumerate(valid_generator, 0):
        # get the inputs; data is a dictionary of [inputs, labels]
        inputs = data['inputs']
        labels = data['labels']
        outputs = net(inputs)
        errortensor = criterion(outputs, labels)
        loss = errortensor.item()
        print(outputs)
        print(labels)
        print('loss is %d' % (loss))
        if (loss < minloss):
            minloss = loss
        if (loss > maxloss):
            maxloss = loss
        sumloss = sumloss + loss
        total += labels.size(0)
        imagecount += batch_size
        print('---------')

And here is a snippet of the output:

---------
tensor([[0.0718, 0.1101, 0.0555, 0.0925, 0.6701]])
tensor([[0.0000, 0.1000, 0.1000, 0.1000, 0.7000]])
loss is 0
---------
tensor([[0.0823, 0.0229, 0.0689, 0.1235, 0.7024]])
tensor([[0.0000, 0.1000, 0.1000, 0.1000, 0.7000]])
loss is 0
---------
tensor([[0.0836, 0.1115, 0.0604, 0.0858, 0.6588]])
tensor([[0.0000, 0.1000, 0.1000, 0.1000, 0.7000]])
loss is 0
---------
tensor([[0.0707, 0.0771, 0.0481, 0.0745, 0.7296]])
tensor([[0.0000, 0.1000, 0.1000, 0.1000, 0.7000]])
loss is 0
---------

Why is the loss always zero? (I suspect it’s something simple I am missing.)

Thanks!

Hi Sally!

This is a python issue, rather than anything specific to pytorch.

%d formats your value as an integer, truncating any fractional part.

By construction, assuming that your labels are between 0 and 1,
your loss will (almost) always be less than one and be truncated
to zero.

That is, MSELoss is not producing 0, but you’re printing it out as 0.

Best.

K. Frank

oh… how embarrassing! Thank you! (I am also new to Python…)