BCE loss returning NaN

I am trying to train an autoencoder with BCE loss on MNIST. It gives me nan out, even if I apply softmax on the labels. Can anyone post a minimum working code with BCE loss ?

Solved by passing both input and output though a softmax layer, then through BCE loss.

I have the excat same problem, can you peovide sample code ?

It is highly possible the values passed to the input variable is either 0 exactly, and the log at 0 is not defined. Use nn.BCELoss(input + eps, target) where eps is 1e-10 or less.

1 Like

I am trying to adapt this:
https://github.com/melissa135/Convolutional_AutoEncoder which uses MSELoss to using BCE loss for Binary classification without success

Or this: https://github.com/melissa135/Convolutional_AutoEncoder
Which does NOT use the labels (but I need the labels)

This is a working example:

class ConvAE(nn.Module):
    def __init__(self):
        super(ConvAE, self).__init__()
        self.encoder = nn.Sequential(
            nn.Conv2d(2, 256, 3, stride=3, padding=1),  # b, 16, 10, 10
            nn.BatchNorm2d(256),
            nn.ReLU(True),
            nn.MaxPool2d(2, stride=2),  # b, 16, 5, 5

            nn.Conv2d(256, 16, 3, stride=2, padding=1),  # b, 8, 3, 3
            nn.ReLU(True),
            nn.BatchNorm2d(16),
            nn.MaxPool2d(2, stride=1),  # b, 8, 2, 2,
            nn.AvgPool2d(2, 2)
        )
        self.decoder = nn.Sequential(
            nn.ConvTranspose2d(16, 256, 3, stride=2),  # b, 16, 5, 5
            nn.ReLU(True),
            nn.ConvTranspose2d(256, 8, 5, stride=3, padding=1),  # b, 8, 15, 15
            nn.ReLU(True),
            nn.ConvTranspose2d(8, 1, 2, stride=2, padding=1),  # b, 1, 28, 28
            nn.Tanh()
        )

        self.sig = nn.Sigmoid()
        self.fc = nn.Linear(784, 1)

    def forward(self, x):
        x = self.encoder(x)
        # print(x.data.size())
        # x = x.view(x.size(0), -1)
        x = self.decoder(x)
        # print(x.data.size())
        x = x.view(x.size(0), -1)
        # print (x.data.size())
        x = self.fc(x)
        x = self.sig(x)

        return x