I am trying to train an autoencoder with BCE loss on MNIST. It gives me nan out, even if I apply softmax on the labels. Can anyone post a minimum working code with BCE loss ?
Solved by passing both input and output though a softmax layer, then through BCE loss.
I have the excat same problem, can you peovide sample code ?
It is highly possible the values passed to the input variable is either 0
exactly, and the log at 0
is not defined. Use nn.BCELoss(input + eps, target)
where eps
is 1e-10
or less.
1 Like
I am trying to adapt this:
https://github.com/melissa135/Convolutional_AutoEncoder which uses MSELoss to using BCE loss for Binary classification without success
Or this: https://github.com/melissa135/Convolutional_AutoEncoder
Which does NOT use the labels (but I need the labels)
This is a working example:
class ConvAE(nn.Module):
def __init__(self):
super(ConvAE, self).__init__()
self.encoder = nn.Sequential(
nn.Conv2d(2, 256, 3, stride=3, padding=1), # b, 16, 10, 10
nn.BatchNorm2d(256),
nn.ReLU(True),
nn.MaxPool2d(2, stride=2), # b, 16, 5, 5
nn.Conv2d(256, 16, 3, stride=2, padding=1), # b, 8, 3, 3
nn.ReLU(True),
nn.BatchNorm2d(16),
nn.MaxPool2d(2, stride=1), # b, 8, 2, 2,
nn.AvgPool2d(2, 2)
)
self.decoder = nn.Sequential(
nn.ConvTranspose2d(16, 256, 3, stride=2), # b, 16, 5, 5
nn.ReLU(True),
nn.ConvTranspose2d(256, 8, 5, stride=3, padding=1), # b, 8, 15, 15
nn.ReLU(True),
nn.ConvTranspose2d(8, 1, 2, stride=2, padding=1), # b, 1, 28, 28
nn.Tanh()
)
self.sig = nn.Sigmoid()
self.fc = nn.Linear(784, 1)
def forward(self, x):
x = self.encoder(x)
# print(x.data.size())
# x = x.view(x.size(0), -1)
x = self.decoder(x)
# print(x.data.size())
x = x.view(x.size(0), -1)
# print (x.data.size())
x = self.fc(x)
x = self.sig(x)
return x