I tried to implement a convolutional auto encoder in pytorch and applied it on MNIST. Unfortunately the train loss makes almost no progress (it does not decrease) during training. Can someone please help? I have posted my code (< 95 lines) at the following Github Gist: code is here.
Hi @rpathak1,
could you try to remove the normalization from the transformations in your Dataset?
While this is generally a good idea for classification use cases, since it will standardize your data to have a mean of zero and a stddev of 1, this might cause some troubles for your autoencoder, since you are using a sigmoid at the output layer, which limits the output values to the range [0, 1].
Alternatively, you could try to remove the sigmoid and let your model learn the standardized inputs.
Let me know, if that helps.
1 Like