I’m implementing VAE(Variational autoencoder) with a prior different from unit Gaussian. Unfortunately I keep getting nans after every few epochs. I have tried reducing the learning rate and clipping gradient.
Any help would be greatly appreciated. Thanks.
First, can you make sure there are no NANs in your data? Sometimes this can accidentally happen during data augmentation.
I.e., just before the forward pass, please print:
torch.sum(torch.isnan(data))