Looking at several denoiser autoencoder, one trains the network with images to which one adds noise, these images serve as input to the network while the original images are the target. Well, to prepare the data set, usually the images are normalized to [0,1], and the noisy images are cliped (clamped) to [0,1] also. And this normalization is done in conjunction of the fact that the last layer of the network is using ‘torch.sigmoid’.
So far so good, but the sigmoid(x) function reaches 0, for large negative value of x, and symetricaly, the function reaches 1 for large positive value of x. In a certain sense, reaching 0,1 demand +/- infinity values.
So, my question, does the normalization [0,1] suited to sigmoid(last layer) and vice versa if one uses sigmoid does the [0,1] is the best normalisation (eg. 0.02, 0.98 may be better ?) ?
It is up to you to comment…
Best.