Data Normalization in MNIST

Hi why do we need data normalization in MNIST Data Loader example ?

Thank you

Normalizing the data usually is beneficial for training the model.
It’s sometimes not necessary, but might stabilize and accelerate the training.

2 Likes

Thank you @ptrblck in order to have all the data sets ranges between 0 and 1 what parameters should I use while normalizing ? or even most of them are 0 and some of them are 1 ?

If you are loading PIL.Images, torchvision.transforms.ToTensor() will create a normalized tensor with values in the range [0, 1].
torchvision.transforms.Normalize() will create standardized tensors with zero mean and a unit variance.
Usually you would use the mean and standard deviation from the training set.
However, often the ImageNet statistics are used for RGB images, especially if you are using a pretrained model (same input statistics) and if your dataset is similar to ImageNet.

1 Like

Thankyou @ptrblck if I understood correctly there is no need for normalization since it will be automatically between 0 and 1 ?

Normalizing the data additionally to zero mean and unit variance could further help in training.

2 Likes

@ptrblck yes but using 0 as mean will create some negative values (since I tied to positive values only)?

Yes, this would produce positive and negative values. Have a look at this wiki for more information.

2 Likes