How can I calculate mean and std before creating loader?

1

I’ve seen many tutorials where first a transform object is created and then it is passed as an argument to ImageFolder() to create dataset. Transform object can normalize data by picking mean and the standart deviation for every channel. An example:

transform = transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize((some_values between [0, 1]), (some_values between [0, 1]))])

train_dataset = ImageFolder(train_dir, transform)

What I don’t understand is how can we know what are mean and std values before we pass it to for example ImageFolder() class. What it looks to me is that we have to KNOW what are means and stds before we even have a chance to process dataset. Do we need to first preprocess all data, and calculate mean and std manually? Many tutorials show how to calculate mean and std after we create loaders, but isn’t it worthless to know these numbers if we already have all loaders created? Am I missing some key idea behind the logic of this or is there a way (a smart way where you wouldn’t need to separately preprocess every image) to find what mean and std for every channel are?

2

I don’t think it’s worthless, as you would then be able to use these calculated stats in your Normalize transformation.
On the other hand you could also try to use e.g. the ImageNet stats, if your data is “similar” to the ImageNet dataset and you expect the mean and stddev to be approx. the same.

1 Like
3

Reply to first statement " I don’t think it’s worthless, as you would then be able to use these calculated stats in your Normalize transformation.". But doesn’t that mean that on the 1-st step I have to create a loader, then on the 2-nd step calculate mean and std and then as a 3-rd step create a new loader with Normalize using mean and std values calculated in a previous step?

4

Hello,

Well that’s one possibility, and other possibility is to iterate over you dataset without creating a loader. Note also that this step is very fast (in comparison to the total training duration), and it helps improving the quality of your final model. As @ptrblck said, if you use real RGB images dataset, that is similar to ImageNet, you can use the same stats.

1 Like
5

Yes, you could use either this workflow (or the Dataset directly as mentioned by @omarfoq).
Note, that calculating the dataset stats would only be needed once (if you have a fixed training, validation, and test split) and you could thus reuse the stats in all future experiments (as is done with the ImageNet stats).

2 Likes