Do we do whitening on batch basis or on dataset basis?

2

nn.BatchNorm layers will do exactly this for each batch.
The Standard score on the other hand is usually calculated for the whole train set.