The validation set for ImageNet has 50,000 images or 50 per each of the 1,000 classes. If you don’t shuffle the data then the expectation indeed is that you only see two classes for a batch size of 100.
1 Like