I am working on image classifying task. I am using torch.utils.data.DataLoader to load validation dataset.
When I change the parameter shuffle from True to False, I got different result, the sensitivity dropped. (Here, the result means sum of results from all validation data, so it should be the same as long as I am using same validation dataset). I found the validation result changes by the order of input data. Can this happen? What do you think is the cause of this instability?
Order of input data does matter for neural network training.
In general, Classificaion data have data ordered class wise and hence, if you are training without shuffling the dataset, then your full batch contains only one set of examples and network only learns for the particular class and does overfits for the current class. Then sequential learning of second class follows and learning is not tuned to second class.
Note that, Neural network is nothing but an approximation function and hence, more varied example it seems, more it generalizes; hence it is important to shuffle and feed images of different classes and not same class examples in sequence.
1 Like
Are you setting your model to evaluation using model.eval() before feeding in your data?
If that’s the case, could you post a (small) executable code snippet, so that we could have a look?
1 Like
Thank you for the replies.
My train-dataset was shuffled, so the train phase itself should be ok.
The cause of the instability was, as ptrblck indicated, lack of model.eval() and instead I was using torch.no_grad().
Since my model contains batch normalization, it makes sence that inference results were different without model.eval().
Thanks again.