I have the same problem as well. My model is fc->bn->relu->fc->bn->logsoftmax
I have already set model.eval() before test,
If I set (shuffle=true), the accuracy is about Accuracy: 2270/3200 (70.94%)
Testing accuracy: 70.94
If I set (shuffle=false), the accuracy is about Test set: Average loss: 3.3379
Accuracy: 757/3200 (23.66%)
Testing accuracy: 23.66
There is a huge difference