I found a very strange problem, I have a trained model net and five test images, if I input the whole five images into the model and separately a single image into the model, I get completely different output, even the number of labels is different (four 1 and one 8 vs one 1 and four 8).
Check if you are using batchnorm layers (or other layers which use a batch size - dependent operation), since the normalization of the input activation would change depending on the batch size.
