My network use batch norm in each layer. during training and validation the error is very law. Once I move to testing and apply model.eval() the error is very high. knowing that if I’m using the model no in eval() mode result are much better.
I tried to test using the same batch size as the training and testing on one batch size, both of them are better than using average values learned from training by using eval() mode. theoretically I can’t use this way and I can’t justify, Any solution please ?
I re-trained the network and I used random shuffling in each epoch to avoid similar grain. the result are worst. whenever I use model.eval().
at the same time, it is not justifiable to test the model by using one test image and recalculate the parameters, neither to test my images in higher mini-batch size more than one.
I think there is a problem in batch-norm implementation, if someone tested eval mode in batch-norm please comment into my experiment.