@mshmoon I have a similar problem, I think it is because during training in the validation phase the model is not set to model.eval() . So in this problem, you should retrain it in the .eval() mode while validating, and .train() mode while training. Then in the test or inference stage, you just use model.eval() . Now the mysterious one is what is the difference of model.train(True) and model.train(False) do. I have posted it in here but no ones reply it.