Performance different after loading saved models

Thanks, You are right. The problem was not due to randomness in PyTorch. I had a simple python bug. I was iterating a set and using the order of its items in my code.